1
Vote

Experiment with parallel computation of value + gradient

description

Try using Parallel extensions for .NET to allow developers to compute gradients and function values faster on multi-code machines.

comments

davidacoder wrote Sep 29, 2011 at 5:52 PM

I think this would be fantastic. At the same time it would be good if this could be purely optional. I sometimes have code where I want to parallize at a more coarse level and then it would be inefficient if AutoDiff did always parallelize as well.

alexshtf wrote May 7, 2012 at 11:54 AM

The dependency analysis doesn't seem simple. If you know of a good algorithm to divide the computation graph in advance to parallel computation sequences (during the compilation stage) it might work.
Right now I have no such good idea. Need to do more reading.