CodePlexProject Hosting for Open Source Software

Although the main objective of the library is to save the developer the tedious work of computing gradients, one of its secondary objectives is performance. The library allows achieving high performance in scenarios where the user needs to compute the gradient or the value of the same function many times at different points. A good example for such scenarios is optimization. An optimization algorithm needs to repeatedly compute the gradient and value of a function at many candidate points, until convergence.

The following code demonstrate term compilation:

Varaible x = new Variable(); Variable y = new Variable(); Term term = x + y * TermBuilder.Exp(x + y); ICompiledTerm compiledTerm = term.Compile(x, y);

Now that we have our super-ninja-fast compiled term we can start computing gradients and values. Computing a value is simple. Use the ICompiledTerm.Evaluate method. Here is an example:

// compute the value at x = 2, y = -3 double value = compiledTerm.Evaluate(2, -3);

// compute value + gradient at x = 3, y = -5 Tuple<double[], double> diff = compiledTerm.Differentiate(3, -5); double[] gradient = diff.Item1; double value = diff.Item2;

As stated before, there is a good reason to use the slightly more complicated compiled terms instead of simply invoking evaluation and differentiation methods on our terms. The reason is performance. You can see a real-world usage in the next two tutorials. However here is a small example that demonstrates the gains:

Random random = new Random(); for(int i = 0; i < 10000; ++i) { double a = random.NextDouble(); double b = random.NextDouble(); Tuple<double[], double> diff = compiledTerm.Differentiate(a, b); }

Random random = new Random(); Variable[] variables = {x, y}; double[] values = new double[2]; for(int i = 0; i < 10000; ++i) { values[0] = random.NextDouble(); values[1] = random.NextDouble(); double[] gradient = term.Differentiate(variables, values); }

- AutoDiff allows compiling a term to create a more efficient internal representation that allows high performance
- It is beneficial to compile terms when the same function needs to be repeatedly evaluated or differentiated
- The Compile extension method compiles terms. It is given the term's variables, and the order does matter.
- The methods ICompileTerm.Evaluate and ICompiledTerm.Differentiate are used to evaluate and differentiate our compiled term at different points.

Last edited Apr 6, 2011 at 11:32 AM by alexshtf, version 2

typo: Varaible x = new Variable();

correctly: Variable x = new Variable();

correctly: Variable x = new Variable();