Is it possible to use fmincon with parallel computing toolbox when option 'GradObj' is set to 'on'?

Hi all,

I'm trying to use fmincon in parallel mode using parallel computing toolbox. My objective function provides gradient calculations to fmincon so there is no need to do numerical gradient calculations. Therefore, 'GradObj' is set to 'on'. I provide the gradient using adjoint technique, so I perform a run of the objective function, and take the results to calculate the gradients. The function evaluated by fmincon is simply:

%============================= function [f,g] = computefun(y)

% The forward function: x1 = myfun(y)

x2 = objfun(x1); f = -x2;

% The adjoints: g = adj(x1); %==============================

My code runs in parallel when I test it with parfor or with fmincon and 'GradObj' is set to 'off' so I know it works. I came across the following mathworks link which says that the benifit of parallelism is to efficiently compute the numerical gradients only: http://www.mathworks.com/help/optim/ug/parallel-computing-in-optimization-toolbox-functions.html

So, I wonder if there is any solution or workaround to allow PCT to work with fmincon when 'GradObj' is set to 'on'?

Thanks, Zeid

Answer 1

[Matt J]

It depends what aspect of the optimization you can/want to parallelize. You can still use parfor loops and so forth inside the code for your objective and gradient if the calculations there can be broken down into parallel tasks.

[Matt J]

Zeid commented:

Could you please elaborate? I use fmincon to find an optimum value of y. So, fmincon is looping to find the optimum value. Using parfor will loop through already assigned value given by the user.

[Matt J]

I don't know what you mean by "already assigned value given by the user". FMINCON calls your objective function repeatedly so it is worthwhile to optimize the speed of each individual execution of objfun(). If the operations inside objfun() have a parallel structure, you can use PARFOR to parallelize them just as you would parallelize any other piece of code.

[Zeid]

within objfun(), I don't use computationaly intensive forloop. My main intensive calculation is matrix inversion which is already implicitely parallized by matlab. My question is, can fmincon compute multiple instances of objfun() at the same time "in parallel" when gradient is provided by user? From my readings, fmincon calculates instances of objfun() at the same time to compute the gradients numerically but not to calculate the function value 'f'.

[Matt J]

Your reading is consistent with my understanding of how fmincon works as well. There is no need for FMINCON to make multiple parallel calls to objfun() when you supply an analytical gradient because in that case only 1 call is needed.

I wouldn't recommend turning GradObj off, though. If you have an efficient way of computing the analytical gradient, it doesn't sound like it would be more efficient to have FMINCON attempt a harder finite difference approach just because that approach is parallelizable.

If you're looking for even greater speed, maybe you could expose your objfun to public view in case we can spot some further performance optimization possibilities.

Answer 2

[Zeid]

Could you please elaborate? I use fmincon to find an optimum value of y. So, fmincon is looping to find the optimum value. Using parfor will loop through already assigned value given by the user.

[Matt J]

I assume this was addressed to my Answer, so I've copied this as a comment to my Answer above.