How to use lsqnonlin with jacobian?

I'm using the Optimization toolbox in order to solve a LS problem. I was using the fminunc solver, that gives me good results when I supply externally the gradient.

Recently I found the lsqnonlin solver, that seems to be more suitable for the problem I'm trying to solve.

However, when I use this solver with the pre-computer gradient, it gives me the same results as I'm not supplying it. In fact, even setting or not the "Jacobian" option to "on" or "off", I obtain always the same results that, even more, are exactly the same that I obtained with the fminunc solver when the "GradObj" option was set to "off".

This is the code (I'm omitting details about the fun):

options = optimoptions('lsqnonlin','Algorithm', 'levenberg-marquardt', 'Display','none','FinDiffType', 'central','Jacobian','on','tolFun',1e-20,'tolX',1e-20);

x = lsqnonlin(fun,k0,[],[],options);

Am I currently missing some options in this few lines or does it can depend on the function itself to which I'm applying the solver?

Answer 1

[Star Strider]

I am not certain what you are asking. The gradient-descent least squares solvers calculate a numeric Jacobian (using the finite-difference method) if you do not provide an analytical Jacobian function to them.

[Savio Sciancalepore]

First of all, thanks for your answer.

What I'm asking is why lsqnonlin does not use the jacobian that I'm supplying externally, even if I'm setting the "Jacobian" option to "on".

Am I missing some options that will further enable its use or can be something related to the Jacobian itself (for example, values are very close to 0).

[Star Strider]

My pleasure.

Since you didn’t post your ‘fun’ code, I can’t determine if it also returns the Jacobian as its second argument, as the lsqnonlin function requires. See the lsqnonlin function documentation under Input Arguments for fun.

The way I read the documentation, if you turn 'Jacobian','on' and provide a Jacobian in ‘fun’, it should use your function.

That you’re getting the same results with your user-supplied Jacobian as you do with the finite-difference Jacobian should be no surprise. The only difference is likely speed — the function using your analytic Jacobian should be faster.

[Savio Sciancalepore]

The suggestion that you gave me, i.e., looking at times, was very useful. In fact, if I observe the number of iterations performed by lsqnonlin, I obtain always 1 single iteration, even if I'm using or not the jacobian.

Rescaling the problem by a factor of 1e8 (just to let you understand, I divide by the light speed, that is 3e8, so the scale of the problem is very low), using the jacobian I obtain results that are very closed to the real solution, and I can concretely observe a variation with respect to the case in which I don't use the jacobian set by myself.

Therefore, I'm concluding that lsqnonlin does not use the user-supplied jacobian if values obtained are very close to 0.

How to solve this problem? It would be interesting to compare results provided by fminunc and lsqnonlin with the real scale of the problem.

[Star Strider]

The only way to know if it is using your analytic Jacobian is to insert a fprintf call in its code to tell you:

fprintf('Analytic Jacobian called\n')

However, it should use your analytic Jacobian if you tell it to.