Adding regulation (L1) to fminunc

Hey!

I am trying to use fminunc for a nonlinear optimization and wish to add an L1 Tihkinov regulation. particularly, I have a real image and dataset, corresponding to the image. I have an algorithm to reconstruct an image out of the data set. The dataset is filtered and using triangulation reconstruct the image. I wish to find the best matching filter to minimize the error between the real image and the reconstruction. In addition I wish to add a regulation factor of the sum of magnitudes of the filter.

What is the best way to do so? Should I only add it to the cost at the objective function? Should I use another solver-based approach?

Thank you in advance,

Omer

Answer 1

[Matt J]

fminunc and fmincon require differentiable cost functions and constraints. The original problem is not differentiable because the L1-norm is not differentiable,

However, you could try reformulating as the following equivalent linearly constrained differentiable probelm,

to which you would apply fmincon. If you take this approach, I would probably derive the initial guess x0 from an unconstrained L2-regularized or Huber-regularized optimization, as a simpler approximation to L1.

You could also try ga() which doesn't require differentiability, but I have no idea what kind of computational burden that would bring, nor how fast you might see it converge.

[Omer Hamburger]

Hey Matt, thank you for your answer. How should I use the L2 regularized approach as part of the objective function?

As I see the things - providing the gradient and hessian matrix of my function f(x) is hard, so how can I avoid this, yet regulating the result by the norm (L2) of the filter?

Thank you , Omer

[Matt J]

You don't have to provide the gradient and Hessian if you don't want to - fminunc will use a finite difference approximation if you don't provide them. But this could be very expensive computationally, depending on the size of your problem.