How can I use fminunc or fminsearch to minimize and f-divergence measure?

I am trying to solve a non-linear optimization problem but I see that fminunc and fminsearch take scaler inputs and give scaler outputs.

I'd like to use non-linear optimization functions to compute distance measure between two curves as follows :

KL(t, X) = sum(input(t) * (log(input(t)) - output(t, X))

I would like to determine X such that the entire function is minimized where X is a vector.

[Shalin Shah]

Also, how can I use fmincon to specify condition that KL(t, X) >= 0?

Answer 1

[Alan Weiss]

I don't know why you think that fminunc and fminsearch take scalar inputs, when their documentation clearly states that they take vector (or even array) inputs. They do insist on objective functions that return real scalar outputs, but that is different.

You are free to minimize your function using either of these minimizers. fminunc would almost certainly be faster and more reliable.

To include your data t in your objective function, see Passing Extra Parameters.

Alan Weiss

MATLAB mathematical toolbox documentation

[Shalin Shah]

Right, so when I try writing this base case code it says output should be scalar.

%% objective Function File

function f = objFun(lambda)
%UNTITLED Summary of this function goes here
%   Detailed explanation goes here
    t = 1 : 0.1 : 20; 
    inputPDF = exppdf(t, 2);
    f = inputPDF .* (log(inputPDF) - log(exppdf(t, lambda)));
end

%% main File

options = optimoptions(@fminunc,'Algorithm','quasi-newton');
initLambda = 100;
[x, fval, exitflag, output] = fminunc(@objFun, initLambda, options);

[Alan Weiss]

Perhaps you are confused about what a scalar objective function is. Your objFun returns a vector. An objective function needs to be a scalar. The reason for this is that only scalars are ordered, meaning you cannot say whether one vector is greater than another. Is [1,2] larger or smaller than [2,1]? The question makes no sense.

So if you want to find a minimum of something, that something has to be a scalar, not a vector or array.

Alan Weiss

MATLAB mathematical toolbox documentation