How to minimize a sum of squares?

Hi,

How can I find v that minimizes the sum of squares of the function below?

f=wts-(rac^(-1)*cov^(-1)*(eq+cov*p*(p'*cov*p+v)^(-1)*(i-p'*eq)))

subject to v>=0.

v, rac, and i are integers.

wts, eq, and p are nx1 vectors.

cov is a nxn matrix.

Thank you.

Answer 1

[Walter Roberson]

See http://www.mathworks.com/help/curvefit/least-squares-fitting.html (Curve Fitting Toolbox) and http://www.mathworks.com/help/optim/ug/least-squares-model-fitting-algorithms.html (Optimization Toolbox)

Answer 2

[Roger Stafford]

If one temporarily ignores the constraint that v be a non-negative integer, this is a linear least squares problem in disguise and therefore has an easy answer. No special toolboxes are required. If you make the following definitions

 a = wts-rac^(-1)*cov^(-1)*eq
 b = rac^(-1)*cov^(-1)*cov*p*(i-p'*eq)
 x = (p'*cov*p+v)^(-1)

then a and b are n x 1 vectors and x is a scalar and your equation

 f = wts-(rac^(-1)*cov^(-1)*(eq+cov*p*(p'*cov*p+v)^(-1)*(i-p'*eq)))

is exactly equivalent to

 f = a-b*x

Hence

 sum(f.^2) = sum((a-b*x).^2)

is easily seen to have a minimum at

 x0 = sum(a.*b)/sum(b.^2)

The corresponding value of v without constraints would then be the scalar

 v0 = 1/x0 - (p'*cov*p)

Now, returning to the constraint on v to be a non-negative integer, since f is a parabolic function of x which necessarily descends for x less than x0 and ascends for x greater than x0, the value of v for which a minimum is achieved has to be one of three discrete possibilities, ceil(v0), floor(v0), or zero. It is easy to determine which yields the minimum by evaluating sum(f.^2) at each of these values of v.

(Corrected)

[Roger Stafford]

A modification to the above statement is needed to make it completely accurate. Four values of sum(f.^2), not three, need to be compared to cover all situations: sum(f.^2) at v equal to ceil(v0), sum(f.^2) at v equal to floor(v0), sum(f.^2) at v equal to zero, and sum(f.^2) itself equal to sum(a.^2). If this last value of sum(f.^2) were the smallest of the four, there could be no finite solution, since it corresponds to v equal to plus infinity, which is to say, to x equal to zero. In such a case, as v approaches plus infinity, sum(f.^2) would approach that minimum, but no finite value of v could ever achieve it.

(Corrected)

[Alexandre]

Hi Roger, thank you very much for your answer. It works!