What is the difference between the 1st order polynomial fit and linear regression?

For a long time, I always think that the 1st order polynomial fit and linear regression are same, until today I process the attached data.

load xy_ask.mat x y
clf; plot(x,y,'o'); hold on
x0=0:20;
[p,S,mu]=polyfit(x,y,1); plot(x0,polyval(p,x0),'b')
[rr,ar,br]=regression(x,y); plot(x0,ar*x0+br,'g')
legend('data','polyfit','regression'); xlabel('x'); ylabel('y')

Here is what I got. The result from polyfit seems not fitting at all. Does anyone know why?

Answer 1

[Matt J]

They are the same. You are just plotting the polyfit result incorrectly,

load xy_ask.mat x y
clf; plot(x,y,'o'); hold on
x0=0:20;

[p,S,mu]=polyfit(x,y,1); plot(x0,polyval(p,(x0-mu(1))/mu(2)),'mo','MarkerFace','m')

[rr,ar,br]=regression(x,y); plot(x0,ar*x0+br,'g')

legend('data','polyfit','regression'); xlabel('x'); ylabel('y')

[qiang li]

Many thanks, Matt. I would not solve it for days without your answer.

But even I read the help of polyfit for many times, I still cannot understand the difference between the output with only [p] and [p,S,mu]. Are the coefficients of p different with these two different outputs?

[Matt J]

Yes, they are different, as you can check yourself,

load xy_ask.mat x y
[p1,S,mu]=polyfit(x,y,1);
p2=polyfit(x,y,1);

p1,

p1 = 1×2

0.0002 0.0018

p2,

p2 = 1×2

0.0001 0.0014

[qiang li]

You are right. It took me a while to understand the difference. p1 is calculated using the normalized x.

Many thanks, Matt.

[Bruno Luong]

By equation p1(x-mu(1))/mu(2), p1 is related to p2 in this manner.

load xy_ask.mat x y
format long

[p1,S,mu] = polyfit(x,y,1);
p1cs = [p1(1)/mu(2), p1(2)-p1(1)*mu(1)/mu(2)],

p1cs = 1×2

0.000061494457721 0.001378534546526

p2 = polyfit(x,y,1),

p2 = 1×2

0.000061494457721 0.001378534546526

[~,ar,br]=regression(x,y);
pr = [ar, br]

pr = 1×2

0.000061494457721 0.001378534546526

But is is more stable nummerically to do regression and evaluation using normalization mu and p1 instead of the polynomial form that applies directly to x (p2, p1cs)

[Paul]

Just wanted to point out that polyval can take x0 and mu as separate inputs as well.

load xy_ask.mat x y
clf; plot(x,y,'o'); hold on
x0=0:20;

%[p,S,mu]=polyfit(x,y,1); plot(x0,polyval(p,(x0-mu(1))/mu(2)),'mo','MarkerFace','m')
[p,S,mu]=polyfit(x,y,1); plot(x0,polyval(p,x0,[],mu),'mo','MarkerFace','m')


[rr,ar,br]=regression(x,y); plot(x0,ar*x0+br,'g')
legend('data','polyfit','regression'); xlabel('x'); ylabel('y')

[Star Strider]

The problem is not your understanding of linear algebra. It is instead the problem of not understanding how polyfit and polyval work.

When you ask for the ‘S’ structure and ‘mu’ outputs, polyfit centres and scales the fit. It is necessary to provide both these outputs as inputs to polyval in order to re-create the desired fit to the data.

See the third item under Description and the documentation section on Use Centering and Scaling to Improve Numerical Properties to understand how this works.

[Bruno Luong]

"It is necessary to provide both these outputs as inputs to polyval in order to re-create the desired fit to the data."

Actually only mu is required by polyval for correct evauation for the fit data. S only used for second ouput delta (estimate standard deviation)

Answer 2

[Image Analyst]

A linear regression means the equation is linear in the coefficients, not the variable. (Google it.) So this can be a model for a linear regression:

y = a2 * x^2 + a1 * x + a0

None of the "a"s are raised to a power or in any kind of weird formula. All the a's are linear, even though the x are not. So a linear regression is more general than a "1st order polynomial fit".

A "1st order polynomial fit" is where the model is a polynomial where the variable goes only to the first order, in other words just up to x, not any powers of x. So the model for a "1st order polynomial fit" is

y = a1 * x + a0