Polyfit not working for line of best fit

Hey evertone,

I have multiple set of results through which I want to draw a line of best fit and take the slope and intersect with the y-axis from this line. There are 3 samples (n_sample). Every sample consists of 3 rounds (n_round) and every round consists of 5 data point. (45 points in total.) The x-values and y-values of the points are in a matrix of 3x3x5, sigma_n and tau_n respectively. I wanted to exclude the values (1,3,5), (2,3,5) and (3,1,5), so I made the following code:

for i = 1:n_sample
  for n = 1:n_rounds
      if i<3 && n==3
          f               = polyfit(sigma_n(i,n,1:end-1), tau_n(i,n,1:end-1), 1);
          line_bf(i,n,:)  = 0;                                                                          %Fill the original matrix with 0, so that no dimension conflict occures                                                     
          line_bf_re(i,:) = polyval(f,sigma_n(i,n,1:end-1));                                            %Make a alternative matrix with 4 wide rows instead of 5
          bf_A(i,n)       = (line_bf_re(i,end)-line_bf_re(i,n,1))/(sigma_n(i,n,end-1)-sigma_n(i,n,1));  %Get the A in the line of best fit equation Ax+B.
          bf_B(i,n)       = line_bf_re(i,n,1) - (bf_A(i,n)*sigma_n(i,n,1));                             %Get the B in the line of best fit equation Ax+B.
      else
          if i==3 && n==1
              f               = polyfit(sigma_n(i,n,1:(end-1)), tau_n(i,n,1:(end-1)), 1);
              line_bf(i,n,:)  = 0;
              line_bf_re(i,:) = polyval(f,sigma_n(i,n,1:(end-1)));
              bf_A(i,n)       = (line_bf_re(i,end)-line_bf_re(i,n,1))/(sigma_n(i,n,end-1)-sigma_n(i,n,1));  
              bf_B(i,n)       = line_bf_re(i,n,1) - (bf_A(i,n)*sigma_n(i,n,1)); 
          else
              f               = polyfit(sigma_n(i,n,:), tau_n(i,n,:), 1);
              line_bf(i,n,:)  = polyval(f,sigma_n(i,n,:));
              bf_A(i,n)       = (line_bf(i,n,end)-line_bf(i,n,1))/(sigma_n(i,n,end)-sigma_n(i,n,1)); 
              bf_B(i,n)       = line_bf(i,n,1) - (bf_A(i,n)*sigma_n(i,n,1));
          end
      end
  end

Everything works well, except that bf_A(1,3,:) and bf_B(1,3,:) are not exactly right. I know the problem is the polyfit in the first if-loop, because it gives values that are pretty different from the other values. The weird thing is that for i==3 everything works well and it is the exact same code.

Do you maybe know what went wrong and how i can get the right line of best fit? The graphs can be found in the attached pictures.

Thanks a lot in advance! I know it is a bit complicated and long, but I appriciate the help!

[John D'Errico]

Yes, polyfit does work. It does exactly what it should do. Unless of course, you pass it the wrong data. Then it will still do exactly as designed.

We don't have your data, so how can we possibly know what is happening? The only way to do so is to step through your code. But we can't do so. On the other hand, YOU can do that, and do so trivially. Look at what points you are using. Look at the specific fit you did. That is all I would do.

Look carefully at what your code does. Check every line. Verify that it did what you thought it should do.

[sven heilbron]

Thank you for your response. The data points can be seen in the pictures. The colors correspont to the the round, which can be found in the legend of the figure.

If we take i=1 and n=3, Polyfit should cover sigma_n(1,3,1:end-1) and tau_n(1,3,1:end-1). These are the first 4 green points in figure 1. However, you can see that the line here is far above these points and thus the line of best fit is wrong, both the slope and the point on x=0.

I could also show all the data, but I dont think it will help very much. I narrowed the problem down to the point that I know that polyfit is the error, but I dont know what there is wrong with it.

Also, when I take the polyfit of all the points, (so not excluding the last point) the polyfit works fine.

[John D'Errico]

Sigh. My point is, you THINK you know what points polyfit is using for the fit. You look at the pretty pictures you generated, and believe them. So lets look at the facts. Only one fact is needed.

Fact: Polyfit is generating a line that does not pass near the points you THINK you have passed it.

The conclusion is simple. Polyfit is using a different set of points from what you have plotted. There is no alternative.

You tell us to just look at the plot. But we have not been given the code that created the plot. We don't have access to the data you are using.

Answer 1

[John D'Errico]

You need to learn to use the debugger, since you are unwilling to give us sufficient information to debug your code. Step through the code, one line at a time. Even better, I'd just inset a call to the function keyboard right at the point where you know the problem happens.

help keyboard

That will stop the function and allow you to inspect your data, and what is being passed into polyfit.

LOOK CAREFULLY at those points. Plot them if necessary. See what the coefficients of polyfit that are generated. Are they consistent with the points that were passed into polyfit?

As an alternative to keyboard, you can just use the editor to set a debugging breakpoint at the line in question.

This is trivial problem to solve, IF you are willing to use the debugging tools available.