What would cause a histogram to have regular spikes in bin counts?

I tried to use histc to plot a histogram, but the resulting histogram has noticeable,regular spikes when the data should fit a smooth gaussian distribution. I am doubtful that these spikes are real, and instead something to do with the calculation in Matlab. Could there be systematic rounding of the data points that so happen to be the major axis tick marks? How would these spikes even occur?

[Rik]

It is impossible to solve this if you don't show some code. It is not possible that the xticks affect the inner workings of histc. A rounding error is not impossible, though unlikely. Also, while the first peak seems to be close to 0.1, the second, third and fourth are not close to their respective ticks.

Your data seems to be a bit noisy anyway. Why is it so unlikely that these peaks are a calculation byproduct?

[Joyce W]

I have a large list of data, N (a nx1 array), then plot

binranges = [0:binsize:max];
bincounts = histc(N,binranges);

I did mean that the problem is from something to do with how I did it in Matlab, sorry if the way I wrote it was unclear. And in addition, the location where the spike appears changes depending on the bin size. By increasing it by half, the original spikes at 0.1 and 0.2 are still there, but more semi-regular spikes appear. That's why I think it is due to the data processing rather than the data being really like that.

[the cyclist]

Can you post the data, or a sample that reproduces the problem?

[Joyce W]

The data:

[Image Analyst]

I think I explained it already below, but perhaps an example would help. For example let's say you had values of 0.5, 1.5, and 2.5. If you had bins of width 1 (0-1, 1-2, and 2-3) you'd have 3 bins with 1 count each. Now if you created fewer bins, like a width of 2 (0-2, and 2-4), the first bin would have a count of 2 (would capture values of 0.5 and 1.5) while the second bin would still have a count of 1 (only counted the value of 2.5). Hence you'd see a spike on the first bin.

[Jan]

 By increasing it by half, the original spikes at 0.1 and 0.2 are
 still there

By increasing what?

[Joyce W]

I meant increasing the bin size 2x, sorry.

Answer 1

[the cyclist]

Your code and your file don't quite match up with each other (e.g. variable named "displacement" rather than "N") and also you have not defined some of the variable you used (e.g. binsize).

Nonetheless, I was able to replicate something close to what you are seeing using your data file and this code:

load data

binsize = 0.001;
maxbin = 0.5;
binranges = 0:binsize:maxbin;

bincounts = histc(displacement,binranges);

figure
plot(binranges,bincounts)

If I just let MATLAB make 100 bins, I get a relatively smooth curve (although you can see some hints of spiky behavior):

But if I increase the number of bins to 800, it gets very spiky like your original:

So, I would conclude that this is definitely a result of how you are enforcing the binning.

[the cyclist]

Addendum --

I think it's actually a little easier to see what is going on if you actually use a histogram (rather than a line plot of bin counts):

figure
histogram(displacement,400)
xlim([0 0.5])

and then playing around with the number of bins in the range 100-800 or so. Your data are a little clumpy.

[Joyce W]

the-cyclist: the graphs you show look very similar to the ones I get from changing the bin size.

From everyone's replies, it seems like changing the bin size can change how evenly the data points are distributed per bin, and it becomes very obvious when there are few data points relative to the number of bins, like the example Image Analyst gave with 3 data points. However I have a larger dataset with 39000+ data points, and it looks like with a large dataset, having larger bin sizes can mask clumping of data that deviates from the overall gaussian distribution. Would you say that the spiking that appears at smaller bin sizes is the real behavior of the data then, not a coding artifact?

[the cyclist]

The data you uploaded are definitely clumped. Try this code, where I have "jittered" the data a little bit, and zoomed in.

figure
plot(sort(displacement)+0.00005*randn(size(displacement)),rand(size(displacement)),'.')
xlim([0.095 0.125])
ylim([-0.5 1.5])

You can see structure at spacing of about 0.001, and the density of points around 0.015. There is also a decreased density at around 0.018, but that is not as easy for the eye to perceive in this chart. The best way to perceive that is -- the histogram!

[Jan]

The historgram looks, like this is not simply a noisy data set, but it looks like 2 different overlapping Poisson distributions. A bold speculation: This is not noise, but a signal. Depending on what you are looking for it might be worth to check or reject this hypothesis.

Answer 2

[Image Analyst]

This can be caused by compressing the dynamic range, for example by taking an image with 256 gray levels and remapping the pixel values so that they only range from 0 to 230 or something.

It can also be caused by having your histogram bins not exactly match up with the values. For example if you had 230 bins for data that ranged from 0-255 and would look "proper" with 256 bins. But with fewer bins than needed, some bins get more pixels than they otherwise would have.

[Jan]

If 256 bins are resampled to 230 bins with a linear mapping, I'd expect the spikes to be positive only and to have a equal distances, because they are caused by rounding.