Checking if two random variables are statistically independent

Hi,

I have this Matlab assignment in which I am supposed to create a vector of five statistically dependent random variables and, among other things, create its covariance matrix. Then I have to check "what would happen if the variables were statistically independent".

I was not told which distribution these variables should be in so I chose Gaussian. Now, from the theoretical material I learned, when I create a covariance matrix for a vector Z which includes a number of statistically independent random variables in a Gaussian distribution, the numbers off the main diagonal need to be 0 (only true for a normal/Gaussian distribution so that's one of the reasons I chose this distribution).

What I did basically is this (I know I'm supposed to have 5, ignore that):

    N=1000;
    Z1 = normrnd(2,2.5,1,N);
    Z2 = normrnd(2.5,4,1,N);
    Z = [Z1; Z2];
    Z_T = transpose(Z);
    Cz = cov(Z_T);

Alas, the numbers that are off the main diagonal are not zero. Instead I get something like:

Cz =

      6.0650    0.2329
      0.2329   16.6527

Searching Google I found this question and answer: http://www.mathworks.com/matlabcentral/answers/117188-testing-for-statistical-independence-of-two-vectors

And tried to use the corrcoef function, what I got is this:

[r,p] = corrcoef(Z_T)

r =

      1.0000   -0.0277
     -0.0277    1.0000

p =

      1.0000    0.3818
      0.3818    1.0000

According to Matlab help, the numbers in the p matrix should be near zero. I assume what I did is right because from searching on the web all I learned is that to create statistically independent random variables I just need to create them, that's all.

To conclude, my questions are:

1. How do I create statistically independent random variables? As you can see, what I did was just use the normrnd function (randn can be used as well). Is that enough?

2. How can I check if they are statistically independent? Like I said, I used the corrcoef function but it didn't come through on the p matrix (or did it?)

3. How is it that the numbers in the off diagonal in the covariance matrix are not zero (if the variables are indeed statistically independent)?

Thank you very much and sorry for the long question!

[Chad Greene]

You may have better luck getting answers to your questions if you can ask them individually. It's hard to tackle so many questions at once. Regarding values in the p matrix, small data sets have a higher chance of being correlated by chance. If you were comparing very large data sets of the same correlation strength, values in p would be smaller, because there's less of a chance that they're accidentally correlated.

[Eran]

Thanks Chad. About the question, it may seem long but actually I just wanted to show everything I have done so you can have all the information. I'm actually asking maybe 3 questions, but I agree it does seem like a lot. About what you said - by data sets, do you mean the number N in the code? I should have mentioned, N = 1000. Or do you mean that I should have Z3,Z4,Z5 as well so they won't be correlated?

[Eran]

Anyone?

[Chad Greene]

Sorry Eran, I am not much of a stats guru, so much of your question is outside my area of expertise. Again, this question is long and multifaceted. It's good to have details, but readers may be overwhelmed by the different directions your question goes in. When you finally get around to asking the question you say, "So my question is - is that true?" As a reader I wonder what you mean by that. Does that mean everything you wrote above? The roundabout wording of your question is tough to respond to.

Although you've done a rare (and very good!) thing of providing lots of information, your question(s?) comes across as a bit unfocused, so it's hard to understand, and harder to understand what kind of answer you're looking for. I recommend condensing the information to only what is relevant, and if you have multiple questions, open a thread for each question. You'll end up providing some redundant information across the threads you open, but that's okay. Good luck!

[Eran]

Thanks, I edited my post so I think it should be clear now, so I would love your opinion. I understand what you are saying with opening a thread for each question, but since they are all related, I don't think it's the best idea.

Answer 1

[Soumojit Bose]

Erin, Alas, there is no short-cut (or a code snippet in MATLAB) that can show that two random vectors are statistically independent. You were taught right in class, that, two random variables, whose Joint PDF is Gaussian, if they are uncorrelated, they are statistically independent. This does not apply to the general case, where Joint PDFs may not be Gaussian. In that scenario, the two variables are independent iff their joint expectation equals the product of their individual expectations. In other words, if you can prove that their joint PDF is separable, then they are independent.

[LEALEM BISHAW]

how can you code or show in matlab "the two variables are independent iff their joint expectation equals the product of their individual expectations"? this your statement

thanks

Answer 2

[Peter Perkins]

Erin, consider the following analogy: generate 10 random values form a standard normal distribution.

>> x = randn(10,1);
>> mean(x)
ans =
     -0.20861

The standard normal has mean zero. So why isn't mean(x) zero? Your sample correlation matrix r doesn't have exact zeros on the off-diagonals for more or less the same reason. Once you get comfortable with that, you'll have understood the difference between a sample statistic and a distribution parameter - one of the two is a random variable.

Also, I think you are misreading the documentation for the seond output of corrcoef -- it's a p-value. Values near zero indicate that the corresponding correlation is statistically significantly different from zero.

[Eran]

Hi Peter,

Thanks for the answer but I don't really understand it. In my course, random processes, I have learned that if you have a correlation matrix of a vector x, then: 1. If cov(x1,x2) = 0 then x1,x2 are uncorrelated. 2. x1,x2 uncorrelated != statistically independent unless they are Gaussian.

That's why I don't really understand why I don't get zeros in the correlation matrix.

About the documentation, yes, I guess I didn't quite understand it, English is not my native language and it's not easy understanding the "mathematical language" in a different language.

But, since I have to submit the assignment today, lets just leave it at that. Like I said, in the assignment I need to check "what would happen if the variables were statistically independent" . What can I show the professor in the assignment that will answer this question? Note - I don't need to attach code.

Thanks.