### Highlights from Mutual Information computation

4.70968

4.7 | 32 ratings Rate this file 376 Downloads (last 30 days) File Size: 297 KB File ID: #14888

# Mutual Information computation

06 May 2007 (Updated 23 Aug 2007)

A self-contained package for computing mutual information, joint/conditional probability, entropy

File Information
Description

A self-contained, cross-platform, package for computing mutual information, joint/conditional probability, entropy, and more. This package has also been used for general machine learning and data mining purposes such as feature selection, Bayesian network construction, signal processing, etc.

Another related package for minimal redundancy feature selection is also available at the Matlab Central exchange site, under the category of "Biotech and Pharmaceutical".

A simple demo is called demo_mi.m.

*** Please note that downloading or use of this package means acceptance of the license of this package. In short, this package is free to non-profit use but cannot be re-distributed in any form, including revised forms, without the explicit permission for the author, Hanchuan Peng. See readme file for further information. ***

MATLAB release MATLAB 7 (R14)
02 Dec 2013
30 Nov 2013

Is there any assumption made when calculating mutualinfo(x, y) if one of vector x,y is binary?
In my data, y is binary and I tried several different x. Surprisingly, several different x give the same mutualinfo to the last digit of double type.
I think some assumption must be made on estimating kernel density. Could author give some insight?

10 Jun 2013

good

03 May 2013

btw, I'm running Matlab 64-bit with linux, which process limit is 8 TB

05 Mar 2013

Maybe I'm missing something, but I think this code produces weird behavior with vectors of low values, e.g. returning entropy of 0 for [1:10] * .0001.

Run this code and see for yourself:
a = [];
for i = 1:20e4
a(i) = entropy([1:10] .* i/10e4);
end
plot(a)
set(gca,'xticklabel',[0:.2:2])
ylabel('Est. Entropy')
xlabel('Scaling factor')

08 Jan 2013

Thanks you all. For win 7 64-bit version, the contents of all .cpp files should be changed from log(2) to log(2.0) if log(2) is used.

03 Jan 2013

ESTPAB problem.

The mex dlls compiled I believe are 32bit. If you have a 64bit MATLAB application you will not be able to run this software.

To solve this

1) Run makeosmex.m found in the source directory "mi".

2) This failed for me as the log functions in the c++ files needed to be adjusted.

3) For each failure, find the line in the .cpp file where the mex compiler falls over (this will be output to the matlab command window) and replaced log(2) with log(2.0)

Best of luck.

13 Dec 2012

10 Dec 2012

I have some problem for using this code.When I run the program, I have the error "Invalid mex-file: the ..........estentropy.dll file isn't a valid win64 application".
Could somebody explain to me what is the problem?
I am using the matlab version (R2010) on a windows machine.
Thank you very much!

16 Jul 2012
20 Jun 2012

Thank you

05 Jun 2012

function 'mutualinfo' can not find function 'estpab'

where is the problem?

06 May 2012

I wanted to try say,
how can I change the time delay of mutual function ??

THanks...

03 May 2012

Hi everyone,
How can ı add decay function or if it already in the program, how can ı can chanhe its valuse.?
if someone can help me, he will save my life :)
Thanks..

15 Apr 2012

Thank you

23 Mar 2012

Does anyone have a recommendation on how to compute entropy conditioned on multiple variables e.g. H(X|Y,Z,W)?

Thanks!

20 Mar 2012

I want to say thank you so much. You saved my day.

01 Feb 2012

For everyone with error C2668 on Windows:

This can be fixed with a simple typecast of the kind:

log(2) -> log(double(2))

Just replace your source code accordingly.

13 Jan 2012

I'm working in Matlab. One file, estpab.m, is missing. Anybody knows what to do?
Thanks

01 Dec 2011

thank u

04 Nov 2011

Hi FEUP, As written above, follow the instructions. What OS do u use? Windows, Linux..etc.? You need to install the compatible compiler (if u don't have one)to make OS compatible mex files
-> Also the Rows in the Data [MXN] and the Rows in the class vector [MX1] should be same.

19 Oct 2011

Hi! Thanks for this work.
The program works well, but I can't find m-file "estpab" that is use in the m-file "mutualinfo". Do you help me?
(Sorry for my english. :))

11 Oct 2011

Ok it works now for me. I will tell how it worked. Firstly download and install the compatible compilers (1. Microsoft visual 2.Microsoft Windows SDK - for windows 64 bit) according to your Matlab version. Once it is installed on your machine. Type mex -setup in Matlab. Choose the installed version and confirm by typing y. Now make sure you have all cpp extension and header files.
Next work is to edit the cpp extension files. Open the files one by one and where you find log() type double like this: log(double()) and save. Now go in the directory where you have the m file makeosmex and run it. After this add all the files and folders to the current working directory. Run the mRMR MID/MIQ/BASE files and et voila its done !
If any body has more questions you are free to ask

04 Sep 2011

Sorry,

I believe entropy(rand(10000,1)) should be high, cause here we have much uncertainty of values of variable.﻿

﻿entropy(sin(rand(10000,1))) should definitely ﻿equal ﻿entropy(﻿cos(rand)), and, I suppose, be small, cause we have less information than with pure random signal (albeit also random)?

04 Sep 2011

Good day! thanks for interesting program. Can anyone please explain me why, having entropy(rand(10000,1))=0 (it's ok),
it gives entropy(sin(rand(10000,1)))=0.9978, BUT entropy(cos(rand(10000,1)))=0
???
:-)
I believe entropy(sin(rand(10000,1))) should also equal 0... or not? with cos(rand) or sin(rand) signals we definitely have less information than with pure rand signal?

13 Jun 2011

I am getting following errors. I tried to recompile it but still getting these errors.

??? Error using ==> estpab
The two vectors/images should have the same length.

Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> mrmr at 50
t(i) = mutualinfo(d(:,i), f);

12 Jun 2011

Although this is a fast routine, the Mutual Information value for small length vector does not seem to be very reliable when using the same vector as X and Y.
Trying the following small test code ...

clc
clear all
close all

nN = 5000;

fMIacc = zeros(1, nN);

figure
hold on
for i = 1:nN
fVec = rand(1,i);
fMI = mutualinfo(fVec,fVec);
fMIacc(1, i) = fMI;
end
plot(fMIacc)
axis([-100 nN+1 0 1.5])

02 May 2011

I solved my problem by recompile everything.

02 May 2011

Actually I have exactly the same problem as Jochen Kumm cause I'm using Mac. I don't know how to fix it.

01 Mar 2011

I am running matlab on mac osx 10.4 (matlab 7.7.0 R2008b) and also on the unix server (matlab 7.11.0 R2010b), would you please tell me how I can compile the files on both of these machines.

I could run makeosmex.m on the unix machine without any error but when I tried to run demo_mi.m, it gave me the following error (I actually replaced log(2) with log(2.0) in estmutualinfo.cpp as well):

??? Undefined function or method 'estpab' for input arguments of
type 'double'.

Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

Error in ==> demo_mi at 25
mutualinfo(a,b)

On mac osx I even could not run makeosmex.m, it gave me the following error:

building mex(dll) of estcondentropy.cpp
mex(list(i).name);

mex: compile of ' "estcondentropy.cpp"' failed.

??? Error using ==> mex at 213
Unable to complete successfully.

Error in ==> makeosmex at 25
mex(list(i).name);

I would really appreciate it if you could help me compile the files and execute mutualinfo.m, it is my first time compiling a C source code on matlab and I have no idea how it works. I would be grateful if you could guide me step by step. Thank you for your time.

14 Jan 2011

Hello, I do not understand the parameter d and f clearly. What will be the value of d? is it individual band for a hyperspectral image, and what will f then? Please inform me.Its better if one can send me with some example data set.

Thanks
Ali

14 Jan 2011

Hello, I o not understand the parameter d and f clearly. What will be the value of d? is it individual band for a hyperspectral image, and what will f then? Please inform me.

Thanks
Ali

19 Dec 2010

I have the same problem as Benjamin Mack.
How can I fix it?

17 Dec 2010

Tried mutual information computation code. Works well!

28 Oct 2010
24 Sep 2010

I was using mutualinfo() and found the following errors

pure virtual method called
terminate called without an active exception
mutualinfo:778(<D,M> <D,C>):(46912990672832, 2433) -> (<T,T>)

Any hint? Thanks!

17 Aug 2010

Hi,
I re-compiled the cpp files with mex to match te current lib versions, but when I compute a mutual information, I always get 0. Did anybody encounter this problem? (I use Linux)

Thanks.

31 Mar 2010

Hi, Marie, have you compiled the estentropy.cpp on your computer?

29 Mar 2010

When I run the program, I have the error "Invalid mex-file: the ..........estentropy.dll file isn't a valid win32 application".
Could somebody explain to me what is the problem?
I am using the matlab version (R2009) on a windows machine.
Thank you very much!

18 Mar 2010

I find the problem.
define "pabhei" "pabwid" with "long" is ok.
about line 32-34 in "estcondentropy.cpp" .

Is it right?

18 Mar 2010

Hi, Prof Peng,

I still met one problem when I compile "estcondentropy.cpp" by run "mex estcondentropy.cpp" after I changed log(2) to log(2.0).

The other files can be compiled successfully. Can you help me for compiling "estcondentropy.cpp"?

Thank you!
Simbest Wang

16 Mar 2010

thank u for the toolbox...

28 Feb 2010

I'm sure the log errors are solved by now. It is obvious that you need to add a .0 to the log(2) to solve the problem if you get that error. (change log(2) to log(2.0)).

25 Feb 2010

Hello,
I try to use this programm but I am totally new to matlab (but have experience in R). When I call the function
mrmr_mid_d (variables, response, 25)
the following error occurs:
??? Undefined function or method 'mrmr_mid_d' for input arguments of type 'double'.
The same occures when I convert my data in int8 or single.

something similar happens when I try to use the demo_mi.m :

mutualinfo(a,b)
??? Undefined function or method 'estpab' for input arguments of type 'double'.
Error in ==> mutualinfo at 21
[p12, p1, p2] = estpab(vec1,vec2);

I think my problem is very a very basic one because I do not know anything about Matlab.
Could anybody please help me. It is very important for me to use this programm and the online version does not work with my data set because it is too large.

Thank you very much in advance,
Ben

02 Feb 2010

I am trying to compile mex files on Unix with Matlab 7.7.0(2008b) using makeosmex.m. I get the following error messages:
building mex(dll) of estcondentropy.cpp
estcondentropy.cpp:19:23: error: miinclude.h: No such file or directory
estcondentropy.cpp:21: error: ?mxArray? has not been declared
estcondentropy.cpp:21: error: expected ?,? or ?...? before ?*? token
estcondentropy.cpp:21: error: ISO C++ forbids declaration of ?mxArray? with no type
estcondentropy.cpp: In function ?void mexFunction(int, int**, int, int)?:
estcondentropy.cpp:24: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:26: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:32: error: ?prhs? was not declared in this scope
estcondentropy.cpp:32: error: ?mxGetPr? was not declared in this scope
estcondentropy.cpp:33: error: ?mxGetM? was not declared in this scope
estcondentropy.cpp:34: error: ?mxGetN? was not declared in this scope
estcondentropy.cpp:41: error: ?mexErrMsgTxt? was not declared in this scope
estcondentropy.cpp:59: error: ?log? was not declared in this scope
estcondentropy.cpp:65: error: ?log? was not declared in this scope
estcondentropy.cpp:67: error: ?mxREAL? was not declared in this scope
estcondentropy.cpp:67: error: ?mxCreateDoubleMatrix? was not declared in this scope

Suggestions?!

12 Nov 2009

I can't compiler the file estmutualinfo.cpp.
estmutualinfo.cpp
estmutualinfo.cpp(42) : warning C4267: 'initializing' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(43) : warning C4267: 'initializing' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(55) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(58) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(65) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(67) : warning C4267: '=' : conversion from 'size_t' to 'long', possible loss of data
estmutualinfo.cpp(120) : error C2668: 'log' : ambiguous call to overloaded function
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(567): could be 'long double log(long double)'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(519): or 'float log(float)'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(121): or 'double log(double)'
while trying to match the argument list '(int)'

30 Sep 2009

Quite some people ask for mex files for different operating system versions, such as the latest Windows, or Mac OS X 10.6 Snow Leopard, etc. Please note that the mex functions can simply be re-generated by running the "makeosmex" command when you enter the folder of this toolbox (I assume you know how to set up mex compiling environment). For Windows, I use mingw to compile.

27 Dec 2008

Many thanks for your contribution, is it possible to adjust estpob file to be able to calculate mutual information for continuous data

26 Sep 2008

Thank you for your source code! It's very good^^

25 Jul 2008

very good

28 Apr 2008

I just needed the normalized MI which was easy to find and use

16 Feb 2008

i want to normalise the mutualinfo to the range (0-1) for comparison of different time series. Could you please help me by proper suggestion for this. Is it possible to change the number of bins used for the histogram so that all the data sets can be analyzed using the same scale.

03 Feb 2008

Can you help me to find joint probability for multivariate data....???how can I contact with you???

23 Jan 2008

i want to write joint probabilty as a function or want to express as an equation....

11 Jan 2008

I need un function that is needed in mutualinfo (estpab)

18 Dec 2007
29 May 2007

very good!!

22 May 2007

a nice piece of work, make my life much easier!

07 May 2007
07 May 2007

great! easy to use.

23 Aug 2007

a missing include file linked to Matlab mex.h and matrix.h