How to create an empty array in C and fill it in Matlab?

This is the problem:

I have a C code that creates an array nRows x 2, called InitialArray filled with random numbers. Then I split that array into two parts, that is the first column is called xProcess and the second column of the array is yProcess. The goal is to pass these variables I created (xProcessand yProcess) to Matlab and there a Matlab function based on probabilities rules will put NaN in some of the elements of xProcess and yProcess, and will fill empty arrays called xC and yCwith the values of xProcess and yProcess before turning to NaN. Finally, the C code gets all four arrays back again, perform some operations on them and send them again to Matlab. This process is repeated for a number of time steps.

Here is a piece of the code:

int main ( void ){


Engine *ep;

if (!(ep = engOpen(""))) {

    fprintf
(stderr, "\nCan't start MATLAB engine\n");
    
return EXIT_FAILURE;

}

mxArray *xProcess = NULL;
mxArray *yProcess = NULL;    
mxArray *xC = NULL;
mxArray *yC = NULL;

int timeStep;
int numberProcess = 4e6;
int numberC = 0;
/*So far I created the mxArray, and the number of random numbers that will be stored in xProcessand yProcess. Now comes the loop.*/
for (timeStep=0 ; timeStep<numberTimeSteps ; timeStep++){

double *arrayTempxProcess;
arrayTempxProcess = (double *)calloc(numberProcess, sizeof(double)); 

double *arrayTempyProcess;
arrayTempyProcess = (double *)calloc(numberProcess, sizeof(double));   


double *arrayTempxC;
arrayTempxC = (double *)calloc(numberC, sizeof(double)); 

double *arrayTempyC;
arrayTempyC = (double *)calloc(numberC, sizeof(double)); 

xProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL); 
yProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL);

xC = mxCreateDoubleMatrix(numberC, 1, mxREAL); 
yC = mxCreateDoubleMatrix(numberC, 1, mxREAL);

memcpy((void *)mxGetPr(yProcess), (void *)arrayTempyProcess, numberProcess*sizeof(double));
memcpy((void *)mxGetPr(xProcess), (void *)arrayTempxProcess, numberProcess*sizeof(double));

As you see xC and yC I create them as empty. I believe this is equivalent to xC = [] and yC = [] in Matlab. And the arrayTempyProcess and arrayTempxProcess are temporal arrays that each of them gets a column of InitialArray.

if(timeStep>0){
memcpy((void *)mxGetPr(yC), (void *)arrayTempyC,(numberC)*sizeof(double));
memcpy((void *)mxGetPr(xC), (void *)arrayTempxC, (numberC)*sizeof(double)); 
}
/*Now I send them to Matlab*/
engPutVariable(ep, "xProcess", xProcess);
engPutVariable(ep, "yProcess", yProcess);
engPutVariable(ep, "xC", xC);
engPutVariable(ep, "yC", yC);


free(arrayTempxProcess);
free(arrayTempyProcess);
free(arrayTempxC);
free(arrayTempyC);

Now the Matlab function puts NaN in xProcess and yProcess, and fills xC and yC.

printf("Entering MATLAB \n");
engEvalString
(ep, "[xProcess,yProcess,xC, yC] = my_matlab_function(xProcess,yProcess,xC, yC);"); 
printf
("Leaving MATLAB \n");

Now comes trouble... Now I am trying to get the results... but the function mxGetM(xC) produces a segmentation fault in the first time step that I don't understand.

xC = engGetVariable(ep,"xC"); 
int mRowsC = mxGetM(xC);

double *XC = NULL;
XC = (double *)malloc (mRowsC*sizeof(double) );
memcpy(XC, mxGetData(xC),mRowsC*sizeof(double));

yC = engGetVariable(ep,"yC");

double *YC = NULL;
YC = (double *)malloc (mmy*sizeof(double) );
memcpy(YC, mxGetData(yC),mmy*sizeof(double));

xProcess = engGetVariable(ep,"xProcess"); 

int mRowsProcess = mxGetM(xProcess);

double *XPROCESS = NULL;
XPROCESS = (double *)malloc (mRowsProcess*sizeof(double) );
memcpy(XPROCESS, mxGetData(xPROCESS),mRowsProcess*sizeof(double));

yProcess = engGetVariable(ep,"yProcess");

double *YPROCESS = NULL;
YPROCESS = (double *)malloc (mRowsProcess*sizeof(double) );
memcpy(YPROCESS, mxGetData(yProcess),mRowsProcess*sizeof(double));

numberC = mRowsC;

mxDestroyArray(xProcess);
mxDestroyArray(yProcess);
mxDestroyArray(xC);
mxDestroyArray(yC);
free(XPROCESS);
free(YPROCESS);
free(XC);
free(YC);

}

}

I have tried to reduce the number of elements in xProcess and yProcess but I get the segmentation fault at some point later than the first time step. The weird thing about this, is that I also tried to save the results in Matlab after printf("Leaving MATLAB \n"); but it didn't work out. It saved nothing.

I'm new coupling C and Matlab, so my code might be entirely wrong, but what is the proper way to create arrays in C and fill them in Matlab? Is there any limits in which C cannot be coupled with Matlab functions?

Any help is appreciated. Thanks!

Answer 1

[James Tursa]

You first need to fix your memory leaks, and then also check the return pointer from engGetVariable. Maybe you are just running out of memory because of the memory leaks. E.g.,

xProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL); 
yProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL);
xC = mxCreateDoubleMatrix(numberC, 1, mxREAL); 
yC = mxCreateDoubleMatrix(numberC, 1, mxREAL);
    :
xC = engGetVariable(ep,"xC");   <-- leaks the xC from above
    :
yC = engGetVariable(ep,"yC");  <-- leaks the yC from above
    :
xProcess = engGetVariable(ep,"xProcess");   <-- leaks the xProcess from above
    :
yProcess = engGetVariable(ep,"yProcess");  <-- leaks the yProcess from above

You need to put in some mxDestroyArray(xC) etc between the first group of lines above and the engGetVariable( ) lines below.

Also, check the return value of engGetVariable( ) for NULL. E.g.,

xC = engGetVariable(ep,"xC");
if( xC == NULL ) {
    /* Put code here to handle error */
}

Frankly, you should also be checking the return values of every malloc and calloc call.

Also I would note that you have a lot of data copying going on in your code. Can you simply work with the pointers returned from mxGetPr( ) directly instead of copying the data back & forth via memcpy each time? E.g., instead of this:

double *XC = NULL;
XC = (double *)malloc (mRowsC*sizeof(double) );
memcpy(XC, mxGetData(xC),mRowsC*sizeof(double));

why can't you just do this:

double *XC = mxGetPr(xC);

As a side note, you don't need those (void *) casts in your mcmcpy arguments ... you always get those conversions automatically from the compiler.

[S. Perez]

Thanks a lot for your answer James.

I have applied the changes you suggested to my code because of the memory leaks and I haven't had any segmentation fault so far.

But I wonder if I still can reduce the amount of memory more efficiently, for example is it posible to destroy xC after this:

xC = engGetVariable(ep,"xC");
double *XC = mxGetPr(xC);

I mean, I changed my code to this:

xProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL);
yProcess = mxCreateDoubleMatrix(numberProcess, 1, mxREAL);
xC = mxCreateDoubleMatrix(numberC, 1, mxREAL); 
yC = mxCreateDoubleMatrix(numberC, 1, mxREAL);

/*Now I send them to Matlab*/
engPutVariable(ep, "xProcess", xProcess);
engPutVariable(ep, "yProcess", yProcess);
engPutVariable(ep, "xC", xC);
engPutVariable(ep, "yC", yC);
/* Free some memory */
free(arrayTempxProcess);
free(arrayTempyProcess);
free(arrayTempxC);
free(arrayTempyC);
mxDestroyArray(xProcess);
mxDestroyArray(yProcess);
mxDestroyArray(xC);
mxDestroyArray(yC);

(...)
    

xC = engGetVariable(ep,"xC");    
double *XC = mxGetPr(xC);
yC = engGetVariable(ep,"yC");
double *YC = mxGetPr(yC);   
/*The same with xProcess and yProcess (...)*/

But xC is going to be rewritten at the next iteration by

xC = mxCreateDoubleMatrix(numberC, 1, mxREAL); 

So my question is how can I get rid off it at the end of the loop in order to make a more efficient use of memory.

Another thing I thought to avoid running out of memory or getting null results from xC is to split xC into smaller arrays. This might be very inefficient in terms of computational time but will it be helpful to avoid memory problems?

Thanks again for your previous answer!

[James Tursa]

If I understand your question, the answer is yes you need to do mxDestroyArray(xC) and others at the end of the loop before you start the next iteration of the loop. You just can't do it until after you are done with the XC pointer. That is, you need one mxDestroyArray(xC) to clean up the xC = mxCreateDoubleMatrix(numberC, 1, mxREAL), and you need another mxDestroyArray(xC) to clean up the xC = engGetVariable(ep,"xC").

To answer your question about minimizing memory and to see if you still have memory leaks I would probably have to see your current full code. Note that memory usage is one of the main drawbacks to using the MATLAB Engine, since each of those engPutVariable( ) and engGetVariable( ) calls creates a deep copy. You can sometimes avoid this if you use the mex routine approach (MATLAB calls your C code) as opposed to the Engine approach (your C code calls MATLAB).