Thread Subject:

How can I improve Matlab processing speed if using built-in functions is the bottleneck?

Hi Pros:

I am trying to find out the way to improve the processing speed, so I used Profiler to identify which subfunctions consumed most of the processing time. However, it turned out that I found some built-in functions e.g. cell2mat() that consumed most of the processing time.

If this is the case and it is necessary to use built-in functions? Are there any ways to improve the processing speed?


Thank you in advance - Kelvin

On 8/15/2012 11:30 AM, Kelvin wrote:
> Hi Pros:
>
> I am trying to find out the way to improve the processing speed, so I
> used Profiler to identify which subfunctions consumed most of the
> processing time. However, it turned out that I found some built-in
> functions e.g. cell2mat() that consumed most of the processing time.
> If this is the case and it is necessary to use built-in functions? Are
> there any ways to improve the processing speed?

Generally one can find improvements, yes, but...not w/o the code in
question. :)

The first thing is to consider algorithmic changes of course.

As for the specific, if you can use an array directly instead of a cell
then you can remove the cell2mat() call entirely. Cells have benefits
of ragged arrays and other abstractions but come at a performance cost
for that convenience.

--

"Kelvin " <kelvinho8@hotmail.com> wrote in message <k0giqv$i4j$1@newscl01ah.mathworks.com>...
> Hi Pros:
>
> I am trying to find out the way to improve the processing speed, so I used Profiler to identify which subfunctions consumed most of the processing time. However, it turned out that I found some built-in functions e.g. cell2mat() that consumed most of the processing time.
===============

CELL2MAT is not a built-in function. It's an mfile just like any other .m file that you would write, and therefore has no guarantees of being particularly fast.


> If this is the case and it is necessary to use built-in functions? Are there any ways to improve the processing speed?
=============

You could try avoiding the use of cell arrays. They are stored/accessed in memory less efficiently than matrices and numeric arrays, for example. It's also always conceivable that your algorithm is bad, even if you're using well-coded functions to implement its building blocks.

You would have to show your code before anyone agrees with you that your current code is "necessary".

On 8/15/2012 12:02 PM, Matt J wrote:
> "Kelvin " <kelvinho8@hotmail.com> wrote in message
> <k0giqv$i4j$1@newscl01ah.mathworks.com>...
...

>> used Profiler to identify which subfunctions consumed most of the
>> processing time. However, it turned out that I found some built-in
>> functions e.g. cell2mat() that consumed most of the processing time.
...
> CELL2MAT is not a built-in function. It's an mfile just like any other
> .m file that you would write, and therefore has no guarantees of being
> particularly fast.
...

Interesting...I'd have thought that would have been a prime target for a
builtin function (postdates release here so I was just presuming it
would have been).

Must be an echo on the subject of perhaps avoiding cell array to begin
with... :)

--

On 8/15/2012 11:30 AM, Kelvin wrote:

> If this is the case and it is necessary to use built-in functions? Are there any
>ways to improve the processing speed?
>

Well, if the slow down is not due to being low in memory or using
a slow CPU, I do not not see we the users can speed up a "build-in"
function unless the source code is there.

For cell2mat, I just checked, and it partly if not all, is in
m source code and can be edited. Just type edit cell2mat

So you can use the editor, and look at it now and see if you can improve
the speed of it. If you do, you can send the improved code to Mathworks
so they can add your updates to the next version of Matlab, and I
am sure they and everyone will thank you if you make speed improvement
to it.

I am not sure on the legality of editing Matlab build-in functions. But
I think if you will be sending the changes to Mathwork and do not try
to sell the improved cell2mat function, it should be OK. But I am not
a lawyer, so you might want to check with someone who knows the law
on this better than I do.

--Nasser

As others have wrote, first option is to avoid using CELL all together of you can.

CELL2MAT is an mfile, and it can accomplish all sort of things, such as working on struct arrays, concatenate arrays with inhomogeneous sizes , working on nd arrays. Beside that it does all sort of error checking. Look at the code, it is well coded.

Still it is possible to improve the speed, using MEX, and if other characteristic of the input cell is known.

For example, if the input cell contains numerical scalar, this MEX file can accelerate by about 10 folds. Be aware that the function does not do any error checking. So it will crash if calling with wrong precondition inputs

/* a = scalarcell2mat(c)
 * Convert numerical scalar 2D cell array to array
 * INPUT:
 * c: 2D cell array where each element is a scalar
 * OUTPUT
 * a: 2D numerical array groupping all elements of C
 * ATTENTION: NO ERROR CHECKING
 *
 * MATLAB MEX scalarcell2mat.c
 * compile: > mex -O scalarcell2mat.c
 *
 * Author: Bruno Luong */

#include "mex.h"

#define C_IN prhs[0]
#define A_OUT plhs[0]

void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[] )
{
    mwIndex m, n, numel, i;
    mxArray *ci;
    double* aptr;
    
    if (mxGetClassID(C_IN) == mxCELL_CLASS)
    {
        m = mxGetM(C_IN);
        n = mxGetN(C_IN);
        A_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
        aptr = mxGetPr(A_OUT);
        numel = m*n;
        for (i=0; i<numel; i++)
            *(aptr++) = *mxGetPr(mxGetCell(C_IN, i)) ;
    }
    return;
} /* scalarcell2mat */

% Check on commad line
% Generate big cell array
c=num2cell(rand(1000));

tic; a1=cell2mat(c); toc % Elapsed time is 0.407101 seconds.
tic; a2=scalarcell2mat(c); toc % Elapsed time is 0.034149 seconds.

% Bruno

Hi All pros,

I think I made a mistake by saying cell2mat() was the function that consumed most of the time. As checked with the result from Profiler, I got the following simple get function that consumed lots of time:

=========================================
classdef Stock
.
.
.
.

  function value = getCurrentOpen(obj)
    value = fts2mat(obj.TimeSeries.OPEN(end:end));
  end
.
.
end
=========================================

However, when I checked the subfunctions of "getCurrentOpen", I found that the culprit would be a subfunction "fints.subsref".

=========================================
Function Name Function Type Calls Total Time % Time Time Plot
fints.subsref function 159948 705.411 s 88.7%
fints.fts2mat function 53316 53.409 s 6.7%
...oduct>FinancialProduct.get.TimeSeries subfunction 159948 9.462 s 1.2%
fints.end function 106632 3.920 s 0.5%
Self time (built-ins, overhead, etc.) 23.244 s 2.9%
Totals 795.447 s 100%
=========================================
Obviously, it is because my program uses financial time series object to store time series data for (open, close, low, high, volume, adjclose). I find this object very useful but it makes the processing speed extremely slow.

If I really want to keep this data structure by using financial time series object to store stock time series data, how can I speed my program up? Should I really modify the m file for fints.subsref? or are there any alternatives?

Thank you in advance - Kelvin


"Kelvin " <kelvinho8@hotmail.com> wrote in message <k0giqv$i4j$1@newscl01ah.mathworks.com>...
> Hi Pros:
>
> I am trying to find out the way to improve the processing speed, so I used Profiler to identify which subfunctions consumed most of the processing time. However, it turned out that I found some built-in functions e.g. cell2mat() that consumed most of the processing time.
>
> If this is the case and it is necessary to use built-in functions? Are there any ways to improve the processing speed?
>
>
> Thank you in advance - Kelvin

"Kelvin " <kelvinho8@hotmail.com> wrote in message <k0j2co$h3m$1@newscl01ah.mathworks.com>...
> Hi All pros,
>
> I think I made a mistake by saying cell2mat() was the function that consumed most of the time.

Still, I don't think this function takes most time, it's actually your post that take most of at least four people's time.

Bruno

"Kelvin " <kelvinho8@hotmail.com> wrote in message
news:k0j2co$h3m$1@newscl01ah.mathworks.com...
> Hi All pros,
>
> I think I made a mistake by saying cell2mat() was the function that
> consumed most of the time. As checked with the result from Profiler, I
> got the following simple get function that consumed lots of time:

*snip*

> However, when I checked the subfunctions of "getCurrentOpen", I found that
> the culprit would be a subfunction "fints.subsref".
> =========================================
> Function Name Function Type Calls Total Time % Time Time Plot
> fints.subsref function 159948 705.411 s 88.7%

So you're indexing into the fints object almost 160,000 times and each such
indexing operation takes about 0.0044 seconds (on average.)

*snip*

> Obviously, it is because my program uses financial time series object to
> store time series data for (open, close, low, high, volume, adjclose). I
> find this object very useful but it makes the processing speed extremely
> slow.
> If I really want to keep this data structure by using financial time
> series object to store stock time series data, how can I speed my program
> up? Should I really modify the m file for fints.subsref? or are there
> any alternatives?

That's impossible to say without seeing some of your code to understand
exactly what you're doing and how you're trying to do it. Learning the size
of your data would be helpful as well; if you're processing very large data
sets, it's going to take some time to process. You're not going to be able
to read War and Peace (not the Cliff Notes version, but the actual book) in
a half hour unless you're The Flash.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Hi Bruno,

I know it was such a rookie mistake since I'm still a rookie. That's why I need to learn from you guys.

Again, thank you for the advice from all of you.

Kelvin

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <k0j2tl$kka$1@newscl01ah.mathworks.com>...
> "Kelvin " <kelvinho8@hotmail.com> wrote in message <k0j2co$h3m$1@newscl01ah.mathworks.com>...
> > Hi All pros,
> >
> > I think I made a mistake by saying cell2mat() was the function that consumed most of the time.
>
> Still, I don't think this function takes most time, it's actually your post that take most of at least four people's time.
>
> Bruno

Hi Steven,

I see what you mean and I know what to do now.

Thank you so much, Steven.


Kelvin

"Steven_Lord" <slord@mathworks.com> wrote in message <k0j50n$7j9$1@newscl01ah.mathworks.com>...
>
>
> "Kelvin " <kelvinho8@hotmail.com> wrote in message
> news:k0j2co$h3m$1@newscl01ah.mathworks.com...
> > Hi All pros,
> >
> > I think I made a mistake by saying cell2mat() was the function that
> > consumed most of the time. As checked with the result from Profiler, I
> > got the following simple get function that consumed lots of time:
>
> *snip*
>
> > However, when I checked the subfunctions of "getCurrentOpen", I found that
> > the culprit would be a subfunction "fints.subsref".
> > =========================================
> > Function Name Function Type Calls Total Time % Time Time Plot
> > fints.subsref function 159948 705.411 s 88.7%
>
> So you're indexing into the fints object almost 160,000 times and each such
> indexing operation takes about 0.0044 seconds (on average.)
>
> *snip*
>
> > Obviously, it is because my program uses financial time series object to
> > store time series data for (open, close, low, high, volume, adjclose). I
> > find this object very useful but it makes the processing speed extremely
> > slow.
> > If I really want to keep this data structure by using financial time
> > series object to store stock time series data, how can I speed my program
> > up? Should I really modify the m file for fints.subsref? or are there
> > any alternatives?
>
> That's impossible to say without seeing some of your code to understand
> exactly what you're doing and how you're trying to do it. Learning the size
> of your data would be helpful as well; if you're processing very large data
> sets, it's going to take some time to process. You're not going to be able
> to read War and Peace (not the Cliff Notes version, but the actual book) in
> a half hour unless you're The Flash.
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com