How I can I save large tables without using save(data, '-V7.4')?

7 views (last 30 days)
I'm saving data in large tables whilst fitting a model to a small set of data. This is producing tables which are quite small. I am repeating this process for 25 data sets.
There are several tables nested within the cells of other table. There's a summary table with each row containing a set of grid parameters each with a measure of fit. Within this subtable are the iterations of fminsearch for the specific combination of parameters.
I have a major problem saving this data. For around 20 data sets the save function works. The file it produces is nice and small. After another data set I get an error during telling me that matlab is 'Out Of Memory during serialization of the Subsystem Data. Error using save. Error closing file'.
After much searching I've saved the file using save(data, '-V7.4') but the filesize is 4.5 GB which is far larger than the 4MB file created before the error message. What is going on and can I save using the normal save function? Or can matlab compress the 4.5GB file it is creating?
Many thanks,
Dr James Tripp
  3 Comments

Sign in to comment.

Answers (1)

Titus Edelhofer
Titus Edelhofer on 13 Jul 2014
Hi James,
generally speaking, the v7.3 is an uncompressed format, whereas the default format (7.0) is a compressed format. It looks like the following happens:
  • Your data are large (what does "memory" in MATLAB tell you or the taskmanager, is this true?)
  • Your data compress very well.
Therefore the file in 7.0 is small, but for creating it, memory is needed for the compression process. In format 7.3 it's as large as before in memory, but no additional memory for compressing is used, therefore you don't get the Out Of Memory error.
Titus

Categories

Find more on Debugging and Analysis in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!