Why is save/load write/read soooo slooooow?
56 views (last 30 days)
Show older comments
I used a save myworkspace at a point when I want to interrupt a long process, and sometime later I do a load myworkspace to restore the interrupt point and continue.
The typical mat file produced by this procedure is 18 GB. The save takes 1200 seconds, the load takes 350 seconds. The times are the same using a single HDD or a SSD cluster in Raid0 configuration.
On the same computer I can use the OS to copy a 18 GB file in 65 seconds on HDD, 30 seconds on SSD.
So it appears that Matlab save/load have some serious bottlenecks. Is there anything I can do differently? Is this something I should raise to Mathworks Support?
9 Comments
Cedric
on 12 Sep 2013
I think that even this partial answer will be useful information for the next persons who are facing this issue and who find your thread!
cui,xingxing
on 24 Aug 2021
similar questions here, matfile and half inefficient storage
similar questions here, The “load” and “save” functions run very slowly?
Accepted Answer
per isakson
on 8 Sep 2013
Doc says:
save(filename, ..., version) saves to MAT-files in the specified version:
'-v4', '-v6', '-v7',or '-v7.3'.
and
'-v7.3'7.3 (R2006b) or laterVersion 7.0 features plus support for data
items greater than or equal to 2 GB on 64-bit systems.
18GB requires v7.3, which is slow. v6 is significantly faster. v6 might be an alternative if you can split the data into <2GB chunks.
0 Comments
See Also
Categories
Find more on HDF5 in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!