How do I load a LARGE file into a DISTRIBUTED ARRAY on a CLUSTER?

1 view (last 30 days)
I have a huge binary file, twoHundered.bin, . It isso huge that you could not feed it into the memory of one machine, but it would fit into a four node cluster.  This file is flat uint8.  Put its contents into a distributed array.  You are not allowed to read it into one node, and then distribute it (because it would not fit onto one node).

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 27 Jun 2009
You can use the memmapfile to help you load a large file in a distributed array.
P>> m = memmapfile('twoHundered.bin');
P>> V = dcolon(1,50); % 50 is number data elements
P>> data = m.Data(local(V));
P>> data_distrib = darray(data);

More Answers (0)

Products


Release

R2007b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!