I'm facing following situation:
- I've retrieved a sizable amount of data (say tens of mb) from a MongoDb, and I've used the MongoDb Java API to load this data to Matlab. This results in a big Java object with a lot of nested fields.
- I would like to transform this Java object to a Matlab datamodel. This requires casting and vectorization, as the original Java object is not vectorized and contains Java objects.
- If I iterate over all fields (including nested fields), I can use the .get('fieldname') method to retrieve one particular value. However, it is not uncommon that I have to do this 100,000 times, and performance is not workable for this. The underlying performance bottleneck is likely related to following thread: http://www.mathworks.com/matlabcentral/newsreader/view_thread/246361. The solution that I'm considering right now:
- write a java method that is able to do the complete transformation to a Matlab object. This java method can construct MWArray, MWArrayStruct, MWArrayCell, etc. by making use of the Matlab JA builder. This would handle the vectorization and casting on java side, and would simply pass back a Matlab recognized object.
- However, the situation feels overly complex due to its dependencies: JA Builder requires the Matlab compiler and the subsequent compiled java class requires installation of the MCR, even though I want to call my transformation method from within Matlab.
Questions:
- Does this seem the right way of approaching this problem? Any other ideas that might work?