Finding specific values in a text Files.

1 view (last 30 days)
Matthew
Matthew on 13 May 2014
Answered: Walter Roberson on 13 May 2014
I have a Text File that contains 2 columns, both columns have millions of numbers. However when im writing my code i want to be able to find specific numbers in that column, for example finding the line in the column that starts at 0. What is the best function to use in doing so?

Answers (1)

Walter Roberson
Walter Roberson on 13 May 2014
This may sound odd, but the best function for searching in large text files (beyond what will fit in memory) is often to system() out to egrep or perl -- though you can also call perl directly from MATLAB which will find perl and do the system() for you.
egrep and perl and some other tools are written in languages that are more directly compiled, and which use optimized I/O buffering.
There is no MATLAB function to search large text files directly. This is partly because none of the operating systems that MATLAB runs on have system functions for searching text files, or system functions for positioning line-by-line in text files.
Will you be processing the same file multiple times? If you are then it can make sense to do a preprocessing stage to help you find entries more quickly.
For example if each file line is independent of position then sorting the file can help make it suitable for a binary search. Or running through the file and recording the ftell() position of the beginning of each line can make it possible to position to particular lines. Combining these two, sorting and recording line positions, can make binary search much easier to write.
If you have line positions it also becomes easier to form patterns such as 2-3 trees, easier to create hash tables, and so on.

Tags

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!