You are now following this question
- You will see updates in your followed content feed.
- You may receive emails, depending on your communication preferences.
How can I extract data from a large ASCII file?
2 views (last 30 days)
Show older comments
anton fernando
on 20 May 2014
I have a ASCII data file with unknown number of columns and rows. In the file there are some unwanted text lines on top. I want to read only some of the columns in the data set with the header by removing the text lines on top. I appreciate if anyone can help.
Accepted Answer
Cedric
on 20 May 2014
>> doc textscan
and use the parameter HeaderLines to skip the header. Supposing that you have 7 header lines and that you need columns 1 and 3 (discarding the rest of each line), you should have something like:
content = fileread( 'myData.txt' ) ;
data = textscan( content, '%f%*f%f%*[^\n]', 'HeaderLines', 7, ...
'CollectOutput', true ) ;
data = data{1} ;
where you see a * in the formatSpec argument to skip column 2 and %*[^\n] at the end to skip the rest of the line.
14 Comments
Cedric
on 20 May 2014
By T and Z you mean columns 20 and 26 (like in Excel), or T_fit and another column whose header contains Z?
anton fernando
on 20 May 2014
Edited: anton fernando
on 20 May 2014
I need to extract the H2o and O3 columns. it is 4th and 6th column in the data set that I posted.
Cedric
on 20 May 2014
Edited: Cedric
on 20 May 2014
If these are always columns 4 and 6, you can use the following formatSpec (2nd argument in the call to TEXTSCAN):
'%*f%*f%*f%f%*f%f%*[^\n]'
where the %*f sequences skip (float) columns 1,2,3,5, the %f sequences read (float) columns 4 and 6, and the %*[^\n] sequence skips the rest of lines. Note that I cannot determine based on your picture whether there are 6 or 7 header lines, so you'll have to adjust this number.
anton fernando
on 20 May 2014
Edited: anton fernando
on 20 May 2014
Actually your code worked. Thank you very much. How can I read the data set when I don't know the column number and only when I know the column heading? Thank you again. You saved me lot of time from wasting. You are the best.
Cedric
on 20 May 2014
See my last comment for columns 4 and 6.
If you are able to build the formatSpec based on the a priori knowledge of which columns (number) you need, the approach above is by far the easiest way to process your file(s).
If you need to find programmatically columns number based on header names 'H2O' and 'O3' (and others), it gets significantly more complicated. If you really need this feature, I can extend my solution though; just let me know.
anton fernando
on 20 May 2014
Hey, I can understand what you have done here now. But there is a problem. I don't get data as they are in the data file. Some data read as 0. I want to get data in (0.350E+13) format. Also what ('CollectOutput', true) part does in your code? Thank you. P.S: I think I can read data by the column number. All data are in same column in each file. Thank you again.
Cedric
on 20 May 2014
Edited: Cedric
on 20 May 2014
My guess is that the code reads data correctly, but you don't display them correctly afterwards. For example, if you just type data, MATLAB will factor 1.0e+12, and display a lot of 0.0000 which are in fact non-zero. Try to access one of these elements direct, e.g. data(10,1). If it isn't that, could you attach (icon staple) one of these files instead of a screenshot so I can try? If files are large, truncate one to e.g. 50 rows.
The CollectOutput parameter groups columns in data{1} (in this case). Without it, you get one cell per column, which is more difficult to use then (you have to merge cells content manually afterwards).
Cedric
on 20 May 2014
Edited: Cedric
on 20 May 2014
Yep, that's it. You probably updated the example to
data = textscan( content, '%*f%*f%*f%f%*f%f%*[^\n]', 'HeaderLines', 12, ...
'CollectOutput', true ) ;
data = data{1} ;
Now you can see your data e.g. per row, and you'll see that they are non-zero:
>> data(50,:)
ans =
1.0e-03 *
0.5940 0.0073
Optionally, if you really wanted to display the whole array and see non-zero entries, you could type
>> format long
Then ..
>> data
data =
1.0e+02 *
0.009380000000000 -9.990000000000000
0.008250000000000 -9.990000000000000
0.007240000000000 -9.990000000000000
0.006350000000000 -9.990000000000000
0.005550000000000 -9.990000000000000
0.004840000000000 -9.990000000000000
0.004200000000000 -9.990000000000000
0.003630000000000 -9.990000000000000
0.003130000000000 -9.990000000000000
0.002670000000000 0.000000614000000
0.002280000000000 0.000000292000000
0.001950000000000 0.000000110000000
0.001680000000000 0.000000076300000
0.001440000000000 0.000000035600000
0.001230000000000 0.000000047500000
0.001060000000000 0.000000042100000
0.000906000000000 0.000000042300000
0.000775000000000 0.000000042200000
0.000664000000000 0.000000043400000
0.000567000000000 0.000000045600000
0.000484000000000 0.000000047000000
0.000413000000000 0.000000048500000
0.000353000000000 0.000000049900000
0.000301000000000 0.000000051300000
0.000256000000000 0.000000052600000
...
but again, the fact that you displayed a lot of zeros initially is just a display artifact (truncation), so there is no need to set the display format to long if all you need is to compute with these numbers afterwards.
anton fernando
on 20 May 2014
Thank you very much. One last question. if I need the data of the 80th column do I have to write %f 80 times?
Cedric
on 20 May 2014
Edited: Cedric
on 20 May 2014
You're welcome. Interestingly, REPMAT works with strings as well (as with numeric/cell arrays), so no, if you needed columns 4, 6, and 80, you could do something like (not tested, you may have to check and adapt/correct):
formatSpec = ['%*f%*f%*f%f%*f%f', repmat('%*f', 1, 73), '%f%*[^\n]'] ;
data = textscan( content, formatSpec, 'HeaderLines', 12, ...
'CollectOutput', true ) ;
data = data{1} ;
More Answers (0)
See Also
Categories
Find more on Text Data Preparation in Help Center and File Exchange
Tags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!An Error Occurred
Unable to complete the action because of changes made to the page. Reload the page to see its updated state.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)