How can I display the progress of textscan

3 views (last 30 days)
Nick Counts
Nick Counts on 22 Oct 2014
Commented: Nick Counts on 25 Oct 2014
I am using textscan to read and parse data files that vary widely in size. For small files it is very fast, but it is not uncommon to read files in excess of 700 megabytes.
Since these files can take over a minute to load, it would be helpful to show the user that progress is being made.
Is there a way to get the progress of textscan - ideally I'd send it to waitbar() or something similar, but any indication I can give my user will be a major improvement.
I have zero ideas at this point without implementing my own textscan, and that's not something I'm excited to undertake.
  8 Comments
dpb
dpb on 25 Oct 2014
Possibly move the sscanf step to scanf on the file would be one suggestion.
I'm thinking probably to get any further real input would need to see a section of the file under discussion and what it is you want from it.
Doesn't have to be large, just enough to illustrate the format and whatever "issues" it has altho sounds like it is at least regular.
Nick Counts
Nick Counts on 25 Oct 2014
Thanks for all your time on this - here is a snippet of data that is representative:
2014/194/15:57:17.272106, , ,RP1 PCVNC-1014 Globe Valve Cmd,SC,FGSE M-P0A-RP1-PCV-1014 RP1 Tanking Low Flow Control PCVNC, , ,
2014/194/15:57:17.272405, , ,RP1 PCVNC-1014 Globe Valve Cmd Def Response,CR,, ,ACK,
2014/194/15:57:17.273198, , ,RP1 PCVNC-1015 Globe Valve Cmd,SC,FGSE M-P0A-RP1-PCV-1015 RP1 Tanking High Flow Control PCVNC, , ,
2014/194/15:57:17.273759, , ,RP1 PCVNC-1015 Globe Valve Cmd Def Response,CR,, ,ACK,
2014/194/15:57:17.283244, , ,AIR RV-0003 Position Mon, A,ECS E-P0A-AIR-RV-0003 AIR Position Sensor,----------------,1.000000000000000000E+02,%open,
2014/194/15:57:17.443121, , ,LO2 PCVNO-2014 Globe Valve Mon, A,FGSE M-P0A-LO2-PCV-2014 LO2 Tanking Low Flow Control PCVNO,----------------,2.785333333333330685E+01,%open,
2014/194/15:57:17.683124, , ,AIR RV-0004 Position Mon, A,ECS E-P0A-AIR-RV-0004 AIR Position Sensor,----------------,1.000000000000000000E+02,%open,
2014/194/15:57:17.783104, , ,AIR RV-0004 Position Mon, A,ECS E-P0A-AIR-RV-0004 AIR Position Sensor,----------------,1.000000000000000000E+02,%open,
2014/194/15:57:17.921458, , ,LO2 PCVNO-2013 Globe Valve Cmd,SC,FGSE M-P0A-LO2-PCV-2013 LO2 Tanking Low Flow Control PCVNO, , ,
2014/194/15:57:17.921643, , ,LO2 PCVNO-2013 Globe Valve Cmd Def Response,CR,, ,ACK,
2014/194/15:57:35.789884, , ,LO2 PCVNC-2059 State,UI,PCVNC-2059 Tank Vapor Sply Press Ctl Valve,----------------,1,
Fortunately, it is regularly structured. But there is a mixture of data types and not every line contains a numerical value to extract.

Sign in to comment.

Answers (0)

Categories

Find more on Large Files and Big Data in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!