How do I separate raw data in columns, effectively?

2 views (last 30 days)
I have been trying to take the raw data from a txt. file that came from a beagleboneblack (small application computer) and have had a few problems separating some of the values, how could I affect the file once it is imported so that specific rows are chosen.
I have a lot of data and cannot afford to sit down and pick them one by one. I have made separate file for most of them, but since they overlap in some cases it is impossible for me to see them correctly. here is an example:
Accel (mg), Mag (mGauss) 946808257.344 (-85.9375, 33.203125, 997.0703125), (-27.27272727272727, -357.27272727272725, 1.8181818181818181)] Gyro (dps) 946808257.346 (-2.532958984375, 0.6103515625, 1.0528564453125)] P(mbar), T(C) 946808257.35 1008.09423828125, 27.545833333333334] Accel (mg), Mag (mGauss) 946808257.443 (-83.984375, 30.2734375, 985.3515625), (-23.636363636363633, -351.81818181818181, 4.545454545454545)] Gyro (dps) 946808257.446 (-2.3651123046875, 0.396728515625, 0.9613037109375)] P(mbar), T(C) 946808257.449 1008.3203125, 27.556249999999999] Accel (mg), Mag (mGauss) 946808257.546 (-81.0546875, 29.296875, 989.2578125), (-23.636363636363633, -357.27272727272725, 0.90909090909090906)] Gyro (dps) 946808257.56 (-2.349853515625, 0.6256103515625, 0.701904296875)] P(mbar), T(C) 946808257.577 1007.97705078125, 27.524999999999999]
Thank you for your help!
  5 Comments
Space Science undergrad
Space Science undergrad on 15 Nov 2013
the problem is that the python is on a small beaglebone that can only process so much at a time, if I ask him to spit it out differently, it wont calculate fast enough to give me the amount of data needed. When I bring it in editor, it separates it in three distinct columns that repeats itself, but the separation of data is overlapping, which does not let me take the data in a productive context. If you copy and paste my example in a notepad and open it with, you might be able to understand my problem. class now, ill be back this afternoon, thanks for your help.
dpb
dpb on 15 Nov 2013
OK, if look in more detail, it appears there is a soft record marker of a ']' -- assuming that is so then it appears there are sets of three records each as follows...
Accel (mg), Mag (mGauss) 9...7.344 (-85, 33, 997), (-27, 357, 1)]
Gyro (dps) 9...7.346 (-2.532958984375, 0.6103515625, 1.0528564453125)]
P(mbar), T(C) 9...7.35 1008.09423828125, 27.545833333333334]
where I shortened the values of the first "record" to fit on a single line for clarity.
I'm guessing the first value 946808257.344 and so on is a timestamp, maybe, followed by the three-coordinate values of the named variables?
That would make the numbers of the values match up with the labels. If this is so, let know--I'll wait to make certain know what it is we're actually parsing before actually trying to do so...

Sign in to comment.

Accepted Answer

dpb
dpb on 15 Nov 2013
OK, I just went ahead with the above assumption...one way reading a group of three as a set--
>> fmt='Accel (mg), Mag (mGauss) %f (%f, %f, %f), (%f, %f, %f)] Gyro (dps) %f (%f, %f, %f)] P(mbar), T(C) %f %f, %f]';
>> c=textscanf(fid,fmt,'collectoutput',0);
>> acc=cell2mat(c(2:4))
acc =
-85.9375 33.2031 997.0703
-83.9844 30.2734 985.3516
-81.0547 29.2969 989.2578
>> mag=cell2mat(c(5:7))
mag =
-27.2727 -357.2727 1.8182
-23.6364 -351.8182 4.5455
-23.6364 -357.2727 0.9091
>> gyr=cell2mat(c(9:11))
gyr =
-2.5330 0.6104 1.0529
-2.3651 0.3967 0.9613
-2.3499 0.6256 0.7019
>> P=cell2mat(c(13))
P =
1.0e+03 *
1.0081
1.0083
1.0080
>> T=cell2mat(c(14))
T =
27.5458
27.5562
27.5250
>>
The various timestamps(?) are in c(1), c(8) and c(12), respectively. The total number of them don't match up with the other values, you'll have to figure out where/how to match them up.
Alternatively, you could instead of defining the format string for the full set of three records do one for each and put the textscan call in a loop and have the timestamps retained with the records instead of sorting them out later. All depends on what you're after and how will use them in the end as to what's more convenient. But, it does get the data variables into usable form automagically.
  1 Comment
Space Science undergrad
Space Science undergrad on 15 Nov 2013
Edited: Space Science undergrad on 15 Nov 2013
this is exactly what I was looking for, I will make independent graphs with their respective timestamps and create a timeline to see where values match up. Thanks a million for your effort and +1 for automagically!

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!