MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

# Thread Subject: Data Manipulation (Re-arranging) and Writing to text file

 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 18 Jun, 2012 03:19:07 Message: 1 of 18 Hello, here’s the details of the problem that I am writing about, and my attempts at a solution: a. I need to extract elements from rows 1-6 from the first column of a file (input.txt, I’ve loaded the file successfully). This file has numerous columns with 200 rows each. I am interested in only the first column. Then I need to print these extracted elements to row 1 of a text file, separated (delimited) by a tab. b. I need to then repeat this in succession 10 times.i.e. extract 7-12 and print them on row 2 of the text file (tab delimited), then extract 13-18 and print them on row 3 of the text file (tab delimited). c. Now, the 11th time (i.e. row 11), I need only 2 elements (rows 61 and 62) to be extracted and printed on row 11. The 4 other spaces (where elements 63, 64, 65, 66 would go) should be left empty (not zeros, but empty). MY ATTEMPTS: a. Here’s what I have come up with (neither is printing more than 1 row, but rather just printing the last row that is extracted). I like the first method (taking the transpose, it seems to be working), but I can’t get it to print as I go along through the loop. First Approach (extract vectors using a loop, use dlmwrite to write them to text file): eee = importdata('input.txt');   for i = 1:6:19         aa = eee.data(i:i+5,1);         dlmwrite('test3.txt',aa','\t') end Second Approach (extract elements indiviually and write them to the text file using fprintf): eee = importdata('input.txt');   sav_fid = fopen('test4.txt','w');   for i = 1:6         aa(i) = eee.data(i,1);         fprintf(sav_fid,'%f',aa(i)); end b. I am thinking I would need to do this:  for j = 1:6:66      end I would put this around the loop in a. Is this correct or is there is a better way to accomplish this step? c. If this was a 1-D array, I would do it easily using: rem = [r1 r2 r3] A(rem) = [] I’m just not sure if this would work if step b. is not working. Is there a better way to do this? Regards: Stan T.
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 04:36:00 Message: 2 of 18 On 6/17/2012 10:19 PM, Stan wrote: ... > a. I need to extract elements from rows 1-6 from the first column of a > file (input.txt, I’ve loaded the file successfully). This file has > numerous columns with 200 rows each. I am interested in only the first > column. Then I need to print these extracted elements to row 1 of a text > file, separated (delimited) by a tab. x=load(...): % you say you've got that x(:,2:end)=[]; % retain column only 1 for simplicity in addressing > b. I need to then repeat this in succession 10 times.i.e. extract 7-12 > and print them on row 2 of the text file (tab delimited), then extract > 13-18 and print them on row 3 of the text file (tab delimited). > > c. Now, the 11th time (i.e. row 11), I need only 2 elements (rows 61 and > 62) to be extracted and printed on row 11. The 4 other spaces (where > elements 63, 64, 65, 66 would go) should be left empty (not zeros, but > empty). x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6 Now om the next step you've gotten yourself into a bind...you can't have a "ragged" array w/ only 2 values in some rows and 6 in all the rest. So, you've got one of two choices--go to a cell array (ugly in this case I think) or process the output in pieces writing the full rows where wanted and less for the others. --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 18 Jun, 2012 06:21:06 Message: 3 of 18 Hi dpb: > So, you've got one of two choices--go to a cell array (ugly in this case > I think) or process the output in pieces writing the full rows where > wanted and less for the others. > The processing of the output in pieces would be much better. But I'm not sure how I would indicate that a row needs to be truncated. Wouldn't you need to specify a fixed row end ---> 6, in this case? I think I'm just not seeing it.
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 11:12:57 Message: 4 of 18 On 6/18/2012 1:21 AM, Stan wrote: > Hi dpb: > >> So, you've got one of two choices--go to a cell array (ugly in this >> case I think) or process the output in pieces writing the full rows >> where wanted and less for the others. >> > > The processing of the output in pieces would be much better. But I'm not > sure how I would indicate that a row needs to be truncated. Wouldn't you > need to specify a fixed row end ---> 6, in this case? > > I think I'm just not seeing it. You have some logic to decide which are/aren't...a loop somewhat like the one you did (mod() comes to mind to use for the test if it is every n-th one). You can write the size of the array that is contiguous and full in one subset w/ fprintf, then the other w/ the proper subset of indices. W/ the offset arguments, I guess dlmwrite() should work as well altho I've not tested that. --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 18 Jun, 2012 14:48:07 Message: 5 of 18 > x(:,2:end)=[]; % retain column only 1 for simplicity in addressing . . . > x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6 Is it possible to extract a subset of a matrix, where you indicate the range of matrix elements to be extracted? I mean, after aplying the second command (reshaping which gave rows of length 6), could the first 11 rows be extracted by specifying (not the row numbers, but) the range of matrix elements to be extracted - in this case [1:66]?
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 15:24:25 Message: 6 of 18 On 6/18/2012 9:48 AM, Stan wrote: >> x(:,2:end)=[]; % retain column only 1 for simplicity in addressing > .. > .. > .. >> x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6 > > Is it possible to extract a subset of a matrix, where you indicate the > range of matrix elements to be extracted? I mean, after aplying the > second command (reshaping which gave rows of length 6), could the first > 11 rows be extracted by specifying (not the row numbers, but) the range > of matrix elements to be extracted - in this case [1:66]? Of course, but the linear address of the first 66 isn't the same as the first 11 rows as Matlab storage order is column-major. But, all you need is to write out x(1:M,:) for whatever value of M you need and then the one or two or however many that aren't full as x(M+N,1:2) % for N = 1, 2, ... , whatever it is you want If there are more remaining of the same pattern then put this in a loop and increment by M+N for the first row index. If the pattern changes, then you'll need to add the logic to compute the M and N as you iterate through as well, but if you know what it is, then that should be no problem even if you have to do it as a lookup table because it's irregular. NB, of course, that owing to the column major storage order you'll need to use the ' (transpose) operator on the submatrices to write them in rows as displayed on the screen to the file... --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 15:33:48 Message: 7 of 18 On 6/18/2012 10:24 AM, dpb wrote: ... > But, all you need is to write out > > x(1:M,:) > > for whatever value of M you need and then the one or two or however many > that aren't full as > > x(M+N,1:2) % for N = 1, 2, ... , whatever it is you want ... x(M+1+N,1:2) % for N = 1, 2, ... , whatever it is you want     __ NB the +1... --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 17:24:54 Message: 8 of 18 On 6/18/2012 9:48 AM, Stan wrote: >> x(:,2:end)=[]; % retain column only 1 for simplicity in addressing > .. > .. > .. >> x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6 > > Is it possible to extract a subset of a matrix, where you indicate the > range of matrix elements to be extracted? I mean, after aplying the > second command (reshaping which gave rows of length 6), could the first > 11 rows be extracted by specifying (not the row numbers, but) the range > of matrix elements to be extracted - in this case [1:66]? Well, my previous posting reminding you to transpose the transpose made me slap me head... :) Try this at the command window and see if it isn't what you're looking for... x=[1:200]'; f1=[repmat('%4d',1,6) '\n']; f2=[repmat('%4d',1,2) '\n']; ix=1; while    ix
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 18 Jun, 2012 20:47:41 Message: 9 of 18 On 6/18/2012 12:24 PM, dpb wrote: ... > x=[1:200]'; > f1=[repmat('%4d',1,6) '\n']; > f2=[repmat('%4d',1,2) '\n']; > ix=1; > while > ix fprintf(f1,x(ix:ix+59)) > fprintf(f2,x(ix+60:ix+61)) > ix=ix+66; > end ... Just a minor cleanup of a formatting problem and a few fewer magic numbers... i1=1; while i1
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 18 Jun, 2012 20:59:07 Message: 10 of 18 ^^^ I'm going through your code and will test if after that. I think I'm starting to get a little more comfortable with some of the thngs you are saying (knock on wood). It may take me some time to reply though.
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 20 Jun, 2012 06:02:05 Message: 11 of 18 I've gone through this. Two questions: 1. Could you explain the two fprintf lines and the first repmat line? 2. I'm getting the following error message: Index exceeds matrix dimensions. Error in test3 (line 7)    fprintf(f1,x(i1:i2)) I'm using MATLAB 2012a. Could there be a version conflict? > dpb wrote in message ... > > Just a minor cleanup of a formatting problem and a few fewer magic > numbers... > > i1=1; > while i1 i2=i1+59; > fprintf(f1,x(i1:i2)) > i1=i2+1; i2=i1+1; > fprintf(f2,x(i1:i2)) > i1=i2+4; > end > > --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 20 Jun, 2012 11:59:24 Message: 12 of 18 On 6/20/2012 1:02 AM, Stan wrote: ...[top posting repaired...please don't; hard follow conversation makes]... >> dpb wrote in message ... >> >> Just a minor cleanup of a formatting problem and a few fewer magic >> numbers... >> >> i1=1; >> while i1> i2=i1+59; >> fprintf(f1,x(i1:i2)) >> i1=i2+1; i2=i1+1; >> fprintf(f2,x(i1:i2)) >> i1=i2+4; >> end >> >> --  > I've gone through this. Two questions:  >  > 1. Could you explain the two fprintf lines and the first repmat line? Try the output of repmat() at the command line...each is just building a format string for the specific number of elements to write per line. (This is a remnant of the C-like formatting strings that lack a repeat specifier...) fprintf() simply outputs the number of elements from x w/ the format specified. Since you have a single column vector, linear addressing of i1:i1 picks up the number you requested--first w/ 6 elements/row, then a row of 2 and skip four from the list...  > 2. I'm getting the following error message:  >  > Index exceeds matrix dimensions.  >  > Error in test3 (line 7)  > fprintf(f1,x(i1:i2))  >  > I'm using MATLAB 2012a. Could there be a version conflict? No, that's because the length of the vector isn't evenly divisible by the number of elements you asked to be printed in each pass...it's the boundary condition I mentioned earlier. You have to decide whether to abort early and ignore the last few elements or what you want to do about that; I don't know as you've not said. You're doing 66 values per outer loop but you said the input was 200 rows in length--200/66 = 3.03... It's that .03 that's the problem. --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 20 Jun, 2012 12:59:18 Message: 13 of 18 On 6/20/2012 6:59 AM, dpb wrote: ... > fprintf() simply outputs the number of elements from x w/ the format > specified. Since you have a single column vector, linear addressing of > i1:i1 picks up the number you requested--first w/ 6 elements/row, then a > row of 2 and skip four from the list... That is i1:i2 of course... > > 2. I'm getting the following error message: > > > > Index exceeds matrix dimensions. ... > No, that's because the length of the vector isn't evenly divisible by > the number of elements you asked to be printed in each pass...it's the > boundary condition I mentioned earlier. You have to decide whether to > abort early and ignore the last few elements or what you want to do > about that; I don't know as you've not said. > > You're doing 66 values per outer loop but you said the input was 200 > rows in length--200/66 = 3.03... It's that .03 that's the problem. Or, of course, if you meant there were _roughly_ 200 rows and exactly 198, then that would solve the problem too... :) Rerun w/ x=[1:198]'; % and all should be well. If the real problem size isn't a multiple of 66 then you have the boundary condition issue to resolve as you wish...if it is, the no sweat. --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 21 Jun, 2012 14:41:07 Message: 14 of 18 It is a multiple of 66. So I changed it to 198 and got this: >> test3    1 2 3 4 5 6    7 8 9 10 11 12   13 14 15 16 17 18   19 20 21 22 23 24   25 26 27 28 29 30   31 32 33 34 35 36   37 38 39 40 41 42   43 44 45 46 47 48   49 50 51 52 53 54   55 56 57 58 59 60   61 62   66 67 68 69 70 71   72 73 74 75 76 77   78 79 80 81 82 83   84 85 86 87 88 89   90 91 92 93 94 95   96 97 98 99 100 101  102 103 104 105 106 107  108 109 110 111 112 113  114 115 116 117 118 119  120 121 122 123 124 125  126 127  131 132 133 134 135 136  137 138 139 140 141 142  143 144 145 146 147 148  149 150 151 152 153 154  155 156 157 158 159 160  161 162 163 164 165 166  167 168 169 170 171 172  173 174 175 176 177 178  179 180 181 182 183 184  185 186 187 188 189 190  191 192 Index exceeds matrix dimensions. Error in test3 (line 7)    fprintf(f1,x(i1:i2)) There's 2 things about this. a. The columns are exactly what I need. That is working very well. However, the second block starts at 66. This block needs to start at 63. So, I changed the last line to i1 = i2 + 1; and got this; >> test3    1 2 3 4 5 6    7 8 9 10 11 12   13 14 15 16 17 18   19 20 21 22 23 24   25 26 27 28 29 30   31 32 33 34 35 36   37 38 39 40 41 42   43 44 45 46 47 48   49 50 51 52 53 54   55 56 57 58 59 60   61 62   63 64 65 66 67 68   69 70 71 72 73 74   75 76 77 78 79 80   81 82 83 84 85 86   87 88 89 90 91 92   93 94 95 96 97 98   99 100 101 102 103 104  105 106 107 108 109 110  111 112 113 114 115 116  117 118 119 120 121 122  123 124  125 126 127 128 129 130  131 132 133 134 135 136  137 138 139 140 141 142  143 144 145 146 147 148  149 150 151 152 153 154  155 156 157 158 159 160  161 162 163 164 165 166  167 168 169 170 171 172  173 174 175 176 177 178  179 180 181 182 183 184  185 186 Index exceeds matrix dimensions. Error in test3 (line 7)    fprintf(f1,x(i1:i2)) It is now fine (as I require it to be), but I am still getting that error message. b. For this line; i2=i1+59; If it was 33 instead of 66, would it be this? i2=i1+26; % 6+1 (# columns + 1 = 7) less than the number (in this case 40)
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 21 Jun, 2012 18:46:37 Message: 15 of 18 On 6/21/2012 9:41 AM, Stan wrote: > It is a multiple of 66. So I changed it to 198 and got this: > >>> test3 > 1 2 3 4 5 6 > 7 8 9 10 11 12 > 13 14 15 16 17 18 ... > 55 56 57 58 59 60 > 61 62 > 66 67 68 69 70 71 > 72 73 74 75 76 77 ... > 185 186 187 188 189 190 > 191 192 > Index exceeds matrix dimensions. > > Error in test3 (line 7) > fprintf(f1,x(i1:i2)) > Sorry, typo/foobah in the conversion to the cleaner version...last i1=12+4; --> i1=12+5; > There's 2 things about this. > a. The columns are exactly what I need. That is working very well. > However, the second block starts at 66. Should have been 67 to make it work out even--11 rows of 6 (4 missing on the 11th) are 66 so next row starts at 67. This block needs to start at 63. Well, that's definitely going to screw up the count as 198 isn't going to divide out evenly now, for sure... > So, I changed the last line to i1 = i2 + 1; and got this; >>> test3 > 1 2 3 4 5 6 > 7 8 9 10 11 12 ... > 49 50 51 52 53 54 > 55 56 57 58 59 60 > 61 62 > 63 64 65 66 67 68 > 69 70 71 72 73 74 ... > 111 112 113 114 115 116 > 117 118 119 120 121 122 > 123 124 > 125 126 127 128 129 130 > 131 132 133 134 135 136 ... > 179 180 181 182 183 184 > 185 186 > Index exceeds matrix dimensions. > Error in test3 (line 7) > fprintf(f1,x(i1:i2)) > > It is now fine (as I require it to be), but I am still getting that > error message. And you've got the end effect of there aren't enough entries this way to do another full set...what you want to do w/ the remainder you've not yet said since you've changed the ground rule from what I thought it was above... You're doing 62 values per outer iteration;  >> rem(198,62) ans =      12  >> 12<62, ergo boom! I've told you several times you'll have to deal w/ the BC if it isn't even and it isn't. You have to decide how to deal w/ the boundary condition...if this is the final result that is desired, then you need to modify the while test condition to terminate when (i1+62) b. For this line; > i2=i1+59; > > If it was 33 instead of 66, would it be this? > i2=i1+26; % 6+1 (# columns + 1 = 7) less than the number (in this case 40) But w/ your new requirement, it's not 66 but 62. Work it out...it's not rocket science; just a little algebra...go thru w/ pencil and paper and look at how i1 and i2 have to update to get the sequences you want of how many at N per column and then M per column. --
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: dpb Date: 22 Jun, 2012 13:35:01 Message: 16 of 18 On 6/21/2012 1:46 PM, dpb wrote: ... > You're doing 62 values per outer iteration; > > >> rem(198,62) > ans = > 12 > >> > > 12<62, ergo boom! I've told you several times you'll have to deal w/ the > BC if it isn't even and it isn't. > > You have to decide how to deal w/ the boundary condition...if this is > the final result that is desired, then you need to modify the while test > condition to terminate when (i1+62)
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 28 Jun, 2012 01:37:09 Message: 17 of 18 My apologies. I re-read through the original post. As you indicated, there was a mistake with the boundary condition. The boundary condition is actually 186.i.e. there are exactly 186 elements in the row vector (62 X 3). So, based on this, I changed that last line to i1 = i2+1. I then went through the loop as you suggested and here is what I have come up with: Loop1: i1 = 1, i2 = i1 + 59 = 1 + 59 = 60 fprintf prints elements 1-60 on rows 1 to 10 on the 11th row, it prints 61 and 62 This is the end of loop1 Loop2: i1 = 63, i2 = 122 fprintf prints elements 63-123 on rows 12 to 21 on the 22nd row, it prints 61 and 62 This is the end of loop2 Loop3: i1 = 125, i2 = 125 + 59 = 184 fprintf prints elements 125-184 on rows 23 to 32 on the 33rd row, it prints 185 and 186 This is the end of loop3 Loop4: i1=187 ? this is > length(x) so the loop exits. fprintf is done printing. That is it for this vector, x. I would repeat the exact same process for 4 other vectors. If this makes sense, it would give me the desired output (same as what I displayed above – 3 blocks of 62). Does this seem reasonable or have I mis-accounted for the boundary condition? Also, regarding (i1+62)
 Subject: Data Manipulation (Re-arranging) and Writing to text file From: Stan Date: 28 Jun, 2012 01:52:09 Message: 18 of 18 Regarding my above pose, the code would be: x=[1:186]'; f1=[repmat('%4d',1,6) '\n']; f2=[repmat('%4d',1,2) '\n']; i1=1; while i1