Thread Subject:

Data Manipulation (Re-arranging) and Writing to text file

Hello, here’s the details of the problem that I am writing about, and my attempts at a solution:

a. I need to extract elements from rows 1-6 from the first column of a file (input.txt, I’ve loaded the file successfully). This file has numerous columns with 200 rows each. I am interested in only the first column. Then I need to print these extracted elements to row 1 of a text file, separated (delimited) by a tab.

b. I need to then repeat this in succession 10 times.i.e. extract 7-12 and print them on row 2 of the text file (tab delimited), then extract 13-18 and print them on row 3 of the text file (tab delimited).

c. Now, the 11th time (i.e. row 11), I need only 2 elements (rows 61 and 62) to be extracted and printed on row 11. The 4 other spaces (where elements 63, 64, 65, 66 would go) should be left empty (not zeros, but empty).

MY ATTEMPTS:
a. Here’s what I have come up with (neither is printing more than 1 row, but rather just printing the last row that is extracted). I like the first method (taking the transpose, it seems to be working), but I can’t get it to print as I go along through the loop.

First Approach (extract vectors using a loop, use dlmwrite to write them to text file):
eee = importdata('input.txt');
 
for i = 1:6:19
        aa = eee.data(i:i+5,1);
        dlmwrite('test3.txt',aa','\t')
end

Second Approach (extract elements indiviually and write them to the text file using fprintf):
eee = importdata('input.txt');
 
sav_fid = fopen('test4.txt','w');
 
for i = 1:6
        aa(i) = eee.data(i,1);
        fprintf(sav_fid,'%f',aa(i));
end

b. I am thinking I would need to do this:
 for j = 1:6:66

 

 

 end

I would put this around the loop in a. Is this correct or is there is a better way to accomplish this step?

c. If this was a 1-D array, I would do it easily using:

rem = [r1 r2 r3]
A(rem) = []

I’m just not sure if this would work if step b. is not working. Is there a better way to do this?

Regards:
Stan T.

On 6/17/2012 10:19 PM, Stan wrote:
...

> a. I need to extract elements from rows 1-6 from the first column of a
> file (input.txt, I’ve loaded the file successfully). This file has
> numerous columns with 200 rows each. I am interested in only the first
> column. Then I need to print these extracted elements to row 1 of a text
> file, separated (delimited) by a tab.

x=load(...): % you say you've got that
x(:,2:end)=[]; % retain column only 1 for simplicity in addressing

> b. I need to then repeat this in succession 10 times.i.e. extract 7-12
> and print them on row 2 of the text file (tab delimited), then extract
> 13-18 and print them on row 3 of the text file (tab delimited).
>
> c. Now, the 11th time (i.e. row 11), I need only 2 elements (rows 61 and
> 62) to be extracted and printed on row 11. The 4 other spaces (where
> elements 63, 64, 65, 66 would go) should be left empty (not zeros, but
> empty).

x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6

Now om the next step you've gotten yourself into a bind...you can't have
a "ragged" array w/ only 2 values in some rows and 6 in all the rest.

So, you've got one of two choices--go to a cell array (ugly in this case
I think) or process the output in pieces writing the full rows where
wanted and less for the others.

--

Hi dpb:

> So, you've got one of two choices--go to a cell array (ugly in this case
> I think) or process the output in pieces writing the full rows where
> wanted and less for the others.
>

The processing of the output in pieces would be much better. But I'm not sure how I would indicate that a row needs to be truncated. Wouldn't you need to specify a fixed row end ---> 6, in this case?

I think I'm just not seeing it.

On 6/18/2012 1:21 AM, Stan wrote:
> Hi dpb:
>
>> So, you've got one of two choices--go to a cell array (ugly in this
>> case I think) or process the output in pieces writing the full rows
>> where wanted and less for the others.
>>
>
> The processing of the output in pieces would be much better. But I'm not
> sure how I would indicate that a row needs to be truncated. Wouldn't you
> need to specify a fixed row end ---> 6, in this case?
>
> I think I'm just not seeing it.

You have some logic to decide which are/aren't...a loop somewhat like
the one you did (mod() comes to mind to use for the test if it is every
n-th one).

You can write the size of the array that is contiguous and full in one
subset w/ fprintf, then the other w/ the proper subset of indices.

W/ the offset arguments, I guess dlmwrite() should work as well altho
I've not tested that.

--

> x(:,2:end)=[]; % retain column only 1 for simplicity in addressing
.
.
.
> x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6

Is it possible to extract a subset of a matrix, where you indicate the range of matrix elements to be extracted? I mean, after aplying the second command (reshaping which gave rows of length 6), could the first 11 rows be extracted by specifying (not the row numbers, but) the range of matrix elements to be extracted - in this case [1:66]?

On 6/18/2012 9:48 AM, Stan wrote:
>> x(:,2:end)=[]; % retain column only 1 for simplicity in addressing
> ..
> ..
> ..
>> x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6
>
> Is it possible to extract a subset of a matrix, where you indicate the
> range of matrix elements to be extracted? I mean, after aplying the
> second command (reshaping which gave rows of length 6), could the first
> 11 rows be extracted by specifying (not the row numbers, but) the range
> of matrix elements to be extracted - in this case [1:66]?

Of course, but the linear address of the first 66 isn't the same as the
first 11 rows as Matlab storage order is column-major.

But, all you need is to write out

x(1:M,:)

for whatever value of M you need and then the one or two or however many
that aren't full as

x(M+N,1:2) % for N = 1, 2, ... , whatever it is you want

If there are more remaining of the same pattern then put this in a loop
and increment by M+N for the first row index. If the pattern changes,
then you'll need to add the logic to compute the M and N as you iterate
through as well, but if you know what it is, then that should be no
problem even if you have to do it as a lookup table because it's irregular.

NB, of course, that owing to the column major storage order you'll need
to use the ' (transpose) operator on the submatrices to write them in
rows as displayed on the screen to the file...

--

On 6/18/2012 10:24 AM, dpb wrote:
...

> But, all you need is to write out
>
> x(1:M,:)
>
> for whatever value of M you need and then the one or two or however many
> that aren't full as
>
> x(M+N,1:2) % for N = 1, 2, ... , whatever it is you want
...

x(M+1+N,1:2) % for N = 1, 2, ... , whatever it is you want
    __

NB the +1...

--

On 6/18/2012 9:48 AM, Stan wrote:
>> x(:,2:end)=[]; % retain column only 1 for simplicity in addressing
> ..
> ..
> ..
>> x=reshape(x,6,length(x)/6)'; % now you've got rows of length 6
>
> Is it possible to extract a subset of a matrix, where you indicate the
> range of matrix elements to be extracted? I mean, after aplying the
> second command (reshaping which gave rows of length 6), could the first
> 11 rows be extracted by specifying (not the row numbers, but) the range
> of matrix elements to be extracted - in this case [1:66]?

Well, my previous posting reminding you to transpose the transpose made
me slap me head... :)

Try this at the command window and see if it isn't what you're looking
for...

x=[1:200]';
f1=[repmat('%4d',1,6) '\n'];
f2=[repmat('%4d',1,2) '\n'];
ix=1;
while
   ix<length(x)
   fprintf(f1,x(ix:ix+59))
   fprintf(f2,x(ix+60:ix+61))
   ix=ix+66;
end

If the vector is actually 200 entries you've got a boundary condition to
fix up on the last row--you'll have to decide what you want done with it.

If that's printing the right elements in the order desired to the screen
then replace the default fprintf to the terminal w/ a valid fid and voila!

--

On 6/18/2012 12:24 PM, dpb wrote:
...

> x=[1:200]';
> f1=[repmat('%4d',1,6) '\n'];
> f2=[repmat('%4d',1,2) '\n'];
> ix=1;
> while
> ix<length(x)
> fprintf(f1,x(ix:ix+59))
> fprintf(f2,x(ix+60:ix+61))
> ix=ix+66;
> end
...

Just a minor cleanup of a formatting problem and a few fewer magic
numbers...

i1=1;
while i1<length(x)
   i2=i1+59;
   fprintf(f1,x(i1:i2))
   i1=i2+1; i2=i1+1;
   fprintf(f2,x(i1:i2))
   i1=i2+4;
end

--

^^^
I'm going through your code and will test if after that. I think I'm starting to get a little more comfortable with some of the thngs you are saying (knock on wood). It may take me some time to reply though.

I've gone through this. Two questions:

1. Could you explain the two fprintf lines and the first repmat line?

2. I'm getting the following error message:

Index exceeds matrix dimensions.

Error in test3 (line 7)
   fprintf(f1,x(i1:i2))

I'm using MATLAB 2012a. Could there be a version conflict?

> dpb <none@non.net> wrote in message <jro45e$lh6$1@speranza.aioe.org>...
>
> Just a minor cleanup of a formatting problem and a few fewer magic
> numbers...
>
> i1=1;
> while i1<length(x)
> i2=i1+59;
> fprintf(f1,x(i1:i2))
> i1=i2+1; i2=i1+1;
> fprintf(f2,x(i1:i2))
> i1=i2+4;
> end
>
> --

On 6/20/2012 1:02 AM, Stan wrote:

...[top posting repaired...please don't; hard follow conversation makes]...

>> dpb <none@non.net> wrote in message <jro45e$lh6$1@speranza.aioe.org>...
>>
>> Just a minor cleanup of a formatting problem and a few fewer magic
>> numbers...
>>
>> i1=1;
>> while i1<length(x)
>> i2=i1+59;
>> fprintf(f1,x(i1:i2))
>> i1=i2+1; i2=i1+1;
>> fprintf(f2,x(i1:i2))
>> i1=i2+4;
>> end
>>
>> --

 > I've gone through this. Two questions:
 >
 > 1. Could you explain the two fprintf lines and the first repmat line?

Try the output of repmat() at the command line...each is just building a
format string for the specific number of elements to write per line.
(This is a remnant of the C-like formatting strings that lack a repeat
specifier...)

fprintf() simply outputs the number of elements from x w/ the format
specified. Since you have a single column vector, linear addressing of
i1:i1 picks up the number you requested--first w/ 6 elements/row, then a
row of 2 and skip four from the list...

 > 2. I'm getting the following error message:
 >
 > Index exceeds matrix dimensions.
 >
 > Error in test3 (line 7)
 > fprintf(f1,x(i1:i2))
 >
 > I'm using MATLAB 2012a. Could there be a version conflict?

No, that's because the length of the vector isn't evenly divisible by
the number of elements you asked to be printed in each pass...it's the
boundary condition I mentioned earlier. You have to decide whether to
abort early and ignore the last few elements or what you want to do
about that; I don't know as you've not said.

You're doing 66 values per outer loop but you said the input was 200
rows in length--200/66 = 3.03... It's that .03 that's the problem.

--

On 6/20/2012 6:59 AM, dpb wrote:
...

> fprintf() simply outputs the number of elements from x w/ the format
> specified. Since you have a single column vector, linear addressing of
> i1:i1 picks up the number you requested--first w/ 6 elements/row, then a
> row of 2 and skip four from the list...

That is i1:i2 of course...

> > 2. I'm getting the following error message:
> >
> > Index exceeds matrix dimensions.
...

> No, that's because the length of the vector isn't evenly divisible by
> the number of elements you asked to be printed in each pass...it's the
> boundary condition I mentioned earlier. You have to decide whether to
> abort early and ignore the last few elements or what you want to do
> about that; I don't know as you've not said.
>
> You're doing 66 values per outer loop but you said the input was 200
> rows in length--200/66 = 3.03... It's that .03 that's the problem.

Or, of course, if you meant there were _roughly_ 200 rows and exactly
198, then that would solve the problem too... :)

Rerun w/

x=[1:198]'; % and all should be well.

If the real problem size isn't a multiple of 66 then you have the
boundary condition issue to resolve as you wish...if it is, the no sweat.

--

It is a multiple of 66. So I changed it to 198 and got this:

>> test3
   1 2 3 4 5 6
   7 8 9 10 11 12
  13 14 15 16 17 18
  19 20 21 22 23 24
  25 26 27 28 29 30
  31 32 33 34 35 36
  37 38 39 40 41 42
  43 44 45 46 47 48
  49 50 51 52 53 54
  55 56 57 58 59 60
  61 62
  66 67 68 69 70 71
  72 73 74 75 76 77
  78 79 80 81 82 83
  84 85 86 87 88 89
  90 91 92 93 94 95
  96 97 98 99 100 101
 102 103 104 105 106 107
 108 109 110 111 112 113
 114 115 116 117 118 119
 120 121 122 123 124 125
 126 127
 131 132 133 134 135 136
 137 138 139 140 141 142
 143 144 145 146 147 148
 149 150 151 152 153 154
 155 156 157 158 159 160
 161 162 163 164 165 166
 167 168 169 170 171 172
 173 174 175 176 177 178
 179 180 181 182 183 184
 185 186 187 188 189 190
 191 192
Index exceeds matrix dimensions.

Error in test3 (line 7)
   fprintf(f1,x(i1:i2))



There's 2 things about this.
a. The columns are exactly what I need. That is working very well. However, the second block starts at 66. This block needs to start at 63. So, I changed the last line to i1 = i2 + 1; and got this;
>> test3
   1 2 3 4 5 6
   7 8 9 10 11 12
  13 14 15 16 17 18
  19 20 21 22 23 24
  25 26 27 28 29 30
  31 32 33 34 35 36
  37 38 39 40 41 42
  43 44 45 46 47 48
  49 50 51 52 53 54
  55 56 57 58 59 60
  61 62
  63 64 65 66 67 68
  69 70 71 72 73 74
  75 76 77 78 79 80
  81 82 83 84 85 86
  87 88 89 90 91 92
  93 94 95 96 97 98
  99 100 101 102 103 104
 105 106 107 108 109 110
 111 112 113 114 115 116
 117 118 119 120 121 122
 123 124
 125 126 127 128 129 130
 131 132 133 134 135 136
 137 138 139 140 141 142
 143 144 145 146 147 148
 149 150 151 152 153 154
 155 156 157 158 159 160
 161 162 163 164 165 166
 167 168 169 170 171 172
 173 174 175 176 177 178
 179 180 181 182 183 184
 185 186
Index exceeds matrix dimensions.

Error in test3 (line 7)
   fprintf(f1,x(i1:i2))

It is now fine (as I require it to be), but I am still getting that error message.

b. For this line;
i2=i1+59;

If it was 33 instead of 66, would it be this?
i2=i1+26; % 6+1 (# columns + 1 = 7) less than the number (in this case 40)

On 6/21/2012 9:41 AM, Stan wrote:
> It is a multiple of 66. So I changed it to 198 and got this:
>
>>> test3
> 1 2 3 4 5 6
> 7 8 9 10 11 12
> 13 14 15 16 17 18
...
> 55 56 57 58 59 60
> 61 62
> 66 67 68 69 70 71
> 72 73 74 75 76 77
...

> 185 186 187 188 189 190
> 191 192
> Index exceeds matrix dimensions.
>
> Error in test3 (line 7)
> fprintf(f1,x(i1:i2))
>

Sorry, typo/foobah in the conversion to the cleaner version...last

i1=12+4; --> i1=12+5;

> There's 2 things about this.
> a. The columns are exactly what I need. That is working very well.
> However, the second block starts at 66.

Should have been 67 to make it work out even--11 rows of 6 (4 missing on
the 11th) are 66 so next row starts at 67.

This block needs to start at 63.

Well, that's definitely going to screw up the count as 198 isn't going
to divide out evenly now, for sure...

> So, I changed the last line to i1 = i2 + 1; and got this;
>>> test3
> 1 2 3 4 5 6
> 7 8 9 10 11 12
...
> 49 50 51 52 53 54
> 55 56 57 58 59 60
> 61 62
> 63 64 65 66 67 68
> 69 70 71 72 73 74
...
> 111 112 113 114 115 116
> 117 118 119 120 121 122
> 123 124
> 125 126 127 128 129 130
> 131 132 133 134 135 136
...
> 179 180 181 182 183 184
> 185 186
> Index exceeds matrix dimensions.
> Error in test3 (line 7)
> fprintf(f1,x(i1:i2))
>
> It is now fine (as I require it to be), but I am still getting that
> error message.

And you've got the end effect of there aren't enough entries this way to
do another full set...what you want to do w/ the remainder you've not
yet said since you've changed the ground rule from what I thought it was
above...

You're doing 62 values per outer iteration;

 >> rem(198,62)
ans =
     12
 >>

12<62, ergo boom! I've told you several times you'll have to deal w/
the BC if it isn't even and it isn't.

You have to decide how to deal w/ the boundary condition...if this is
the final result that is desired, then you need to modify the while test
condition to terminate when (i1+62)<length(x)

> b. For this line;
> i2=i1+59;
>
> If it was 33 instead of 66, would it be this?
> i2=i1+26; % 6+1 (# columns + 1 = 7) less than the number (in this case 40)

But w/ your new requirement, it's not 66 but 62.

Work it out...it's not rocket science; just a little algebra...go thru
w/ pencil and paper and look at how i1 and i2 have to update to get the
sequences you want of how many at N per column and then M per column.

--

On 6/21/2012 1:46 PM, dpb wrote:
...

> You're doing 62 values per outer iteration;
>
> >> rem(198,62)
> ans =
> 12
> >>
>
> 12<62, ergo boom! I've told you several times you'll have to deal w/ the
> BC if it isn't even and it isn't.
>
> You have to decide how to deal w/ the boundary condition...if this is
> the final result that is desired, then you need to modify the while test
> condition to terminate when (i1+62)<length(x)
...

NB the above test "throws away" the last 12 elements to leave a
symmetric-looking result. If you want to do something w/ them as well,
then you'll have to do that specifically for them based on the size and
formatting desired. That's the boundary (or some would use edge or end)
condition...

--

My apologies. I re-read through the original post. As you indicated, there was a mistake with the boundary condition.

The boundary condition is actually 186.i.e. there are exactly 186 elements in the row vector (62 X 3).
So, based on this, I changed that last line to i1 = i2+1. I then went through the loop as you suggested and here is what I have come up with:

Loop1:
i1 = 1, i2 = i1 + 59 = 1 + 59 = 60
fprintf prints elements 1-60 on rows 1 to 10
on the 11th row, it prints 61 and 62
This is the end of loop1

Loop2:
i1 = 63, i2 = 122
fprintf prints elements 63-123 on rows 12 to 21
on the 22nd row, it prints 61 and 62
This is the end of loop2

Loop3:
i1 = 125, i2 = 125 + 59 = 184
fprintf prints elements 125-184 on rows 23 to 32
on the 33rd row, it prints 185 and 186
This is the end of loop3

Loop4:
i1=187 ? this is > length(x) so the loop exits.

fprintf is done printing.

That is it for this vector, x. I would repeat the exact same process for 4 other vectors.

If this makes sense, it would give me the desired output (same as what I displayed above – 3 blocks of 62). Does this seem reasonable or have I mis-accounted for the boundary condition?

Also, regarding (i1+62)<length(x):
This would no longer be needed. Is this correct or is the loop not going to exit correctly?

Regarding my above pose, the code would be:

x=[1:186]';
f1=[repmat('%4d',1,6) '\n'];
f2=[repmat('%4d',1,2) '\n'];
i1=1;
while i1<length(x)
i2=i1+59;
fprintf(f1,x(i1:i2))
i1=i2+1; i2=i1+1;
fprintf(f2,x(i1:i2))
i1=i2+1;
end