How to concentrate matrices of different row length (same column length) into one matrix by unfolding each of the matrices to the smallest row length conatining numbers not nan

Hi all,

I am new to MATLAB, and I also asked previously something similar and got pretty good answer but I am not quite sure how to solve the issue which I didn't explain quite well. So, I have some 1010 matrices which have different row lengths but same number of columns. So I have matrices lets say 75x50, 73x50, 74x50, 85x50, 83x50, etc. But within each matrix, lets say the first one 75x50, columns have different lengths with numbers and then are filled with nan. Firstly, I want to crop them all into the smallest length of the column which contains only numbers and then in the other case extend all the matrices to the largest row length when filling with nan. I want to unfold matrices and concetrate them in a single matrix. Below the code crops the matrices at the smallest length of rows but not the smallest length of rows where all the columns contain only numbers and not nan.

Thank you in advance!

%% Initialize variables.
d='directory';

for i=1:1010
filename = [d sprintf('/name%05d/csvnames.csv', i)];
delimiter = ',';
startRow = 2;

%% Create output variable
frametable1final1 = cell2mat(raw);

     H(i).matrix = frametable1final1(:,[2 3 13 14 15 16 17 18 19 20 24 36 48 49 50 58 59 60 72 73 74 75 76 88 89 90 91 99 263 316 317 318 320 344 357 870 910 927 928 929 948 949 950 969 970 971 975 976 977 981]); 
        
     
end

 minheight = min(arrayfun(@(s) size(s.matrix, 1), H));  %get height of each matrix and get minimum
 trimmed = arrayfun(@(s) reshape(s.matrix(1:minheight, :), 1, []), H, 'UniformOutput', false);  %trim and reshape matrices
 X = vertcat(trimmed{:});  %vertically concatenate the lot
 
 maxheight = max(arrayfun(@(s) size(s.matrix, 1), H));  %get height of each matrix and get maximum
 padded = arrayfun(@(s) reshape([s.matrix; nan(maxheight - size(s.matrix, 1), size(s.matrix, 2))], 1, []), H, 'UniformOutput', false); %pad and reshape
 Xprime = vertcat(padded{:});

Answer 1

[Andrei Bobrov]

M = struct2cell(H);
n = min(cellfun(@(x)find(all(~isnan(x),2),1,'last'),M));
M = cellfun(@(x)reshape(x(1:n,:)',1,[]),M,'un',0);
X = cat(1,M{:})

and (part 2 - fixed)

M = struct2cell(H);
[m,n] = cellfun(@size,M);
mm = max(m);
n = max(n);
for jj = 1:numel(H)
    H(jj).matrix(isnan(H(jj).matrix)) = 0;
    if m(jj) < mm
        H(jj).matrix(mm,n) = 0;
    end
end

[Spresi]

Hi, thanks a lot for your answer. The first one works very well.

The second one however gives maximal matrices but everything filled with 0. But I actually want to fill with 0 only when I have NaN in matrices. So final matrices must contain either the actual number value it had before or if was NaN then it should be 0. Also extending shorter matrices to the larger ones must be filled with 0.

I think the correct answer for the second part should be:

for jj = 1:numel(H)
    H(jj).matrix(isnan(H(jj).matrix)) = 0;
end

 maxheight = max(arrayfun(@(s) size(s.matrix, 1), H));  %get height of each matrix and get maximum
 padded = arrayfun(@(s) reshape(normalize([s.matrix; zeros(maxheight - size(s.matrix, 1), size(s.matrix, 2))],'center','mean'), 1, []), H, 'UniformOutput', false); %pad and reshape
 Xprime = vertcat(padded{:});

[Andrei Bobrov]

I fixed "part 2" in my answer.

Answer 2

[Kelly Kearney]

Do any of your matrices have non-trailing blank/all-NaN lines that you want to preserve? Or are you just trying to strip out the NaN lines altogether?

If the former, you can do this pretty simply with a loop:

% Some example data

H(1).matrix = [...
    1 2 3
    NaN NaN NaN
    4 5 6
    NaN NaN NaN];
H(2).matrix = [...
    7 8 9
    10 11 12
    13 14 15
    16 17 18];
H(3).matrix = [19 20 21; nan(20,3)];

% Remove only trailing all-NaN rows

newh = cell(size(H));
for ii = 1:length(H)
    [ridx,~] = find(~isnan(H(ii).matrix)); % identify row index of all numbers
    newh{ii} = H(ii).matrix(1:max(ridx),:); % truncate matrix to max row with number
end
newh = cat(1, newh{:}) % concatenate

If you don't need to preserve any all-NaN rows, you can concatenate first and then just strip out the unwanted rows:

newh = cat(1, H.matrix); % concatenate
newh = newh(~all(isnan(newh),2),:) % remove all-NaN rows

[Spresi]

Thank you so much for your answer. But your code assumes that I have rows which are NaN and removes them, but thats not actually my issue. I am trying to simplify my problem below:

I have matrices like this:

H(1).matrix = [1 2 3
               3 4 5
               3 4 4
               6 NaN 7
               NaN NaN NaN]
           
H(2).matrix = [3 4 5
               5 6 7
               4 4 4
               NaN 3 3
               NaN 4 4
               NaN NaN 7]
H(3).matrix = [3 3 3
               2 1 3
               NaN 1 2]
           

Now, I see that these matrices all have 3 columns but different row number, but also within columns they have different row length with actual numbers and not NaN. I see that the shortest row length where each columns have non NaN values is two (third matrix has the first row and the second row with all non NaN values). So now i crop all the matrices to have size 2x3, then unfold them, then concentrate them in a single big matrix.

In other words

               H(1).matrix = [1 2 3 3 4 5] %unfold to the shortest row with non nan
               H(2).matrix = [3 4 5 5 6 7]
               H(3).matrix = [3 3 3 2 1 3]
               
               X = [1 2 3 3 4 5
                    3 4 5 5 6 7
                    3 3 3 2 1 3] %concentrate them in a single matrix

And in the other case I want to extend all the matrices to size 6x3 (second matrix dimensions-biggest one) by substituting all the existing NaN values to 0 and adding 0 to other two matrices which have dimensions 5x3 and 3x3.

                H(1).matrix = [1 2 3 
                               3 4 5 
                               3 4 4 
                               6 0 7
                               0 0 0
                               0 0 0]
                H(2).matrix = [3 4 5
                               5 6 7
                               4 4 4 
                               0 3 3 
                               0 4 4 
                               0 0 7]
                H(3).matrix = [3 3 3
                               2 1 3
                               0 1 2
                               0 0 0
                               0 0 0
                               0 0 0]

Thank you again!