read multiple .csv files in 'for loop', extract and write data to .txt

I am having some difficulties reading multiple .csv files in a 'for loop', extracting and writing data to .txt My code is below. There are ~150 .csv files in a single folder. I want to extract selected rows/columns from each file (the same row and column #'s in every file) then write the extracted data to .txt file with ~150 readings in 3 columns for processing. Any help or advice is appreciated. My code below only extracts one data point.

% Read multiple .csv noise files (LD831) LCeq-LAeq values and 1/3-OB
files = dir('*.csv');
% Then iterate over the files struct to read each file.
for i=1:length(files)
    %data = readmatrix(files(i).name); % or csvread(files(i).name)
    % process data
    data = xlsread(files(i).name, 'B81:AK82'); % read noise file 1/3-OB data
    LC_LA = xlsread(files(i).name, 'B63:B63'); % LCeq - LAeq reading
    % Extract dB values for VLFN and LFN
    VLFN  = 10*log10(10.^(data(2,1)/10)+10.^(data(2,2)/10)+10.^(data(2,3)/10)+10.^(data(2,4)/10)+10.^(data(2,5)/10)+10.^(data(2,6)/10));
    LFN = 10*log10(10.^(data(2,7)/10)+10.^(data(2,8)/10)+10.^(data(2,9)/10)+10.^(data(2,10)/10)+10.^(data(2,11)/10)+10.^(data(2,12)/10)+10.^(data(2,13)/10)+10.^(data(2,14)/10));
    %MFN =  
    %HFN =
    writematrix(LC_LA,'deltaC_Rox.txt'); % save to txt file
    % need to add VLFN, LFN values to .txt file ...HOW?
end

[Stephen23]

You can set the WRITEMATRIX option 'WriteMode' to 'append' and then carefully specify the range you wish to write to each time:

https://www.mathworks.com/help/matlab/ref/writematrix.html#namevaluepairarguments

But most likely it would be simpler to collect the data into one array within the loop and save it once after the loop.

[Doug Leaffer]

adding 'WriteMode', 'append' to the code di not result in a compilation vector of additional values culled from each .csv file

Answer 1

[Benjamin Thompson]

So building on Stephen's comment. You may also get better results using readmatrix or readtable than xlsread.

% Read multiple .csv noise files (LD831) LCeq-LAeq values and 1/3-OB

files = dir('*.csv');

outputmatrix = zeros(length(files),3) % Not sure how many columns here but you said three

% Then iterate over the files struct to read each file.

for i=1:length(files)

%data = readmatrix(files(i).name); % or csvread(files(i).name)

% process data

data = xlsread(files(i).name, 'B81:AK82'); % read noise file 1/3-OB data

LC_LA = xlsread(files(i).name, 'B63:B63'); % LCeq - LAeq reading

% Extract dB values for VLFN and LFN

VLFN = 10*log10(10.^(data(2,1)/10)+10.^(data(2,2)/10)+10.^(data(2,3)/10)+10.^(data(2,4)/10)+10.^(data(2,5)/10)+10.^(data(2,6)/10));

LFN = 10*log10(10.^(data(2,7)/10)+10.^(data(2,8)/10)+10.^(data(2,9)/10)+10.^(data(2,10)/10)+10.^(data(2,11)/10)+10.^(data(2,12)/10)+10.^(data(2,13)/10)+10.^(data(2,14)/10));

%MFN =

%HFN =

outputmatrix(i,:) = [data LC_LA];

% need to add VLFN, LFN values to .txt file ...HOW?

end

writematrix(outputmatrix,'deltaC_Rox.txt'); % save to txt file

[Doug Leaffer]

Thank you for you comments Ben. This only resulted in more errors. Also - readtable and readmatrix did not work

[Benjamin Thompson]

Without having all the data files I was only guessing but glad you ended up with a good result anyway.

Answer 2

[Doug Leaffer]

The code below seems to work for my needs. Thanks for your input Steven and Benjamin.

%% READ CSV Noise Files LCeq-LAeq and VLFN, LFN, MFN, HFN
% Get a list of all txt files in the current folder, or subfolders of it.

fds = fileDatastore('*.csv', 'ReadFcn', @importdata)
fullFileNames = fds.Files
numFiles = length(fullFileNames)
% Loop over all files reading them in and plotting them.
for k = 1 : numFiles
    fprintf('Now reading file %s\n', fullFileNames{k});
    data = xlsread(fullFileNames{k}, 'B81:AK82'); % read noise file data
    LC_LA = xlsread(fullFileNames{k}, 'B63:B63'); % LCeq - LAeq reading
% Extract dB values for VLFN and LFN
VLFN  = 10*log10(10.^(data(2,1)/10)+10.^(data(2,2)/10)+10.^(data(2,3)/10)+10.^(data(2,4)/10)+10.^(data(2,5)/10)+10.^(data(2,6)/10));
LFN = 10*log10(10.^(data(2,7)/10)+10.^(data(2,8)/10)+10.^(data(2,9)/10)+10.^(data(2,10)/10)+10.^(data(2,11)/10)+10.^(data(2,12)/10)+10.^(data(2,13)/10)+10.^(data(2,14)/10));
%MFN = 
%HFN = 
%
writematrix(LC_LA, 'delta_C_Rox.txt', 'WriteMode', 'append'); % save to txt file
writematrix(VLFN, 'VLFN_Rox.txt', 'WriteMode', 'append');
writematrix(LFN, 'LFN_Rox.txt', 'WriteMode', 'append');
%vwritematrix(MFN, 'LFN_Rox.txt', 'WriteMode', 'append');
%vwritematrix(HFN, 'LFN_Rox.txt', 'WriteMode', 'append');
% 
end

vlfn = readmatrix('VLFN_Rox.txt');
lfn = readmatrix('LFN_Rox.txt');
%mfn = readmatrix('MFN_Rox.txt');
%hfn = readmatrix('HFN_Rox.txt');
deltaC = readmatrix('delta_C_Rox.txt');
%
T = table(deltaC, vlfn, lfn); %ADD mfn, hfn);