Looping through tables in my workspace, plotting them, and saving a picture of the plot.

Hello, in my workspace I have tables, they all contain a column for time and a column for signal. In the code I have attached I am trying to loop through the tables in the workspace, plot them onto a figure and save the resultant figure. However, the code keeps getting stuck at:

FP.rawdata(:,1) = currentTable.time;

Unrecognized function or variable 'tableName'.

The error message I receive is:

Dot indexing is not supported for variables of this type. I think where

currentTable = evalin('base', tableName);

the evalin is not currently assigning the table I am looking at into the temporary currentTable variable. Which is weird because when I try running the lines one by one in the command window, currentTable is correctly displaying the contents of tableName(1).

If someone could help trouble shoot this code help me figure this out I will greatly appreciate it:

I have attached the code

tableNames = who;
    
    for i = 1:length(tableNames)
        
        
            
        tableName = tableNames{i};

            
                
        currentTable = evalin('base', tableName);

        
        FP.rawdata(:,1) = currentTable.time;
        FP.rawdata(:,2) = currentTable.signal;
                    
        startpoint = FP.rawdata(1,1);
        FP.rawdata(:,1) = (FP.rawdata(:,1) - startpoint);
                
        FP.calcium_dependent(:,1) = downsample(FP.rawdata(2:end,1),2); 
        FP.calcium_dependent(:,2) = downsample(FP.rawdata(2:end,2),2);
                
                    
        FP.isosbestic(:,1) = downsample(FP.rawdata(:,1),2);
        FP.isosbestic(:,2) = downsample(FP.rawdata(:,2),2);
                
        temp_fit1 = fit(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2),'exp2');
                
        temp_fit2 = fit(FP.isosbestic(:,1),FP.isosbestic(:,2),'exp2');
                    
        %Generate and save figure as a picture
                
        figure(1)
                   
        plot(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2)-temp_fit1(FP.calcium_dependent(:,1)),'b');
                
        hold on
                
                   
        plot(FP.isosbestic(:,1),FP.isosbestic(:,2)-temp_fit2(FP.isosbestic(:,1)),'r');
                
        grid on
                
        ylabel({'Linearized Raw fluorescence'});
        xlabel({'Time (seconds)'});
        title({tableName ', ACh SNFR in dHPC'});
        legend ACh-dependent-signal Isosbestic
                
        saveas(gcf, [tableName '_plot.png']);
                    
                    
        close all
        clear FP

    end
    
    

[Stephen23]

Having lots of variables in the workspace and then trying to access them dynamically is the immediate cause. That in turn is the result of the bad design decision to force meta-data into the variable names. Best avoided:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

You can avoid the whole thing by e.g. using arrays and indexing.

Note that meta-data (e.g. filenames, test parameters, etc) is data, and data is best stored in variables (not in variable names)... and then you can start to write simpler, more robust code that does not rely on evil EVALIN or ASSIGNIN.

Answer 1

[Voss]

The error message, "Dot indexing is not supported for variables of this type." indicates that some code is trying to access a column of a table (or a field of a struct), but the variable is not a table (not a struct, respectively). I suspect that happens on the line you point out:

FP.rawdata(:,1) = currentTable.time;

and it happens because currentTable is not a table.

This would happen if your workspace contains non-table variables in addition to the tables you care about. For example, perhaps you start with a workspace containing only tables but then you run your script, which populates the workspace with some other non-table variables as well that don't get cleared, e.g., temp_fit_1, temp_fit_2; then when you run the script again, those non-table variables still exist, their names are stored in tableNames, your code tries to treat them as tables even though they are not, and you get the error.

One way to fix this would be to make sure tableNames contains only names of tables (i.e., no names of any variables of any other class) in the workspace, and one way to do that is to clear the non-tables before calling tableNames = who. Something like this:

% clear the non-table variables:
S = whos();
S(strcmp({S.class},'table')) = [];
vars_to_clear = {S.name};
if ~isempty(vars_to_clear)
    clear(vars_to_clear{:});
end
clear S vars_to_clear

% get the table names:
tableNames = who;

% etc. (rest of code the same)

[Logan]

Hey thanks for the responding and for taking the time to look into this! So I tried something similar to what you proivded but I did it manually and just put "clear data temp_fit_1 temp_fit_2 etc..." Because to your point and Matthew's from above, there might still be non-table variables. Its weird also because if you run these lines, one by one in the command window:

tableNames = who;
   
 tableName = tableNames{1};
       
 currentTable = evalin('base', tableName);

currentTable gets turned into a table but when its in the for loop it doesnt??

I avoided including this because I thought it wasnt a part of the issue and would just cause confusion on here but there is more to the code I included. So the function I have intends to import csv files from my working folder and names them according to the naming structure on the csv files: for example, "0G_B5401right_1G_B5404right_pwriter.csv" is a csv that contains multiple columns. However given the naming structure, I want one table called B5401right and this table will only contain the Timestamp column and the Region0G column. Whereas the second table being generated will be called B5405right and will contain the Timestamp column and the Region1G column. The part of the code that does this works perfeclty fine, and I'll include it here as well as a folder I am testing it with that contains the csv files. But the second part of my function which plots the tables and saves the pictures is giving me the issue above. I hope this doesnt make things too confusing but brings more context to the issue in case the issue lies in this first part of the code. Thank you again for your time and help with this issue.

I can't upload a folder but I put the csv files in my test folder.

The input to the function is the path to the folder:

Example: processFiles('/Users/logan/Desktop/folder_name')

 function processFiles(folderName)
    
% Check if the folder exists
    
    if ~isfolder(folderName)
        error('Folder not found.');
    end
    
    % Get a list of all CSV files in the folder
    files = dir(fullfile(folderName, '*.csv'));
    
    for i = 1:length(files)
        % Get the file name and split it by underscores
        fileName = files(i).name;

        contains0G = contains(fileName, '0G');
        contains1G = contains(fileName, '1G');
        
        if contains0G && contains1G

            parts = strsplit(fileName, '_');
            prefix1 = parts{1};
            prefix2 = parts{3};
            table1Name = parts{2};
            table2Name = parts{4};

            %read csv file
            data = readtable(fullfile(folderName, fileName));
            
            %Create tables
            assignin('base', table1Name, table(data.Timestamp, data.(['Region' prefix1]), 'VariableNames', {'time', 'signal'}));
            assignin('base', table2Name, table(data.Timestamp, data.(['Region' prefix2]), 'VariableNames', {'time', 'signal'}));
        
        elseif contains0G
            
            parts = strsplit(fileName, '_');
            prefix1 = parts{1};
            table1Name = parts{2}
            assignin('base', table1Name, table(data.Timestamp, data.(['Region' prefix1]), 'VariableNames', {'time', 'signal'}));
        
        elseif contains1G

            parts = strsplit(fileName, '_');
            prefix2 = parts{1};
            table2Name = parts{2}
            assignin('base', table2Name, table(data.Timestamp, data.(['Region' prefix2]), 'VariableNames', {'time', 'signal'}));

      
        end
        
        
    end

    clear contains0G contains1G data fileName files folderName
    
   tableNames = who;
    
    for i = 1:length(tableNames)
        
        
            
        tableName = tableNames{i};

            
                
        currentTable = evalin('base', tableNames{i});

        
        FP.rawdata(:,1) = currentTable.time;
        FP.rawdata(:,2) = currentTable.signal;
                    
        startpoint = FP.rawdata(1,1);
        FP.rawdata(:,1) = (FP.rawdata(:,1) - startpoint);
                
        FP.calcium_dependent(:,1) = downsample(FP.rawdata(2:end,1),2); 
        FP.calcium_dependent(:,2) = downsample(FP.rawdata(2:end,2),2);
                
                    
        FP.isosbestic(:,1) = downsample(FP.rawdata(:,1),2);
        FP.isosbestic(:,2) = downsample(FP.rawdata(:,2),2);
                
        temp_fit1 = fit(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2),'exp2');
                
        temp_fit2 = fit(FP.isosbestic(:,1),FP.isosbestic(:,2),'exp2');
                    
        %Generate and save figure as a picture
                
        figure(1)
                   
        plot(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2)-temp_fit1(FP.calcium_dependent(:,1)),'b');
                
        hold on
                
                   
        plot(FP.isosbestic(:,1),FP.isosbestic(:,2)-temp_fit2(FP.isosbestic(:,1)),'r');
                
        grid on
                
        ylabel({'Linearized Raw fluorescence'});
        xlabel({'Time (seconds)'});
        title({tableName ', ACh SNFR in dHPC'});
        legend ACh-dependent-signal Isosbestic
                
        saveas(gcf, [tableName '_plot.png']);
                    
                    
        close all
        clear FP

    end

 end

[Logan]

Actually, from the above, I tried running the first part of the function first, AND THEN running the second part as a separate script/function and it worked. I just cant get it to work all together in one go, is there an issue with my indentation or something perhaps?

[Matthew Blomquist]

Going through your entire code, it looks like the "assignin" function doesn't assign variables to your workspace until the entire function is completed. So, when you call "tableNames = who", there are no tables in your workspace to call since the function is still being run.

That is why when you separated the code, it runs as expected, because the first function finishes running and outputs the tables for the second function/script to read and plot.

[Voss]

The issue is that the table variables are being assigned to the base workspace, but the call to who gives you the names of the variables from the processFiles function's workspace (i.e., the workspace in which who is called).

The way I would solve this is to avoid assignin and evalin altogether, and instead store the tables in a struct. The processFiles shown below takes that approach.

[Note that I also modified the logic so that each file is read (using readtable), not only the ones that have both 0G and 1G in their name, which is what your code was doing, and which is a bad idea because then the value of the data variable is leftover from the previous iteration, which, even if that's what you want, will cause an error if the first file you read doesn't have both 0G and 1G in its name, so it assumes the files are listed in a particular order.]

processFiles('.')

% list the .png files in the current directory:
png_files = dir('*.png');
{png_files.name}

ans = 1×11 cell array
  Columns 1 through 8

    {'B5401right_plot.…'}    {'B5406left_plot.p…'}    {'B5408right_plot.…'}    {'B5410left_plot.p…'}    {'B5410right_plot.…'}    {'B5412right_plot.…'}    {'B5415right_plot.…'}    {'B5418left_plot.p…'}

  Columns 9 through 11

    {'B5419right_plot.…'}    {'B5420left_plot.p…'}    {'B5420right_plot.…'}

% display one of the .png files:
imshow(png_files(1).name)

function processFiles(folderName)
    
% Check if the folder exists
    
    if ~isfolder(folderName)
        error('Folder not found.');
    end
    
    % Get a list of all CSV files in the folder
    files = dir(fullfile(folderName, '*.csv'));
    
    
    % struct containing tables:
    S = struct();
    
    
    for i = 1:length(files)
        
        % Get the file name and split it by underscores
        fileName = files(i).name;
        contains0G = contains(fileName, '0G');
        contains1G = contains(fileName, '1G');
        
        if contains0G && contains1G
            
            parts = strsplit(fileName, '_');
            prefix1 = parts{1};
            prefix2 = parts{3};
            table1Name = parts{2};
            table2Name = parts{4};
            
            %read csv file
            data = readtable(fullfile(folderName, fileName));
            
            %Create tables
            S.(table1Name) = table(data.Timestamp, data.(['Region' prefix1]), 'VariableNames', {'time', 'signal'});
            S.(table2Name) = table(data.Timestamp, data.(['Region' prefix2]), 'VariableNames', {'time', 'signal'});
            
        elseif contains0G
            
            parts = strsplit(fileName, '_');
            prefix1 = parts{1};
            table1Name = parts{2};
            
            %read csv file
            data = readtable(fullfile(folderName, fileName));
            
            S.(table1Name) = table(data.Timestamp, data.(['Region' prefix1]), 'VariableNames', {'time', 'signal'});
        
        elseif contains1G
            
            parts = strsplit(fileName, '_');
            prefix2 = parts{1};
            table2Name = parts{2};
            
            %read csv file
            data = readtable(fullfile(folderName, fileName));
            
            S.(table2Name) = table(data.Timestamp, data.(['Region' prefix2]), 'VariableNames', {'time', 'signal'});
      
        end
        
        
    end
    
    
    tableNames = fieldnames(S);
    
    
    for i = 1:numel(tableNames)
        
        
        tableName = tableNames{i};
        
        
        currentTable = S.(tableName);
        

        FP.rawdata(:,1) = currentTable.time;
        FP.rawdata(:,2) = currentTable.signal;
                    
        startpoint = FP.rawdata(1,1);
        FP.rawdata(:,1) = (FP.rawdata(:,1) - startpoint);
                
        FP.calcium_dependent(:,1) = downsample(FP.rawdata(2:end,1),2); 
        FP.calcium_dependent(:,2) = downsample(FP.rawdata(2:end,2),2);
                
                    
        FP.isosbestic(:,1) = downsample(FP.rawdata(:,1),2);
        FP.isosbestic(:,2) = downsample(FP.rawdata(:,2),2);
                
        temp_fit1 = fit(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2),'exp2');
                
        temp_fit2 = fit(FP.isosbestic(:,1),FP.isosbestic(:,2),'exp2');
                    
        %Generate and save figure as a picture
                
        figure(1)
                   
        plot(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2)-temp_fit1(FP.calcium_dependent(:,1)),'b');
                
        hold on
                
                   
        plot(FP.isosbestic(:,1),FP.isosbestic(:,2)-temp_fit2(FP.isosbestic(:,1)),'r');
                
        grid on
                
        ylabel({'Linearized Raw fluorescence'});
        xlabel({'Time (seconds)'});
        title({tableName ', ACh SNFR in dHPC'});
        legend ACh-dependent-signal Isosbestic
                
        saveas(gcf, [tableName '_plot.png']);
                    
                    
        close all
        clear FP
    end
end

[Logan]

Ohhhh that makes sense! Is this happenning because of the way the loop is structured? Is there an easy fix to have it finish assigning before moving on?

[Voss]

@Logan: Please see my most recent comment for an explanation of and solution to the problem.

https://www.mathworks.com/matlabcentral/answers/2034424-looping-through-tables-in-my-workspace-plotting-them-and-saving-a-picture-of-the-plot#comment_2925984

[Logan]

Oh wow I see that totally makes sense, thats so interesting too I didn't even know you could be storing variables in a function's workspace vs the base workspace! No wonder sometimes it would give me the error that it didn't recognize a variable 'data' , for example, and it wasn't even present in the workspace. Thank you so much @Voss and @Matthew Blomquist for your time and patience the new function provided by @Voss works perfectly, really appreciate the help!

[Voss]

The error about unrecognized variable 'data' was due to not reading the file (readtable was only being called when both 0G and 1G were in the file name).

I'm any case, you're welcome and I'm glad it's working now!

Answer 2

[Matthew Blomquist]

I downloaded your mat file and tested your code and it works fine for me. I'm guessing the error occurs because of the first line: "tableNames = who"

Is your workspace clear of all variables except the n x 2 tables? When I ran your code a second time (after successfully running it the first time), I received the same error as above because not all the variables in my workspace were the n x 2 tables.

Let me know if that works, and if it doesn't, I can look closer!

[Logan]

Thanks for the response and for taking the time to look into this, I made sure my workspace is only composed of n x 2 tables and I keep running into the same issue. Did it work for you the firs time? Like you got the images of the plots in your folder? Check out the comment below under Voss' response, I'm providing more context to my code cause I think maybe the issue has something to do with its earlier parts. Thanks again!

[Voss]

When I ran the code with only tables in the workspace, it appeared to work correctly, and I got the .png files.

Here's a demo:

load('Workspace example.mat')

clear ans cmdout % these variables exist in the online environment workspace.
                 % clear them to have only whatever is in 'Workspace example.mat' remaining

tableNames = who

tableNames = 19×1 cell array
    {'B5401right'}
    {'B5404right'}
    {'B5406left' }
    {'B5407left' }
    {'B5408right'}
    {'B5409left' }
    {'B5410left' }
    {'B5410right'}
    {'B5411left' }
    {'B5411right'}
    {'B5412right'}
    {'B5413left' }
    {'B5415right'}
    {'B5416left' }
    {'B5418left' }
    {'B5419left' }
    {'B5419right'}
    {'B5420left' }
    {'B5420right'}

for i = 1:length(tableNames)
    
    
        
    tableName = tableNames{i};
        
            
    currentTable = evalin('base', tableName);
    
    FP.rawdata(:,1) = currentTable.time;
    FP.rawdata(:,2) = currentTable.signal;
                
    startpoint = FP.rawdata(1,1);
    FP.rawdata(:,1) = (FP.rawdata(:,1) - startpoint);
            
    FP.calcium_dependent(:,1) = downsample(FP.rawdata(2:end,1),2); 
    FP.calcium_dependent(:,2) = downsample(FP.rawdata(2:end,2),2);
            
                
    FP.isosbestic(:,1) = downsample(FP.rawdata(:,1),2);
    FP.isosbestic(:,2) = downsample(FP.rawdata(:,2),2);
            
    temp_fit1 = fit(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2),'exp2');
            
    temp_fit2 = fit(FP.isosbestic(:,1),FP.isosbestic(:,2),'exp2');
                
    %Generate and save figure as a picture
            
    figure(1)
               
    plot(FP.calcium_dependent(:,1),FP.calcium_dependent(:,2)-temp_fit1(FP.calcium_dependent(:,1)),'b');
            
    hold on
            
               
    plot(FP.isosbestic(:,1),FP.isosbestic(:,2)-temp_fit2(FP.isosbestic(:,1)),'r');
            
    grid on
            
    ylabel({'Linearized Raw fluorescence'});
    xlabel({'Time (seconds)'});
    title({tableName ', ACh SNFR in dHPC'});
    legend ACh-dependent-signal Isosbestic
            
    saveas(gcf, [tableName '_plot.png']);
                
                
    close all
    clear FP
end


% list the .png files in the current directory:
ls *.png

B5401right_plot.png  B5407left_plot.png   B5410left_plot.png   B5411right_plot.png  B5415right_plot.png  B5419left_plot.png   B5420right_plot.png
B5404right_plot.png  B5408right_plot.png  B5410right_plot.png  B5412right_plot.png  B5416left_plot.png	 B5419right_plot.png
B5406left_plot.png   B5409left_plot.png   B5411left_plot.png   B5413left_plot.png   B5418left_plot.png	 B5420left_plot.png

% display one of the .png files:
imshow([tableNames{1} '_plot.png'])

[Logan]

I'm so confused, I'm glad its working on your end but why is it giving me the error:

Dot indexing is not supported for variables of this type.

Error in processFiles (line 70)

FP.rawdata(:,1) = currentTable.time;

Is it something with my version of matlab? Should I try running it in a colleagues computer?

[Matthew Blomquist]

What is the output of

tableNames = who

(without the semi colon to suppress the output) as Voss had in his code above? Is it just the tables that you want, or are there extra variables that are not tables included?

[Logan]

It's just the table names