Trying to sort a text document by alphabetical order and count how many times a word appears

Hello,

I am trying to take a .txt file, import it, and then count how many times a word appears and then how sort all the words in order.

I want to take the .txt file, make it into a string, chop the string up into individual words, and then put the words into a matrix along with there word count.

Take the following sentance as an example, "The cat is a cat and would like to have a cat."

The output would look like the following:

word:a                    count:2    
word:and                  count:1
word:cat                  count:3
word:have                 count:1
word:like                 count:1  
word:the                  count:1 
word:to                   count:1
word:would                count:1

Here is what I have right now.

fid = fopen('Theodore_Roosevelt_The_Duties_Of_American_Citizenship.txt'); 
Line = fgetl(fid); 
textfile = strings(1,1); 
k = 1; 
while ischar(Line) textfile(k,1) = Line; 
    Line = fgetl(fid); 
    k = k+1; 
end 
fclose(fid); 
 %removing all nonAlpha characters from the text file 
punc = [".",";",",","(",")","--","-",]; 
textfile = replace(textfile,punc," "); 
textfile=lower(textfile); 
%for loop is used to split string 'textFile' into individula words 
words = strings(0); 
for i = 1:length(textfile) 
    words = [words;split(textfile(i))]; 
end 
k=convertStringsToChars(words); 
h=1;
F=2;
G=1;
sorted=zeros(1);
for j = 1:length(k)  
        T=strncmp(k(h),k(F),1); 
        if T==1 
        %if true, put h word in sorted before word F 
        sorted=sorted(k(h),G); 
        G=G+1; 
        sorted=sorted(k(F),G); 
        end 
    h=h+2; 
    F=F+2; 
end 
disp(sorted)

This is the error I get when executing the code:

Error using sort

Input argument must be a cell array

of character vectors.

Error in sorted (line 28)

[ignored,index] =

sort([meshsites(:).' sites(:).']);

Error in fgetltest (line 35)

sorted=sorted(k(h),G);

This is for a homework question, but I am lost about this matrix part I want to put it in.

[Walter Roberson]

The error appears to be in sorted(), which you do not show us the code for.

Also, which MATLAB release are you using?

[Walter Roberson]

I do not see an initialization for G ?

[Andy T]

the matlab is 2019, I also forgot to put those in there when I was putting the code into the question. Let me fix that. I also forgot to even make a vector called sorted.

Answer 1

[Adam Danz]

Here's a different approach. See inline comments for details.

% Read text
C = fileread('Theodore_Roosevelt_The_Duties_Of_American_Citizenship.txt'); 

% Split into words by spaces
words = strsplit(strtrim(C)); 

% Remove problematic characters
% But be careful: this removes any non-letter from each word. 
% cat's turns into cats; But without this "cats" with quotes 
% or cats! will not be recognized.  If that's a problem you'll
% need to use a regular expression approach. 
words = cellfun(@(x)x(isletter(x)), words, 'UniformOutput', false);

% make all letters lower case 
words = lower(words); 

% sort them into alphabetical order
words = sort(words); 

% Count frequency of each word
wordList = unique(words); 
wordCount = histcounts(categorical(words), categorical(wordList)); 

% Output table
T = table(wordList(:), wordCount(:), 'VariableNames', {'Word', 'Count'}); 

[Andy T]

the sort function helped me out, but I am not supposed to put it in a table. I realized my expected output looked like a table(sorry about that, I changed what its supposed to look like). would it be like a while loop counting the number of times the word appears? I could then have it print out the word and the number of time it appears. In the while loop it would have to compare the next word in words, and it its not the same, the loop starts over.

[Adam Danz]

"I am not supposed to put it in a table"

That's fine. The wordList and wordCount arrays are your final output and you can organize them any way you'd like.

"would it be like a while loop counting the number of times the word appears"

Yes, fileread() read the entire file at once and histcounts() counts the number of times each word exists. Neither requires a loop.

I'm not sure I follow the rest of your comment. Have you tried the code? Is there something you'd like to alter to suit your needs?

[Andy T]

I tried running the code and got an error about how all variables must have the same number of rows. I got the word count to work and the sort tho. I am just trying to get it to display them. I was talking about a loop that would look at the the list of sorted words. It would compare word(1) with word(2), if they are different, then there is only one of word(1). It would then display the word and how many times it appeared. But if word(3) and word(4) are the same, then it would count that as two. It would continue to count until the next word is different. The loop would do this till the end.

[Adam Danz]

I wonder what line in my answer caused an error.

There are a few ways to show the results from my answer. One is to just display them in a table or a cell array. Examples below.

% As a table (cleanest option)
T =
  21×2 table 
         Word          Count
    _______________    _____
    {'a'          }      2  
    {'and'        }      1  
    {'animal'     }      1  
    {'are'        }      1  
    {'best'       }      1  
    {'but'        }      1  
    {'cat'        }      5  
    % the rest of the rows have been cut
    
% As a cell array    
[wordList(:), num2cell(wordCount(:))]

  21×2 cell array
    {'a'          }    {[2]}
    {'and'        }    {[1]}
    {'animal'     }    {[1]}
    {'are'        }    {[1]}
    {'best'       }    {[1]}
    {'but'        }    {[1]}
    {'cat'        }    {[5]}
    % the rest of the rows have been cut

If you need to display them within a loop you can do this (but this seems odd and unnecessary)

for i = 1:numel(wordList)
    fprintf('%s %d\n', wordList{i}, wordCount(i));
end

% Displays: 
a 2
and 1
animal 1
are 1
best 1
but 1
cat 5
% the rest of the rows have been cut

[Andy T]

Thank you so much for the help. This really helped me out.

[Adam Danz]

Glad I could help!

The 2 mini lessons here are the use of fileread() and histcounts().