How to import large textfile

5 views (last 30 days)
Rajab Omar
Rajab Omar on 9 Jun 2014
Edited: Cedric on 12 Jun 2014
Hi EVERYONE I'm new to matlab, I've been trying to import large text file .asc to matlab. is there anyone can help. the file contains 59615 (24*24)matrix. I've attached small part of the file. When I import the whole file I get an error while if I import small part it works fine!! I don't think it's memory problem, can anyone help please???
  4 Comments
Cedric
Cedric on 9 Jun 2014
Edited: Cedric on 9 Jun 2014
It is not too large, but not small either. Assuming 10 chars per element (including separators, etc), it is more than 300MB
>> 59615*24*24 * 10 / 1e6
ans =
343.3824
Could you provide a ~5MB sample?
Rajab Omar
Rajab Omar on 9 Jun 2014
here's the error I got: "import operation failed. the most likely reason is that there are unimortable cells in the selection. try to adding arule in the toolstrip to convert unimortable cells into numbers."
I've a;ready excluded the raws with unimortable cells.

Sign in to comment.

Accepted Answer

Cedric
Cedric on 9 Jun 2014
Edited: Cedric on 9 Jun 2014
Here is one way to proceed
fId = fopen( 'omar_1.txt', 'r' ) ;
buf = fscanf( fId, '%f', Inf ) ;
fclose( fId ) ;
bSize = 24*24 + 1 ;
nBlocks = length( buf ) / bSize ;
data = cell( nBlocks, 1 ) ;
for bId = 1 : nBlocks
bBnds = ((bId-1)*bSize+2) : (bId*bSize) ;
data{bId} = reshape( buf(bBnds), 24, 24 ).' ;
end
At the end, you have your matrices in the cell array data:
>> data
data =
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
[24x24 double]
so data{1} is the first matrix. There are other approaches, but they a probably less efficient if your file is really large.
  4 Comments
Rajab Omar
Rajab Omar on 12 Jun 2014
Edited: per isakson on 12 Jun 2014
Thanks a lot Cedric Wannaz I've different question now, I'm not sure if it's OK to ask it within this question. I've got time series data and need to get the power spectral density (PSD), and the probability density function (PDF). I wrote code for PSD, then I realised that there's a function called (pwelch). However the results were not identical. I shall be grateful if you can suggest the best option to estimate both PSD and PDF for time series data. HERE'S WHAT I'VE DONE:
%clf;clc;clear;
VFF_1=VFF.*hanning(length(VFF));%%windowing
N= length(VFF_1);
dt=0.001;
fs=1/dt
tmax=(N-1)*dt;
t=0:dt:tmax;
tn=N/fs%%Total sample length
fmin=1/tn
fmax=0.5*fs;%%nyquist
f=(1:N/2)/(N/2)*fmax;%define frequency
f=f';
length(f)
y=fft(VFF_1);
y(1)=[];
py=abs(y(1:N/2)).^2;%%Define power spectrum
length(py)
figure
plot(f,py)%%plot power spectrum against Frequency
xlabel('Frequency (HZ)');title('Power Spectrum');ylabel('power');
REGARDS
Cedric
Cedric on 12 Jun 2014
Edited: Cedric on 12 Jun 2014
Hi, as it is a different topic, you should post a new question. You'll also get answers from people who are more proficient than I on this particular matter!
Regards,
Cedric

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!