Thread Subject:

skip rows with textscan? import txt file problem

Hello all,

I know it has been posted a couple of times but my problem seems to be a little different. I have a large textfile. The text file looks like this (without 6 headerlines):

0 SignalQuality 3
0 Data -1.310255E+07 -1.310299E+07 -1.310337E+07 -1.31027E+07 -1.310189E+07 -1.31022E+07 -1.310325E+07 -1.310317E+07 -1.310274E+07 -1.310333E+07 -1.310412E+07 -1.310401E+07 -1.310369E+07 -1.310385E+07 -1.31039E+07 -1.310349E+07 -1.310327E+07 -1.310352E+07 -1.31041E+07 -1.310447E+07 -1.310404E+07 -1.310351E+07 -1.3104E+07 -1.310508E+07 -1.310458E+07 -1.310371E+07 -1.310433E+07 -1.310535E+07 -1.31053E+07 -1.310409E+07
0.46152 SignalQuality 2
0.46152 Data -1.31032E+07 -1.310304E+07 -1.310254E+07 -1.310169E+07 -1.310147E+07 -1.310215E+07 -1.310333E+07 -1.310305E+07 -1.310244E+07 -1.310306E+07 -1.310332E+07 -1.310282E+07 -1.310203E+07 -1.310166E+07 -1.310215E+07 -1.310276E+07 -1.310261E+07 -1.310202E+07 -1.310231E+07 -1.310347E+07 -1.310341E+07 -1.310227E+07 -1.310217E+07 -1.310289E+07 -1.310288E+07 -1.310205E+07 -1.310139E+07 -1.310193E+07 -1.310246E+07 -1.31016E+07
27.22968 Comment blablabla

I want to import only the data columns. It tried the following

fid = fopen('datafile.txt');
mydata = textscan(fid, '%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f', 'headerLines', 6);
fclose(fid);

but it turns out when it detects a comment it stops.

the problem is that when i try for example strncmp() it still reads from mydata which is incomplete.
 
Could someone help me with this problem? Many thanks!

ps just thinking out loud, is het possible to only read the first column of the file. Then compare the current with the previos value. if these are equal read the full row and store it. the first values of signal quality and data are always equal, the first value of comment is always different.

Rik

On 5/27/2012 6:12 AM, Rik wrote:
...
> ... I have a large textfile. The text file looks like this
> (without 6 headerlines):
>
> 0 SignalQuality 3
> 0 Data -1.310255E+07 -1.310299E+07 -1.310337E+07 -1.31027E+07
> -1.310189E+07 -1.31022E+07 -1.310325E+07 -1.310317E+07 -1.310274E+07
...
> -1.310409E+07
> 0.46152 SignalQuality 2
> 0.46152 Data -1.31032E+07 -1.310304E+07 -1.310254E+07 -1.310169E+07
> -1.310147E+07 -1.310215E+07 -1.310333E+07 -1.310305E+07 -1.310244E+07
...
> -1.31016E+07
> 27.22968 Comment blablabla
>
> I want to import only the data columns. It tried the following
>
> fid = fopen('datafile.txt');
> mydata = textscan(fid,
> '%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f',
> 'headerLines', 6);
> fclose(fid);
>
> but it turns out when it detects a comment it stops.
>
> the problem is that when i try for example strncmp() it still reads from
> mydata which is incomplete.
>
> Could someone help me with this problem? Many thanks!
>
> ps just thinking out loud, is het possible to only read the first column
> of the file. Then compare the current with the previos value. if these
> are equal read the full row and store it. the first values of signal
> quality and data are always equal, the first value of comment is always
> different.

Certainly it's possible but you'll have to use low level routines to do
it; textscan() and friends won't/can't automagically.

I'd use the facility to call textscan() iteratively in a loop or some
similar variant. If the comment lines are scattered and not consistent
in their number/location you'll be forced to deal with it by reverting
to fgetl() and parsing, unfortunately.

As general as TMW has tried to make textscan() it simply can't deal with
irregular structures well.

BTW, for things like

  '%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f',

the idiom

fmt = ['%f%*s' repmat(1,30,'%f')];

is a saver of eyesight and sanity. The lack of a repeat specifier is a
real shortcoming in the C-like format string formulation.

--

Thanks for your response! Because I am not very familiar with all the functions of matlab I solved it using excel and the data filtering option to only show the data I want and saved it to a new file.
I am sure matlab can do it as well but this is less time consuming for me (a)

Thanks again!

"Rik" wrote in message <jpt267$a3j$1@newscl01ah.mathworks.com>...
> Hello all,
>
> I know it has been posted a couple of times but my problem seems to be a little different. I have a large textfile. The text file looks like this (without 6 headerlines):
>
> 0 SignalQuality 3
> 0 Data -1.310255E+07 -1.310299E+07 -1.310337E+07 -1.31027E+07 -1.310189E+07 -1.31022E+07 -1.310325E+07 -1.310317E+07 -1.310274E+07 -1.310333E+07 -1.310412E+07 -1.310401E+07 -1.310369E+07 -1.310385E+07 -1.31039E+07 -1.310349E+07 -1.310327E+07 -1.310352E+07 -1.31041E+07 -1.310447E+07 -1.310404E+07 -1.310351E+07 -1.3104E+07 -1.310508E+07 -1.310458E+07 -1.310371E+07 -1.310433E+07 -1.310535E+07 -1.31053E+07 -1.310409E+07
> 0.46152 SignalQuality 2
> 0.46152 Data -1.31032E+07 -1.310304E+07 -1.310254E+07 -1.310169E+07 -1.310147E+07 -1.310215E+07 -1.310333E+07 -1.310305E+07 -1.310244E+07 -1.310306E+07 -1.310332E+07 -1.310282E+07 -1.310203E+07 -1.310166E+07 -1.310215E+07 -1.310276E+07 -1.310261E+07 -1.310202E+07 -1.310231E+07 -1.310347E+07 -1.310341E+07 -1.310227E+07 -1.310217E+07 -1.310289E+07 -1.310288E+07 -1.310205E+07 -1.310139E+07 -1.310193E+07 -1.310246E+07 -1.31016E+07
> 27.22968 Comment blablabla
>
> I want to import only the data columns. It tried the following
>
> fid = fopen('datafile.txt');
> mydata = textscan(fid, '%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f', 'headerLines', 6);
> fclose(fid);
>
> but it turns out when it detects a comment it stops.
>
> the problem is that when i try for example strncmp() it still reads from mydata which is incomplete.
>
> Could someone help me with this problem? Many thanks!
>
> ps just thinking out loud, is het possible to only read the first column of the file. Then compare the current with the previos value. if these are equal read the full row and store it. the first values of signal quality and data are always equal, the first value of comment is always different.
>
> Rik