Fortran: How to skip many lines of data file efficiently -

i have formatted data file typically billions of lines long, several lines of headers of variable length. data file takes form:

     # header 1     # header 2     # headers of variable length.     # data begins next line.     1.23  4.56  7.89  0.12     2.34  5.67  8.90  1.23     :     :     # billions of lines of data, each row same length, same format.     -- end of file --     

i extract portion of data file, , current code looks like:

<pre> j=1,jmax !suppose want extract jmax lines of data file.    [algorithm determine number of lines skip, "n(j)"]   !this determines number of lines skip previous file   !position, when data read on j-1th iteration.    !skip n-1 lines go next data line read off:   i=1,n-1     read(unit=unit,fmt='(a)')   end   !now read off line of data want:   read(unit=unit,fmt='(data_format)'),data1,data2,etc.   !data stored in arrays. end </pre> 

the problem is, n(j) can anywhere between 1 , several billion, takes time run code.

my question is, there more efficient way of skipping millions of lines of data? way can think of, while sticking fortran, open file direct access , jump desired line upon opening file.

as suggest, direct access seems best option. requires records have same length, headers violate. also, why used formatted output? file of length, hard imagine person reading file. if use unformatted io, file both smaller , io faster. perhaps create 2 files, 1 headers (metadata) in human reader form, , other data in native form. native / binary representation means loss of portability, consider if want move files different computer architectures or have them useable decades. otherwise it's worth convenience. other options use more sophisticated file format combines metadata , data, such hdf5 or fits, communication between 2 programs of 1 person, that's excessive.