Pandas read_csv with uneven length of rows as header

Question

There is a txt file with uneven length

LS-DYNA user input                                                      
                         ls-dyna mpp.78769 s              date 01/02/2013

 constraint #      axial        shear         time  failure                                       length  rslt moment      torsion
       1720  8.39282E-01  6.55466E-01  1.20000E+03      0.0    spotweld beam  ID     938970  4.47325E+01  2.24041E+00
       1721  3.30134E-01  5.08016E-01  1.20000E+03      0.0    spotweld beam  ID     938971  4.47310E+01  1.70857E+00
       1722  9.52039E-01  2.24977E+00  1.20000E+03      0.0    spotweld beam  ID     938972  3.50040E+00  1.14531E+01
       1723  1.37947E+00  3.75614E+00  1.20000E+03      0.0    spotweld beam  ID     938973  2.99986E+00  3.72429E+01
       1724 -1.29900E+00  8.59783E-01  1.20000E+03      0.0    spotweld beam  ID     938974  3.50112E+00  1.11357E+01
       1725 -1.39978E+00  5.05035E+00  1.20000E+03      0.0    spotweld beam  ID     938975  2.99934E+00  1.69379E+01
       1726 -8.28811E-01  2.36767E+00  1.20000E+03      0.0    spotweld beam  ID     938976  3.50022E+00  1.01569E+01
       1727 -8.02390E-01  2.83158E+00  1.20000E+03      0.0    spotweld beam  ID     938977  2.99945E+00  5.26153E+01
       1728  2.45994E+01  2.55278E+02  1.20000E+03      0.0    spotweld beam  ID     938978  3.51565E+00  1.03888E+01
       1729  3.79365E+01  1.91420E+01  1.20000E+03      0.0    spotweld beam  ID     938978  2.99987E+00  8.96939E+00

Without resorting to skiprows , as the rows without data would change in different cases, I am trying to read the file by

pd.read_csv(File, header=None, delim_whitespace=True)

It would throw me an error with

pandas.parser.CParserError: Error tokenizing data. C error: Expected 3 fields in line 2, saw 5

Then I redefine the pandas parameters, as

my_cols = ['A', 'B', 'C', 'D', 'E','F','G']
elout= pd.read_csv(File, names=my_cols, header=None, delim_whitespace=True)

There would be no issue. Except this dumb way, is there any other settings I could resort to solve this issue?

Thank you!

Answer 1

If you don't want to use skiprows , an alternative is to open the file yourself like f = open(File) . Then you f.readline() and parse manually the first lines that are not of interest for you. Once you extract the useful parts of the header through f and the file pointer reached the beginning of the table, simply pass f to read_csv as first argument, and pandas will start processing the data from that point.