读取在观察中具有不相等列的熊猫数据框

Question

I am trying to read this small data file, Link - https://drive.google.com/open?id=1nAS5mpxQLVQn9s_aAKvJt8tWPrP_DUiJ

I am using the code -

df = pd.read_table('/Data/123451_date.csv', sep=';', index_col=0,  engine='python', error_bad_lines=False)

It has ';' as a seprator, and values are missing in the file for some columns values in some observations (or rows).

How can I read it properly. I see the current dataframe, which is not loaded properly.

Answer 1

It looks like the data you use has some garbage in it. Precisely, rows 1-33 (inclusive) have additional, unnecessary (non-GPS) information included. You can either fix the database by manually removing the unneeded information from the datasheet, or use following code snippet to skip the rows that include it:

from pandas import read_table

data = read_table('34_2017-02-06.gpx.csv', sep=';', skiprows=list(range(1, 34)).drop("Unnamed: 28", axis=1)

The drop("Unnamed: 28", axis=1) is simply there to remove an additional column that is created probably due to each row in your datasheet ending with a ; (because it reads the empty space at the end of each line as data).

The result of print(data.head()) is then as follows:

   index  cumdist   ele    ...     esttotalpower        lat       lon
0     49      340 -34.8    ...                 9  52.077362  5.114530
1     51      350 -34.8    ...                17  52.077468  5.114543
2     52      360 -35.0    ...               -54  52.077521  5.114551
3     53      370 -35.0    ...              -173  52.077603  5.114505
4     54      380 -34.8    ...               335  52.077677  5.114387

[5 rows x 28 columns]

To explain the role of the drop command even more, here is what would happen without it (notice the last, weird column)

   index  cumdist   ele     ...             lat       lon  Unnamed: 28
0     49      340 -34.8     ...       52.077362  5.114530          NaN
1     51      350 -34.8     ...       52.077468  5.114543          NaN
2     52      360 -35.0     ...       52.077521  5.114551          NaN
3     53      370 -35.0     ...       52.077603  5.114505          NaN
4     54      380 -34.8     ...       52.077677  5.114387          NaN

[5 rows x 29 columns]