separator doesn't work while reading csv in Python

Question

I have data which consists 3004 rows without header, and each row has different number of fields (eg for row number 1,2,3,4 has 16,17,21,12, respectively). Here is my code when I call the csv.

 df = pd.read_csv(file,'rb', delimiter ='\t', engine='python')

here is the output:

$GPRMC,160330.40,A,1341.,N,10020.,E,0.006,,150517,,,A*7D
$GPGGA,160330.40,1341.,N,10020.,E,1,..
$PUBX,00,160330.40,1341.,N,10020.,E,...
$PUBX,03,20,2,-,056,40,,000,5,U,014,39,41,026,...
$PUBX,04,160330.40,150517,144210.39,1949,18,-6...
ÿ$GPRMC,160330.60,A,1341.,N,10020.,E...
$GPGGA,160330.60,1341.,N,10020.,E,1,...

It seemed like delimiter didn't work at all to separate the data into column by column. Hence, I tried with columns function based on number of fields from ($PUBX, 00). Here is the code when I add columns:

my_cols = ['MSG type', 'ID MSG', 'UTC','LAT', 'N/S', 'LONG', 'E/W', 'Alt', 'Status','hAcc', 'vAcc','SOG', 'COG', 'VD','HDOP', 'VDOP', 'TDOP', 'Svs', 'reserved', 'DR', 'CS', '<CR><LF>']

df = pd.read_csv(file, 'rb', header = None, na_filter = False, engine = 'python', index_col=False, names=my_cols)

and the result be like the picture below. The file becomes into one column in 'MSG type'.

the output

My purpose after success to call this csv is to read rows only with combination between $PUBX, 00,... and one column of $PUBX, 04,... and write it to csv. But, I am still struggling how to separate the file into columns. Please advice me on this matter. Thank you very much.

Answer 1

Hat Tip to Ilja Everilä

@Hasanah Based on your code:

df = pd.read_csv(file,'rb', delimiter ='\\t', engine='python')

delimiter='\\t' tells pandas to separate the data into fields based on tab characters.

The default delimiter when pandas reads in csv files is a comma, so you should not need to define a delimiter:

df = pd.read_csv(file,'rb', engine='python')

Answer 2

pd.read_csv

is used for reading CSV(comma separated values) Files hence you don't need to specify a delimiter.

If you want to read a TSV (Tab separated values) File, you can use:

pd.read_table(filepath)

The default separator is tab