熊猫中的复杂定界符read_csv

Question

I'm trying to read some log files using Pandas, where the columns are delimited by whitespace, and some columns consist of single quoted strings with whitespace (eg 'string ' ). I am having a hard time reading these files with read_csv . For example (using some dummy data):

import pandas as pd
from io import StringIO

data = StringIO("""\
  1   2   'asdf    ' 3
  4   5   'asdfg   ' 4  
""")

columns = ['a','b','c','d']
df = pd.read_csv(data, delim_whitespace=True, names=columns)

For the first row, this results in columns 1 , 2 , 'asdf , ' , 3 , where I would prefer to have it as 1 , 2 , asdf , 3 . The behavior makes total sense, but I can't find a way to make read_csv parse such files "correctly" (as I want it).

Is this at all possible?

Answer 1

You have to use the quotechar argument while parsing from read_csv

df = pd.read_csv(filename, quotechar = "'", delim_whitespace=True, names=columns)

Although this will result in column c having extra whitespaces. You can get rid of those using

df.c = df.c.str.strip()