pandas - 读取非结构化 csv 并将其保存在 dataframe

Question

I have an unstructured CSV, which does not have a consitent number of columns for all the rows.

Input CSV looks like this:

Row1,Col11,Col12,Col13
Row2,Col21,Col22,Col23,Col24,Col25
Row3,Col31,Col32
Row4,,,,Col44

Note that for this is comma seperated file, and few rows may even have just commas to represent nulls (eg. example row 4), but few may have less values it self, other values have to considered nulls (eg. rest of the rows)

I want it to be read this raw file into the pandas dataframe as is, in case of Nulls, put NaN.

Something like this:

     0        1        2        3        4        5
0    Row1     Col11    Col12    Col13    NaN      NaN
1    Row2     Col21    Col22    Col23    Col24    Col25
2    Row3     Col31    Col32    NaN      NaN      NaN
3    Row4     NaN      NaN      NaN      Col44    NaN

I am using pandas.read_csv function to read this, but it looks like it uses the first row to determine number of columns, and since it is inconsistent it gives error.

Code:

df= pd.read_csv(path, engine='python',  header = None)

error:

Expected 4 fields in line 2, saw 6

How can I fix this?

Answer 1

Some thing like this would help:

with open('out.txt') as f:
    df = pd.DataFrame([line.strip().split(',') for line in f.readlines()]
                     ).replace('', None).fillna(np.nan)

Output:

      0      1      2      3      4      5
0  Row1  Col11  Col12  Col13    NaN    NaN
1  Row2  Col21  Col22  Col23  Col24  Col25
2  Row3  Col31  Col32    NaN    NaN    NaN
3  Row4  Col31  Col32    NaN  Col44    NaN