Pandas pd.read_csv() splits each character into a new row

Question

Weird (and possibly bad) question - when using the pd.read_csv() method, it appears as if pandas separates each character into a new row in the csv.

My code:

import pandas as pd
import csv
# doing this in colab
from google.colab import files

# downloading a csv with my apps sdk 
data = sdk.run_look('2131','csv')

with open('big.csv', 'w') as file:
    csvwriter = csv.writer(file, delimiter=',')
    csvwriter.writerows(data)
df = pd.read_csv('big.csv', delimiter=',')
files.download('big.csv')

Output:

What I'm getting from line 4 ( data=sdk... ) looks like this:

Orders Status,Orders Count
complete,31377
pending,505
cancelled,375

However, what I get back from pandas looks like this:

0  r
1  d
2  e
3  r
4  s
...

I think it's line 6 ( df=read_csv... ) because if I do print(data) then compare to print(df.head()) , I see that print data returns correct data, print df.head returns the weirdly formatted data.

Any idea of what I'm doing wrong here? I'm a complete noob, so probably pebkac:)

Answer 1

It appears that your data is just coming in as a big string. If this is the case, you don't need to use the csv writer at all, and can just write it directly to your out file.

import pandas as pd

# doing this in colab
from google.colab import files

# downloading a csv with my apps sdk 
data = sdk.run_look('2131', 'csv')

with open('big.csv', 'w') as file:
    file.write(data)

df = pd.read_csv('big.csv', delimiter=',')

files.download('big.csv')