Opening a file that has been uploaded in Flask

Question

I'm trying to modify a csv that is uploaded into my flask application. I have the logic that works just fine when I don't upload it through flask.

import pandas as pd
import StringIO
with open('example.csv') as f:
    data = f.read()

data = data.replace(',"', ",'")
data = data.replace('",', "',")
df = pd.read_csv(StringIO.StringIO(data), header=None, sep=',', quotechar="'")
print df.head(10)

I upload it to flask and access it using

f = request.files['data_file']

When I run it through the code above, replacing open('example.csv') with open(f), I get the following error

coercing to Unicode: need string or buffer, FileStorage found

I have figured out that the problem is the file type here. I can't use open on my file because open is looking for a file name and when the file is uploaded to flask it is the instance of the file that is being passed to the open command. However, I don't know how to make this work. I've tried skipping the open command and just using data = f.read() but that doesn't work. Any suggestions?

Thanks

Answer 1

FileStorage is a file-like wrapper around the incoming data. You can pass it directly to read_csv .

pd.read_csv(request.files['data_file'])

---

You most likely should not be performing those replace calls on the data, as the CSV module should handle that and the naive replacement can corrupt data in quoted columns. However, if you still need to, you can read the data out just like you were before.

data = request.files['data_file'].read()

If your data has a mix of quoting styles, you should fix the source of your data.

Answer 2

Answering my own question in case someone else needs this.

FileStorage objects have a .stream attribute which will be an io.BytesIO

f  = request.files['data_file']
df = pandas.read_csv(f.stream)