I am downloading a CSV file directly from a URL using requests
.
How can I parse the resulting string with csv.DictReader
?
Right now I have this:
r = requests.get(url)
reader_list = csv.DictReader(r.text)
print reader_list.fieldnames
for row in reader_list:
print row
But I just get ['r']
as the result of fieldnames
, and then all kinds of weird things from print row
.
From documentation of csv
, the first argument to csv.reader
or csv.DictReader
is csvfile
-
csvfile can be any object which supports the iterator protocol and returns a string each time its next() method is called — file objects and list objects are both suitable
In your case when you give the string as the direct input for the csv.DictReader()
, the next()
call on it only provides a single character, and hence that becomes the header, and then next() is continuously called to get each row.
Hence, you need to either provide a in-memory stream of the string (Using StringIO) or a list of lines (using splitlines
)
You can use io.StringIO()
and then use it in csv.DictReader
. Example/Demo -
>>> import csv
>>> s = """a,b,c
... 1,2,3
... 4,5,6
... 7,8,9"""
>>> import io
>>> reader_list = csv.DictReader(io.StringIO(s))
>>> print reader_list.fieldnames
['a', 'b', 'c']
>>> for row in reader_list:
... print row
...
{'a': '1', 'c': '3', 'b': '2'}
{'a': '4', 'c': '6', 'b': '5'}
{'a': '7', 'c': '9', 'b': '8'}
Or as indicated in the comments , you can split the lines before giving as input to csv.DictReader
. Example/Demo -
>>> reader_list = csv.DictReader(s.splitlines())
>>> print reader_list.fieldnames
['a', 'b', 'c']
>>> for row in reader_list:
... print row
...
{'a': '1', 'c': '3', 'b': '2'}
{'a': '4', 'c': '6', 'b': '5'}
{'a': '7', 'c': '9', 'b': '8'}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.