Python skip header row from request response

Question

I'm querying a service to get csv data and I would like to append this csv extract to another file by skipping the header information Code -

response = requests.get('http://myservice.com&format=csv')
with open('out.csv', 'a', new like='') as f:
   writer = csv.writer(f)
   for line in response.iter_lines():
        next(line)
        writer.writerow(line.decode('utf-8').split('|'))

Tried the above but I get error

'bytes' object is not an iterator

Answer 1

response.iter_lines() is the iterator, lines is the value from iterating. So calling next() on that doesn't make much sense.

You can:

iter_lines = response.iter_lines()
next(iter_lines, None)
for line in iter_lines:
    writer.writerow(line.decode('utf-8').split('|'))

You could also try reading straight into a csv.DictReader , eg:

response = requests.get('http://myservice.com&format=csv')
with open('out.csv', 'a') as f:
    reader = csv.DictReader(response.text.splitlines(), delimiter='|')
    writer = csv.DictWriter(f, reader.fieldnames)
    writer.writerows(reader)

Answer 2

Try with

response = 
requests.get('http://myservice.com&format=csv')
with open('out.csv', 'a', new like='') as f:
    writer = csv.writer(f)
    rows = response.iter_lines()
    next(rows)
    for line in rows:
        writer.writerow(line.decode('utf-8').split('|'))

rows is the iterator that needs to skip the header, whereas line is the actual content of each line read.

The for loop simply follows the iteration protocol, ie

• it obtains an iterator by calling iter on the object (an iterator returns itself, as in this case)
• it keeps calling next on it until the StopIteration exception is raised