What is the most efficient way of writing a dictionary into a csv in Python?

Question

I am writing to code to generate a csv file by scraping a html table from a website. The function will look at each row of the table <tr> and store the data columns in a dictionary like shown below

 def write_data():
    table_date = get_data()   # call function to get data from html table into a dict
    // write table_date to csv

 def get_data():
   data = {}
   for row in tr:
      data['name'] = 'John'
      data['id'] = 12
      return data

This is a simplified version but essentially I need a way to get the dictionary object data for each table row and write it into a csv, where the keys will be the header row. What is an efficient way to do this?

Answer 1

Use the csv.DictWriter() class ; just send it dictionaries for each row:

writer = csv.DictWriter(open_writable_file, fieldnames=['id', 'name'])
writer.writeheader()  # write a row the fieldnames

and for each dictionary you produce:

writer.writerow(table_data)

Do make sure you open the writable file with the newline='' option to let the csv module control the line endings:

with open(filename, 'w', newline='') as open_writable_file):

This advise is omitted from the csv.DictWriter documentation example, for some reason; but the object is a subclass of the csv.writer() class and the advice there applies just as much.