将所有值写入一行csv.DictWriter

Question

I'm having trouble to generate a well formatted CSV file out of some data i fetched from the leadfeeder API. In the csv file that is currently being created, not all values are in one row, id and leads are one column higher then the rest. Like here:

CSV Output

I later also like to load another json file and use it to map some values over the id and then put also the visits per lead into my csv file.

Do you also have some advice for this?

This is my code so far:

import json
import csv

csv_columns = ['name', 'industry', 'website_url', 'status', 'crm_lead_id', 'crm_organization_id', 'employee_count', 'id', 'type' ]

with open('data.json', 'r') as d:    
    d = json.load(d)

csv_file = 'lead_daten.csv'

try:
    with open('leads.csv', 'w', newline='') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=csv_columns, extrasaction='ignore')
        writer.writeheader()
        for item in d['data']:
            writer.writerow(item)
            writer.writerow(item['attributes'])

except IOError:
    print("I/O error")

My json data has the following structure: I need also some of the nested values like the id in relationships!

{
    "data": [
        {
            "attributes": {
                "crm_lead_id": null, 
                "crm_organization_id": null, 
                "employee_count": 5000, 
                "facebook_url": null, 
                "first_visit_date": "2019-01-31", 
                "industry": "Furniture", 
                "last_visit_date": "2019-01-31", 
                "linkedin_url": null, 
                "name": "Example Inc", 
                "phone": null, 
                "status": "new", 
                "twitter_handle": "example", 
                "website_url": "http://www.example.com"
            }, 
            "id": "s7ybF6VxqhQqVM1m1BCnZT_8SRo9XnuoxSUP5ChvERZS9", 
            "relationships": {
                "location": {
                    "data": {
                        "id": "8SRo9XnuoxSUP5ChvERZS9", 
                        "type": "locations"
                    }
                }
            }, 
            "type": "leads"
        }, 
        {
            "attributes": {
                "crm_lead_id": null, 

Answer 1

When you write to a csv, you must write one full row at a time. You current code writes one row with only id and type, and then a different row with the other fields.

The correct way is to first fully build a dictionary containing all the fields and only then write it in one single operation. Code could be:

    ...
    writer.writeheader()
    for item in d['data']:
        item.update(item["attributes"])
        writer.writerow(item)
    ...