将段元组列表写入csv文件

Question

The following code is designed to write a tuple, each containing a large paragraph of text, and 2 identifiers behind them, to a single line per each entry.

import urllib2
import json
import csv

base_url = "https://www.eventbriteapi.com/v3/events/search/?page={}
writer = csv.writer(open("./data/events.csv", "a"))
writer.writerow(["description", "category_id", "subcategory_id"])

def format_event(event):
    return event["description"]["text"].encode("utf-8").rstrip("\n\r"), event["category_id"], event["subcategory_id"]

for x in range(1, 2):
    print "fetching page - {}".format(x)
    formatted_url = base_url.format(str(x))
    resp = urllib2.urlopen(formatted_url)
    data = resp.read()
    j_data = json.loads(data)
    events = map(format_event, j_data["events"])
    for event in events:
        #print event
        writer.writerow(event)

    print "wrote out events for page - {}".format(x)

The ideal format would be to have each line contain a single paragraph, followed by the other fields listed above, yet here is a screenshot of how the data comes out.

If instead I this line to the following:

writer.writerow([event])

Here is how the file now looks:

It certainly looks much closer to what I want, but its got parenthesis around each entry which are undesirable.

EDIT here is a snippet that contains a sample of the data Im working with.

Answer 1

Change your csv writer to be DictWriter .

Make a few tweaks:

def format_event(event):
    return {"description": event["description"]["text"].encode("utf-8").rstrip("\n\r"), 
            "category_id": event["category_id"], 
            "subcategory_id": event["subcategory_id"]}

May be a few other small things you need to do, but using DictWriter and formatting your data appropriately has been the easiest way to work with csv files that I've found.

Answer 2

Can you try writing to the CSV file directly without using using the csv module? You can write/append comma-delimited strings to the CSV file just like writing to typical text files. Also, the way you deal with removing \\r and \\n characters might not be working. You can use regex to find those characters and replace them with an empty string "" :

import urllib2
import json
import re

base_url = "https://www.eventbriteapi.com/v3/events/search/?page={}"

def format_event(event):
    ws_to_strip = re.compile(r"(\r|\n)")
    description = re.sub(ws_to_strip, "", event["description"]["text"].encode("utf-8"))
    return [description, event["category_id"], event["subcategory_id"]]

with open("./data/events.csv", "a") as events_file:
    events_file.write(",".join(["description", "category_id", "subcategory_id"]))

    for x in range(1, 2):
        print "fetching page - {}".format(x)
        formatted_url = base_url.format(str(x))
        resp = urllib2.urlopen(formatted_url)
        data = resp.read()
        j_data = json.loads(data)
        events = map(format_event, j_data["events"])

        for event in events:
            events_file.write(",".join(event))

        print "wrote out events for page - {}".format(x)