How to read CSV file column by column with Python

Question

I have a CSV file in the following format

  Date, Company, Company, Company
1/1/12,      10,     100,      50
1/2/12,      12,      99,      53
1/3/12,      11,      97,      49

I'm trying to input the data into a PSQL database.

How would I go about going column by column on the data, so that I would have have something like INSERT INTO table VALUES(company, date, price); ?

Each column corresponds to a company

Answer 1

I'm wondering if something like this would work:

import csv

with open("file.csv") as f:
    reader = csv.reader(f)
    for i, column in enumerate(zip(*reader)):
        if i == 0:
            _, dates = column
        else:
            # PY3.x
            company, *prices = column
            # PY2.7
            company, prices = column[0], column[1:]

            # Do SQL command here

The idea is to use to transpose the data that is read in by rows with csv.reader by using zip(*reader). I can't test this out right now but you might have to use zip(*list(reader)) to transpose all the data, however this will load the entire file and probably make a copy. Since your data is small that's probably ok.

For this size of data you can also use pandas. Which would just be something like:

import pandas as pd

data = pd.read_csv ("file.csv", index_col=0, parse_dates=False)

dates = data.index.values

for company in data.columns:
    price = data[company].values

    #SQL command here