Python: How to select columns from a csv on python?

Question

New to Python and I've been trying to select these fields "Incident_Number, Address, Incident_Date, Primary_Situation, Property_Use, Supervisor_District, 'Neighborhood _District', Location" from a Fire inspection dataset.

I've tried with codes I've found on here but I get an invalid syntax. You might find this code here, but I've tweaked it a bit:

import csv
from collections import defaultdict

columns = defaultdict(list) # each value in each column is appended to a list

with open('Fire_Incidents_06052017.csv') as f:
    reader = csv.DictReader(f) # read rows into a dictionary format
    for row in reader: # read a row as {column1: value1, column2: value2,...}
        for (k,v) in row.items(): # go over each column name and value 
            columns[k].append(v) # append the value into the appropriate list
                                 # based on column name k
#Incident_Date, Primary_Situation, Property_Use, Supervisor_District, "Neighborhood _District", Location

print(columns['Incident_Number'])
print(columns['Address'])
print(columns['Incident_Date'])
print(columns['Primary_Situation'])
print(columns['Property_Use'])
print(columns['Supervisor_District'])
print(columns['Neighborhood _District'])
print(columns['Location'])

Is there an easier way to pull these columns with python? The csv is way too big for excel and SQLite to open/export without crashing.

Answer 1

A little searching led me to the Fire Incidents list for SF list. Using the first 10 rows:

Incident Number,Exposure Number,Address,Incident Date,Call Number,Alarm DtTm,Arrival DtTm,Close DtTm,City,Zipcode,Battalion,Station Area,Box,Suppression Units,Suppression Personnel,EMS Units,EMS Personnel,Other Units,Other Personnel,First Unit On Scene,Estimated Property Loss,Estimated Contents Loss,Fire Fatalities,Fire Injuries,Civilian Fatalities,Civilian Injuries,Number of Alarms,Primary Situation,Mutual Aid,Action Taken Primary,Action Taken Secondary,Action Taken Other,Detector Alerted Occupants,Property Use,Area of Fire Origin,Ignition Cause,Ignition Factor Primary,Ignition Factor Secondary,Heat Source,Item First Ignited,Human Factors Associated with Ignition,Structure Type,Structure Status,Floor of Fire Origin,Fire Spread,No Flame Spead,Number of floors with minimum damage,Number of floors with significant damage,Number of floors with heavy damage,Number of floors with extreme damage,Detectors Present,Detector Type,Detector Operation,Detector Effectiveness,Detector Failure Reason,Automatic Extinguishing System Present,Automatic Extinguishing Sytem Type,Automatic Extinguishing Sytem Perfomance,Automatic Extinguishing Sytem Failure Reason,Number of Sprinkler Heads Operating,Supervisor District,Neighborhood  District,Location
09030109,0,310 Colon Av.,04/12/2009,091020273,04/12/2009 06:09:13 PM,04/12/2009 06:13:45 PM,04/12/2009 07:23:13 PM,SF,,B09,15,,1,5,0,0,0,0,T15,,,0,0,0,0,,551 - assist pd or other govern. agency,none,52 - forcible entry,-,-,-,"000 - property use, other",,,,,,,,,,,,,,,,,,,,,,,,,,,,,
13067402,0,20 Lansdale Av,07/18/2013,131990117,07/18/2013 10:32:03 AM,07/18/2013 10:37:15 AM,07/18/2013 10:39:55 AM,SF,,B09,39,8571,3,11,0,0,0,0,E39,,,0,0,0,0,,745 - alarm system sounded/no fire-accidental,none,86 - investigate,-,-,-,429 - multifamily dwellings,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
12044490,0,7th St. / Folsom St.,05/13/2012,121340051,05/13/2012 03:55:37 AM,05/13/2012 04:01:57 AM,05/13/2012 04:05:44 AM,SF,94103,B03,01,,3,10,0,0,0,0,B03,,,0,0,0,0,,"711 - municipal alarm system, street box false",none,86 - investigate,-,-,-,963 - street or road in commercial area,,,,,,,,,,,,,,,,,,,,,,,,,,,6,South of Market,"(37.7767460000297, -122.407844)"
13033326,0,2799 Pacific Av,04/09/2013,130990286,04/09/2013 04:34:07 PM,04/09/2013 04:39:31 PM,04/09/2013 05:20:27 PM,SF,,B04,10,4163,3,10,0,0,0,0,B04,,,0,0,0,0,,"746 - co detector activation, no co",none,86 - investigate,-,-,-,419 - 1 or 2 family dwelling,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
11101416,0,Polk St. / Pine St.,11/01/2011,113050357,11/01/2011 06:07:45 PM,11/01/2011 06:10:17 PM,11/01/2011 06:11:09 PM,SF,94109,B04,03,,2,9,0,0,0,0,E03,,,0,0,0,0,,"711 - municipal alarm system, street box false",none,86 - investigate,-,-,-,"960 - street, other",,,,,,,,,,,,,,,,,,,,,,,,,,,3,Nob Hill,"(37.7896190000297, -122.420497)"
12077793,0,2824 Scott St.,08/22/2012,122350085,08/22/2012 08:58:06 AM,08/22/2012 09:10:06 AM,08/22/2012 09:13:18 AM,SF,94123,B04,16,,3,11,0,0,0,0,E51,,,0,0,0,0,,"700 - false alarm or false call, other",none,86 - investigate,-,-,-,"400 - residential, other",,,,,,,,,,,,,,,,,,,,,,,,,,,2,Marina,"(37.7960234800297, -122.43998326)"
07021713,0,San Bruno Av. / Teddy Av.,03/15/2007,070740223,03/15/2007 01:55:35 PM,03/15/2007 01:59:24 PM,03/15/2007 01:59:56 PM,SF,94134,B10,44,,1,4,0,0,0,0,E42,,,0,0,0,0,,"711 - municipal alarm system, street box false",none,86 - investigate,-,-,-,"962 - residential street, road or residential dr",,,,,,,,,,,,,,,,,,,,,,,,,,,10,Visitacion Valley,"(37.7129950000296, -122.4021)"
15065212,0,1532 Noriega Street,06/20/2015,151710341,06/20/2015 02:44:56 AM,06/20/2015 02:51:39 AM,06/20/2015 03:18:48 AM,San Francisco,94122,B08,40,7431,8,28,1,2,1,2,,800,,0,0,0,0,,"113 cooking fire, confined to container",n none,52 forcible entry,,,2 detector did not alert occupants,161 restaurant or cafeteria,,,,,,,,,,,,na,,,,,,,,,,,,,,,4,Sunset/Parkside,"(37.7541490000296, -122.480073)"
10097225,0,1215 Carolina St.,10/31/2010,103040303,10/31/2010 05:14:49 PM,10/31/2010 05:19:50 PM,10/31/2010 05:27:14 PM,SF,94107,B10,37,,3,10,0,0,0,0,E37,,,0,0,0,0,,745 - alarm system sounded/no fire-accidental,none,86 - investigate,-,-,-,"500 - mercantile, business, other",,,,,,,,,,,,,,,,,,,,,,,,,,,10,Potrero Hill,"(37.7533230000297, -122.399403)"

and the following code:

import csv
columns = {}
with open("incidents.csv", "r") as f:
    reader = csv.DictReader(f)
    for row in reader:
        for key, value in row.items():
            try:
                columns[key].append(value)
            except KeyError:
                columns[key] = [value]

print(columns['Incident Number'])

resulted in the following output:

['09030109', '13067402', '12044490', '13033326', '11101416', '12077793', '07021713', '15065212', '10097225']

Using your code, which results in a similar data structure gives me the same result.

A thing to note, there are two different columns with similar names in the dataset. Neighborhood District and Neighborhood District . It appears the column with two spaces in the one with data in it.

After all this I can only conclude there was some kind of typo in your code that didn't get transferred here to stackoverflow. Could you please post your error in full?