熊猫，将元组列表列表转换为DataFrame尴尬的列标题。

Question

I have data from parsed addresses that I obtained from the usaddress python library: https://github.com/datamade/usaddress

The data is a list of lists of tuples. Each address has a list like this associated with it:

[('Robie', 'BuildingName'),
('House,', 'BuildingName'),
('5757', 'AddressNumber'),
('South', 'StreetNamePreDirectional'),
('Woodlawn', 'StreetName'),
('Avenue,', 'StreetNamePostType'),
('Chicago,', 'PlaceName'),
('IL', 'StateName'),
('60637', 'ZipCode')]

However, for some addresses a certain field may, or may not be present. I want to export this data into a pandas DataFrame with all the column headers (BuildingName, Address...ect) and if that that column header isn't present in the list, then the cell is just left blank.

What I have at the moment is:

newAddr = []
for index, row in df.iterrows():
    newAddr.append(usaddr.parse(row['FullAddress']))

df2 = DataFrame(newAddr)

But this produces a file with no column headers and no real organization by column, since the missing values just shift everything over.

Help is greatly appreciated.

Answer 1

Assuming the following:

• You use usaddress.tag
• have ways to handle the errors that may be raised from usaddress.tag
• only want the first part of the return from usaddress.tag

Then, you can do the following

import usaddress
import pandas as pd

# your list of addresses dataframe
df = pd.read_csv('PATH_TO_ADDRESS_CSV')

# list of orderedDict
ordered_dicts = []

# loop through addresses and get respective information
for index, row in df.iterrows():
    # here you should try/except for cases that fail
    addr = usaddress.tag(row['FullAddress'])

    # append to list
    ordered_dicts.append(addr[0])

# **get all relevant keys in your list
cols = set().union(*(d.keys() for d in ordered_dicts))

# create new dataframe
df_new = pd.DataFrame(ordered_dicts, columns=cols)

df_new.to_csv('PATH_TO_DESIRED_CSV_ENDPOINT')

The ** represents an alternative solution to this part of the function. Because we know exactly all the columns that the .tag function can return, you can just initially set the columns as such (see all tags here and API here ):

cols = ['AddressNumberPrefix', 'AddressNumber', ...]

I hope this helps! Know that when you do pd.DataFrame with dictionaries and specify exact columns, it will automatically fill in the non-existing keys with pd.NaN .