Map 通过搜索另一个新列的值 dataframe

Question

I have two dataframes: df_geo and df_event . I want to create two new columns in df_event . The data frames resemble the following, although additional columns have been removed for the sake of simplicity:

data_geo =  [['040','01','000','00000','00000','00000','Alabama'],
             ['050','01','001','00000','00000','00000','Autauga County'],
             ['050','01','097','00000','00000','00000','Mobile County'],
             ['050','01','101','00000','00000','00000','Montgomery County'],
             ['050','01','115','00000','00000','00000','St. Clair County'],
             ['040','09','000','00000','00000','00000','Connecticut'],
             ['061','09','001','04720','00000','00000','Bethel town'],
             ['040','17','000','00000','00000','00000','Illinois'],
             ['061','17','109','05638','00000','00000','Bethel township'],
             ['050','17','163','00000','00000','00000','St. Clair County']] 

dfgeo = pd.DataFrame(data_geo, columns = ['summary_level', 'state_fips','county_fips','subdivision_code_fips','place_code_fips','city_code_fips','area_name']) 

df_geo.info()

RangeIndex: 43847 entries, 0 to 43846
Data columns (total 7 columns):
summary_level            43847 non-null object
state_fips               43847 non-null object
county_fips              43847 non-null object
subdivision_code_fips    43847 non-null object
place_code_fips          43847 non-null object
city_code_fips           43847 non-null object
area_name                43847 non-null object

data_event = [['event_id','_','Alabama'], 
              ['event_id','_','Connecticut'],
              ['event_id','Autauga County','Alabama'],
              ['event_id','Fairfield County','Connecticut'],
              ['event_id','Fairbanks North Star Borough','Alaska']] 

df_event = pd.DataFrame(data_event, columns = ['event_id','county','state']) 

df_event.info()

RangeIndex: 1261 entries, 0 to 1260
Data columns (total 3 columns):
event_id                1261 non-null object
county                   999 non-null object
state                   1261 non-null object
dtypes: object(3) 

GOAL to create a function that can take the county and state inputs from df_event and in order to create two new columns in the same dataframe. The new columns are based on the values of state_fips and county_fips in df_geo . An example of this would look like the following:

inputA fun('df_geo','Connecticut','Fairfield County'):   

resultA = ['event_id','Connecticut','Fairfield County','09','001']
                                                       ^New columns

inputB fun('df_geo','Alaska','Fairbanks North Star Borough'):   

resultB = ['event_id','Alaska','Fairbanks North Star Borough','02','090']
                                                              ^New columns

This is a PROBLEM because I also need to use this function on a list of 1,200 (and growing) events the function would have to work within a lamba function or something else that can map it across the entire dataframe.

This is complicated by identical County names like "St. Clair County" that appear in several states. Although their area_names are identical the value of state_fips will be different.

The state_fips of St. Clair Illinois is 17 , the same as all other counties in Illinois and the state itself. The state_fips of St. Clair Alabama is 01 , the same as all other counties in Alabama, and so on...

I would like to use the same search and map function all the way down to city_code_fips . At that level any search terms have to be exactly the same to avoid picking up "Bethel town" when I intend to find "Bethel township". Exact inputs are also important because some states, like Louisiana, call their county level geographies by another name.

In df_event a '_' indicates that the county is unknown.

df_event['event_id'] is a unique string. There are rows in the dataframe that are nearly identical but with different ids indicating that an event has occurred on multiple occasions. This has no affect on the. state_fips or county_fips .

I know this is a multi step process but all help is appreciated. Thank you.

Answer 1

You can do this using df.merge :

In [289]: df_event['state_fips'] = df_event.merge(dfgeo[['state_fips','area_name']], left_on='state', right_on='area_name', how='left')['state_fips']    
In [290]: df_event['county_fips'] = df_event.merge(dfgeo[['county_fips','area_name']], left_on='county', right_on='area_name', how='left')['county_fips']

In [291]: df_event
Out[291]: 
  unique_str                        county        state state_fips county_fips
0   Event Id                             _      Alabama         01         NaN
1   Event Id                             _  Connecticut         09         NaN
2   Event Id                Autauga County      Alabama         01         001
3   Event Id              Fairfield County  Connecticut         09         001
4   Event Id  Fairbanks North Star Borough       Alaska         02         090

Answer 2

If there are duplicates in use area_name column first remove them by DataFrame.drop_duplicates :

dfgeo = dfgeo.drop_duplicates('area_name')

And then Series.map , what is faster like merge , so it should be preferable:

df_event['state_fips'] = df_event['state'].map(dfgeo.set_index('area_name')['state_fips'])
df_event['county_fips'] = df_event['county'].map(dfgeo.set_index('area_name')['county_fips'])
print (df_event)
  unique_str                        county        state state_fips county_fips
0   Event Id                             _      Alabama         01         NaN
1   Event Id                             _  Connecticut         09         NaN
2   Event Id                Autauga County      Alabama         01         001
3   Event Id              Fairfield County  Connecticut         09         001
4   Event Id  Fairbanks North Star Borough       Alaska         02         090