如何从列中的字符串中提取与 python 列表中的另一个字符串匹配的 substring

Question

I have a dataframe which is as follows:

     col 1                                     col 2
0       59       538 Walton Avenue, Chester, FY6 7NP
1       62 42 Chesterton Road, Peterborough, FR7 2NY
2      179       3 Wallbridge Street, Essex, 4HG 3HT
3      180     6 Stevenage Avenue, Coventry, 7PY 9NP

With a list similar to:

[Stevenage, Essex, Coventry, Chester]

Following the solution from here: How to check if Pandas rows contain any full string or substring of a list? which went like this:

city_list = list(cities["name"])
df["col3"] = np.where(df["col2"].str.contains('|'.join(city_list)), df["col2"], '')

I found that some in col 2 match the strings in a list but that the col3 is the same as col2. I want col3 to be the values in the list rather the same as col3. This would be:

     col 1                                     col 2     col3
0       59       538 Walton Avenue, Chester, FY6 7NP  Chester 
1       62 42 Chesterton Road, Peterborough, FR7 2NY 
2      179       3 Wallbridge Street, Essex, 4HG 3HT    Essex
3      180     6 Stevenage Avenue, Coventry, 7PY 9NP Coventry

I have tried:

pat = "|".join(cities.name)
df.insert(0, "name", df["col2"].str.extract('(' + pat + ')', expand = False))

But this returned an error saying 456 inputs when expecting 1.

Also:

df["col2"] = df["col2"].apply(lambda x: difflib.get_close_matches(x, cities["name"])[0])
df.merge(cities)

But this came back with the error list index out of range.

Is there anyway to do this? df1 is around 160,000 entries with each address in col2 from different countries so there is no standard way they are written, while the city list is around 170,000 entries

Thank you

Answer 1

You could do as follows:

city_list = ["Stevenage", "Essex", "Coventry", "Chester"]

def get_match(row):
    col_2 = row["col 2"].replace(",", " ").split() # Here you should process the string as you want
    for c in city_list:
        if difflib.get_close_matches(col_2, c)
            return c
    return ""

df["col 3"] = df.apply(lambda row: get_match(row), axis=1)

Answer 2

Lean on an auxiliary function like this:

df = pd.DataFrame({'col 1': [59, 62, 179, 180],
                   'col 2': ['538 Walton Avenue, Chester, FY6 7NP',
                             '42 Chesterton Road, Peterborough, FR7 2NY',
                             '3 Wallbridge Street, Essex, 4HG 3HT',
                             '6 Stevenage Avenue, Coventry, 7PY 9NP'
                             ]})

def aux_func(x):

    # split by comma and select the interesting part ([1])
    x = x.split(',')
    x = x[1]

    aux_list = ['Stevenage', 'Essex', 'Coventry', 'Chester']
    for v in aux_list:
        if v in x:
            return v
    return ""

df['col 3'] = [aux_func(name) for name in df['col 2']]

Answer 3

have a look at str.contains function that tests if a pattern match a series:

df = pd.DataFrame([[59, '538 Walton Avenue, Chester,', 'FY6 7NP'],
                   [62, '42 Chesterton Road, Peterborough', '4HG 3HT'],
                   [179, '3 Wallbridge Street, Essex', '4HG 3HT'],
                   [180, '6 Stevenage Avenue, Coventry', '7PY 9NP']])
city_list = ["Stevenage", "Essex", "Coventry", "Chester"]
for city in city_list:
    df.loc[df[1].str.contains(city), 'match'] = city

Answer 4

Thanks for your last replied, try this

def aux_func(address):
    aux_list = ['Stevenage', 'Essex', 'Coventry', 'Chester']

    # remove commas
    address = address.split(',')

    # avoide matches with the first part of the address
    if len(address)>1:
        # remove the first element of the address
        address = address[1:]

    for v in aux_list:
        for chunk in address:
            if v in chunk:
                return v

    return ""

df['col 3'] = [aux_func(address) for address in df['col 2']]