Replace words in a string from a dataframe with words from a separate dataframe

Question

I have the following dataset: 

Date       User     comments
9/20/2019 user1    My car model is 600.
9/21/2019 user2    My car model is viper.
9/23/2019 user3    I have a car. The model is civic. 
9/23/2019 user4    Washington is name of the city. 
9/23/2019 user5    I like the freedom I feel when I drive my chevy. 

These are sample comments that were scrapped. I'm trying to use this dataframe:

Brand     Model
ford       600
chevrolet chevy
dodge     viper
honda     civic
pontiac    gto
honda     freed

I am trying to replace the model described in the comment on the dataframe with the brand.

Here is my code:

file = pd.read_csv('test_dataset.csv')
file['comments'] = file['comments'].astype(str)
file["comments"] = file["comments"].str.lower()
brandconverter = pd.read_csv("brandconverter.csv")
def replacemodel(comment):
    return pd.Series(comment).replace(brandconverter.set_index('Model')['Brand'], regex=True)[0]

file['test'] = file['comments'].apply(replacemodel)

My expected output should be:

     Date      User     comments                              test
    9/20/2019 user1    My car model is 600.               My car model is ford. 
    9/21/2019 user2    My car model is viper.             My car model is dodge.
    9/23/2019 user3    I have a car. The model is civic.  I have a car. The model is honda. 
    9/23/2019 user4    Washington is name of the city.    Washington is name of the city.

But the output I am getting is:

     Date      User     comments                              test
    9/20/2019 user1    My car model is 600.               My car model is ford. 
    9/21/2019 user2    My car model is viper.             My car model is dodge.
    9/23/2019 user3    I have a car. The model is civic.  I have a car. The model is honda. 
    9/23/2019 user4    Washington is name of the city.    Washinpontiacn is name of the city.

I would like my function to ignore when the car model is inside a word like in 'Washington'. At the moment, it is looking for any case where the model is present in the comment even if it is inside a word. I would like the function to not consider the 'gto' in 'Washington'. I was hoping to apply this function to different comments too. This is just a sample.

Answer 1

You can use Series.replace with optional parameter regex=True to replace the model in comments with the corresponding brand from df2 :

s = brandconverter.set_index('Model')['Brand']
s.index = r'\b' + s.index + r'\b' # Takes care of word boundary condition

file['test'] = file['comments'].replace(s, regex=True)

Result:

       Date   User                           comments                               test
0  9/20/2019  user1               My car model is 600.              My car model is ford.
1  9/21/2019  user2             My car model is viper.             My car model is dodge.
2  9/23/2019  user3  I have a car. The model is civic.  I have a car. The model is honda.
3  9/23/2019  user4    Washington is name of the city.    Washington is name of the city.

Answer 2

You can use the following:

ids = {'from':['ford','chevrolet','dodge'],
      'to':['600','chevy','viper']}
ids = dict(zip(ids['from'], ids['to']))
df['test'] = df['comments'].replace(ids, regex=True)

Answer 3

You could try using your brandconverter dataframe as a dictionary, and then loop through it, this example doesn't have a loop but the key variable could easily be just an iterator:

import pandas as pd

file = pd.DataFrame({'User': ['user1', 'user2', 'user3', 'user4'],
                     'comments': ['A gto is what I love',
                                  'A gtoto is what I love',
                                  'Washington is name of the city.',
                                  'My car model is a gto.']})

brandconverter = {'gto': 'pontiac'}



key = 'gto'
file['test'] = file['comments'].replace(f'\\b{key}\\b', brandconverter[key], regex=True)

print(repr(file))

This prints out:

    User                         comments                             test
0  user1             A gto is what I love         A pontiac is what I love
1  user2           A gtoto is what I love           A gtoto is what I love
2  user3  Washington is name of the city.  Washington is name of the city.
3  user4           My car model is a gto.       My car model is a pontiac.