比较 Dataframe 两列的相同和不同

Question

I have a small dataframe, like this.

import pandas as pd
import numpy as np
 
# data's stored in dictionary
details = {
    'address_id': [1, 1, 1, 2, 2],
    'business': ['verizon', 'verizon', 'comcast', 'sprint', 'att']
}
 
df = pd.DataFrame(details)
 

print(df)

I am trying to find out if, and when a person switched to a different cell phone service.

I tried this logic; didn't work.

df['new'] = df.Column1.isin(df.Column1) & df[~df.Column2.isin(df.Column2)] 

Basically, given index row 0 and row 1, when the address_id was the same, the business was the same, but the business changed from verizon to comcast in index row 2. Also, given index row 3 and row 4, the address_id was the same, but the business changed from sprint to att in index row 4. I'd like to add a new column to the dataframe to flag these changes. How can I do that?

Answer 1

UPDATE : Here is an even simpler way than my original answer using join() (see below) to do what your question asks:

df['new'] = df.address_id.map(df.groupby('address_id').first().business) != df.business

Explanation:

• Use groupby() and first() to create a dataframe whose business column contains the first one encountered for each address_id
• Use Series.map() to transform the original dataframe's address_id column into this first business value
• Add column new which is True only if this new business differs from the original business column.

---

Here is a simple way to do what you've asked using groupby() and join() :

df = df.join(df.groupby('address_id').first(), on='address_id', rsuffix='_first')
df = df.assign(new=df.business != df.business_first).drop(columns='business_first')f

Output:

   address_id business    new
0           1  verizon  False
1           1  verizon  False
2           1  comcast   True
3           2   sprint  False
4           2      att   True

Explanation:

• Use groupby() and first() to create a dataframe whose business column contains the first one encountered for each address_id
• Use join() to add a column business_first to df containing the corresponding first business for each address_id
• Use assign() to add a column new containing a boolean indicating whether the row contains a new business with an existing address_id
• Use drop() to eliminate the business_first column.

Answer 2

First, groupby on address_id .

groups = df.groupby("address_id")

Then, iterate over the groups, and find where the value of business changes:

for address_id, grp_data in groups:
    changed = grp_data['business'].ne(grp_data['business'].shift().bfill())
   
    df.loc[grp_data.index, "changed"] = changed

.shift().bfill() shifts all data one index over ( 0 -> 1 , 1 -> 2 , and so on), and then backfills the first value. For example:

>>> df["business"]
0    verizon
1    verizon
2    comcast
3     sprint
4        att
Name: business, dtype: object

>>> df["business"].shift()
0        NaN
1    verizon
2    verizon
3    comcast
4     sprint
Name: business, dtype: object

>>> df["business"].shift().bfill()
0    verizon
1    verizon
2    verizon
3    comcast
4     sprint
Name: business, dtype: object

Running the loop makes the following dataframe:

   address_id business changed
0           1  verizon   False
1           1  verizon   False
2           1  comcast    True
3           2   sprint   False
4           2      att    True