选择在一列中具有相同值而在另一列中具有不同值的行 Pandas Python

Question

I have a dataframe with about 400,000 rows but I want to correct some things in it. So I want to select all the rows that have an equal value to the previous one in one column and a different value that the previous one in another column. In other words, x[i] == x[i+1] AND x[j].= x[j+1]. So what I thought about doing is sorting the values using i and j then shifting the dataframe and grouping by. However I am having problems getting out the dataset

Example dataset:

 i         j           year       
"foo"     "jar"          5
"foo"     "jam"          5
"hi"      "hell"         6
"hi"      "hello"        6
"good"    "happy"        8
"bad"     "happy"        8
"happy"   "good"         8

Desidered output:

 i         j           year       
"foo"     "jar"          5
"foo"     "jam"          5
"hi"      "hell"         6
"hi"      "hello"        6

Current Code:

shifteddf = df.shift()

df1 = df[df["i"]==shifteddf["i"]]

df2 = df[df["j"]!=shifteddf["j"]]

pd.merge(df1, df2, left_index=True, right_index=True)

Current output:

 i         j           year       
"foo"     "jam"          5
"hi"      "hello"        6

So I am missing some rows, and I think I am currently doing a fatal error

Answer 1

This can be done based on identical strings and length of strings.

Method 1

Based on strings You need eq with shift:

match = df.i.eq(df.i.shift())
print(df[match | match.shift(-1)])

Output #

     i      j           year         
0  foo    jar            5    
1  foo    jam            5    
2   hi   hell            6    
3   hi  hello            6    

Method 2

If you want by length then simply calculate length first and then group accordingly

df['len'] = df['i'].astype(str).map(len)
match = (df.len.groupby(df.i).diff().eq(0))

print(df[match | match.shift(-1)])

which also outputs #

     i      j          year  len
0  foo    jar            5    3
1  foo    jam            5    3
2   hi   hell            6    2
3   hi  hello            6    2