Python pandas dataframe：在数组列中，如果第一项包含特定字符串，则从数组中删除该项

Question

I have a dataframe which has some column like below which contains arrays of different sizes:

column
["a_id","b","c","d"]
["d_ID","e","f"]
["h","i","j","k","l"]
["id_m","n","o","p"]
["ID_q","r","s"]

I want to remove first item from the array of every row if the first item contains "ID" or "id". So, expected output will look like:

column
["b","c","d"]
["e","f"]
["h","i","j","k","l"]
["n","o","p"]
["r","s"]

How do we check for this in the column containing array elements in the dataframe?

Answer 1

Edit: It seems I misread your question. This solution is meant to remove any element that has 'id' in it, not just the first.

Option 1
I believe the most straightforward solution is using apply :

df

               col
0  [a_id, b, c, d]
1     [d_ID, e, f]
2  [h, i, j, k, l]
3  [id_m, n, o, p]
4     [ID_q, r, s]


df.col = df.col.apply(lambda y: (y[1:] if 'id' in y[0].lower() else y))

df
               col
0        [b, c, d]
1           [e, f]
2  [h, i, j, k, l]
3        [n, o, p]
4           [r, s]

---

Option 2
Alternatively, use a list comprehension :

df.col = [(y[1:] if 'id' in y[0].lower() else y)  for y in df.col]  

df

               col
0        [b, c, d]
1           [e, f]
2  [h, i, j, k, l]
3        [n, o, p]
4           [r, s]

---

Timings

df = pd.concat([df] * 100000)

%%timeit
m = df['col'].str[0].str.contains('ID', case=False)
df['col'].mask(m, df['col'].str[1:])

1 loop, best of 3: 917 ms per loop

%timeit [(y[1:] if 'id' in y[0].lower() else y)  for y in df.col]  
1 loop, best of 3: 272 ms per loop

%timeit df.col.apply(lambda y: (y[1:] if 'id' in y[0].lower() else y))
1 loop, best of 3: 309 ms per loop

Answer 2

Use str[0] for select first values in list and then check ID by contains :

m = df['column'].str[0].str.contains('ID', case=False)
print (m)
0     True
1     True
2    False
3     True
4     True
Name: column, dtype: bool

And then remove it by mask with str[1:] :

df['column'] = df['column'].mask(m, df['column'].str[1:])
print (df)
            column
0        [b, c, d]
1           [e, f]
2  [h, i, j, k, l]
3        [n, o, p]
4           [r, s]