How to use regexp in endswith() in conditional subsetting of pandas df column?

Question

I want to use .endswith() or regexp in conditional subsetting of Sender name column in my dataframe.

Dataframe df has two columns Sender email , Sender name which I will use to define a subsetting rule, to select all mail coming from a specific shop and specific email of this shop:

df = df[(df["Sender name"]=="Shop_name"]) & (df["Sender email"]=="reply@shop.com")]

• But then I figured out that there are also mails from buy@shop.com , noreply@shop.com , etc. Is there an any way to neatly introduce all this mailboxes into something like *@shop.com in the second condition?

• I tried using endswith() , but couldn't figure out how to make it work for series object. I figured out I may first form a list with all mails from the column, and then check if sending mailserver is in it with pd.Series.isin . But maybe there is something more elegant out there?

Answer 1

Use Series.str.endswith or Series.str.contains with regex - $ for end of string and also escape . by \\ , because . is special regex value - any character:

df1 = df[(df["Sender name"]=="Shop_name"]) & (df["Sender email"].str.endswith("@shop.com"))]

Or:

df1 = df[(df["Sender name"]=="Shop_name"]) & (df["Sender email"].str.contains("@shop\.com$"))]

Answer 2

Using .query

Since pandas >= 0.25.0 we can use .query with pandas methods ( .eq & str.endswith ) and using the backtick (`) to query column names with spaces:

df.query('`Sender name`.eq("Shop_name") & `Sender email`.str.endswith("@shop.com")')

Output

       Sender email Sender name
2    reply@shop.com   Shop_name
3      buy@shop.com   Shop_name
4  noreply@shop.com   Shop_name

Example dataframe used:

# Example dataframe
df = pd.DataFrame({'Sender email':['ex@example.com', 'ex2@example.com', "reply@shop.com", "buy@shop.com", "noreply@shop.com"],
                   'Sender name': ['example', 'example', 'Shop_name', 'Shop_name', 'Shop_name']})

       Sender email Sender name
0    ex@example.com     example
1   ex2@example.com     example
2    reply@shop.com   Shop_name
3      buy@shop.com   Shop_name
4  noreply@shop.com   Shop_name