检查数据帧中的结束字符并替换它们

Question

I would like to add two new columns in my pandas dataframe based on the following conditions

• if a sentence ends with '...' then add a new column with value 1, otherwise 0;
• if a sentence ends with '...' then add a new column without '...' at the end

Something like this:

Text
bla bla bla ...
once upon a time
pretty little liars
Batman ...

Expected

    Text                T    Clean
    bla bla bla ...     1    bla bla bla 
    once upon a time    0    once upon a time 
    pretty little liars 0    pretty little liars
    Batman ...          1    Batman

I tried to apply regex, but probably str.endwith would be a better approach to check if a sentence ends with ..., since assigns a boolean value (my T column).

I have tried as follows: df['Text'].str.endswith('...') but I would need to create a new column with 1 and 0. For cleaning the text I would check if T is true: if it is true, I would remove the ... at the end.

df['Clean'] = df['Text'].str.rstrip('...') 

or df['Clean'] = df['Text'].str[:-3] (but it does not include any logical condition or information on ... )

or df['Clean'] = df['Text'].str.replace(r'...$', '')

It is important that I consider the sentence ending with ... in order to avoid to delete ... in the middle of sentence which have a different meaning.

Answer 1

For the first column, I would use the approach you suggested:

df['T'] = df['Text'].str.endswith('...')

(Technically this will create a boolean column, not an integer column. You can use astype() to convert if you care about this.)

For the second column, I would unconditionally replace:

df['Clean'] = df['Text'].str.replace(r'...$', '')

If it doesn't end in ..., it won't do anything.

Answer 2

In case you want to replace the "ending" ellipsis only on those text rows with that property:

df.loc[df['Text'].str.endswith('...') == True, 'ends_in_ellipsis'] = 1

df.loc[df['ends_in_ellipsis'] == 1, 'Text_2'] = df.loc[df['ends_in_ellipsis'] == 1, 'Text'].str.rstrip('...') 

Now if you want to do it all in one line (although less readable for others but you save a dummy column and the memory it takes up):

df.loc[df['Text'].str.endswith('...') == True, 'Text_2'] = df.loc[df['Text'].str.endswith('...') == True, 'Text'].str.rstrip('...') 

Answer 3

Let us try endswith + rstrip

df['new1']=df.Text.str.endswith('...').astype(int)
df['new2']=df.Text.str.rstrip(' ...') # notice rstrip will not remove any ... in the mid 
df
                  Text  new1                 new2
0      bla bla bla ...     1          bla bla bla
1     once upon a time     0     once upon a time
2  pretty little liars     0  pretty little liars
3           Batman ...     1               Batman