Remove specific words from column using Python

Question

The data originally is derived from PDF for doing further analysis on the data, There is an [identity] column where some the values are spelled wrong, ie it contains wrong spelling or Special characters.

Looking out to remove the Unwanted characters from the column.

Input Data:

identity

UK25463AC
ID:- UN67342OM
#ID!?
USA5673OP

Expected Output:

identity

UK25463AC
UN67342OM
NAN
USA5673OP

Script I have Tried so far:

stop_word = ['#ID!?','ID:-']
pat = '|'.join(r"\b{}\b".format(x) for x in stop_words)
df['identity'] = df['identity'].str.replace(pat, '')

So I have no clue how to handle this problem

Answer 1

From expected output is necessary remove words boundaries \b\b and because special regex chcarecer is added re.escape , then is used Series.replace for empty string and if only empty string to missing value:

import re
stop_words = ['#ID!?','ID:-']
pat = '|'.join(r"{}".format(re.escape(x)) for x in stop_words)
df['identity'] = df['identity'].replace(pat, '', regex=True).replace('', np.nan)
print (df)
     identity
0   UK25463AC
1   UN67342OM
2         NaN
3   USA5673OP