remove characters from a python string

Question

I have several python strings from which I want unwanted characters removed.

Examples:

"This is '-' a test" 
     should be "This is a test"
"This is a test L)[_U_O-Y OH : l’J1.l'}/"
     should be "This is a test"
"> FOO < BAR" 
     should be "FOO BAR"
"I<<W5§!‘1“¢!°\" I" 
     should be "" 
     (because if only words are extracted then it returns I W I and none of them form words)
"l‘?£§l%nbia  ;‘\\~siI.ve_rswinq m"
     should be ""
"2|'J]B"
     should be ""

this is what I have so far, however, it is not keeping the original spaces between words.

>>> line = re.sub(r"\W+","","This is '-' a test")
>>> line
'Thisisatest'
>>> line = re.sub(r"\W+","","This is a test L)[_U_O-Y OH : l’J1.l'}/")
>>> line
'ThisisatestL_U_OYOHlJ1l' 
#although i would prefer this to be "This is a test" but if not possible i would 
 prefer "This is a test L_U_OYOHlJ1l"
>>> line = re.sub(r"\W+","","> FOO < BAR")
>>> line
'FOOBAR'
>>> line = re.sub(r"\W+","","I<<W5§!‘1“¢!°\" I")
>>> line
'IW51I'
>>> line = re.sub(r"\W+","","l‘?£§l%nbia  ;‘\\~siI.ve_rswinq m")  
>>> line
'llnbiasiIve_rswinqm'
>>> line = re.sub(r"\W+","","2|'J]B")
>>> line
'2JB'

I will be filtering the regex cleaned words through a list of predefined words later.

Answer 1

I'd go with a split and filter, like this:

' '.join(word for word in line.split() if word.isalpha() and word.lower() in list)

This will remove all non-alphabetic words and alphabetic words that are not in the list.

Examples:

def myfilter(string):
    words = {'this', 'test', 'i', 'a', 'foo', 'bar'}
    return ' '.join(word for word in line.split() if word.isalpha() and word.lower() in words)

>>> myfilter("This is '-' a test")
'This a test'
>>> myfilter("This is a test L)[_U_O-Y OH : l’J1.l'}/")
'This a test'
>>> myfilter("> FOO < BAR")
'FOO BAR'
>>> myfilter("I<<W5§!‘1“¢!°\" I")
'I'
>>> myfilter("l‘?£§l%nbia  ;‘\\~siI.ve_rswinq m")
''
>>> myfilter("2|'J]B")
''

Answer 2

This one clears out any group of non-space symbols with at least one non alphabetic character. It will leaves some unwanted group of letters though :

re.sub(r"\w*[^a-zA-Z ]+\w*","","This is a test L)[_U_O-Y OH : l’J1.l'}/")

gives :

'This is a test  OH  '

It will also leave groups of more than one space :

re.sub(r"[^a-zA-Z ]+\w*","","This is '-' a test")
'This is  a test'  # two spaces