从文件中删除所有非字母数字字符，并在python中以相同的名称保存

Question

I have a .txt document containing a long body of text mixed with some special characters that I want to remove. I want to do something like re.findAll to extract all words and save the file with the other characters filtered out. How can I open the file, extract all non alphanumeric characters, and then save it with the same file name (with all spaces in place, obviously)? Is there a better way with re.sub?

import re
hand = open('document.txt')
for line in hand:
    line = line.rstrip()
    re.sub(r'\W+', ' ', line) 

The document looks like this: But was it a ride after! I loved all the characters mainly because everyone in the book has shades of gray and that is how real characters are supposed to be. The emotions were real and took their time to settle in and yet the story was fast paced. Definitely recommend. Hsnan Hn wqt \\'n \\'ltqT \\'nfsy w\\'`yd lhdw l~Wan b`d tlk lrHl@ lmt`b@ lmlyy\\'@ blGmwD wlbHth wlshkwk ..fy lbdy@ , `lyk `zyzy lqry lntbh qbl qr@ tlk lrwy@ l~ `d@ \\'shy , \\'wlh \\'n lrwy@ lyst rwmnsy@ \\'w drm `ks m ywHy smh , fh~ rwy@ tntmy l~ lfy\\'@ lbwlysy@ , fy\\'@ ljry\\'m wlkhywT lmtshbk@ .thny tlk l\\'shy hw \\'n tlk lrwy@ stj`lk mtsmran \\'mmh s`t Twyl@ dwn mll , wmn thm `lyk \\'n tnthy mn mshGlk wtj`l nfsk \\'syran lhdh l`ml dwn swh ..fy \\'wl~ lSfHt , stjd nfsk l tstw`bm yHdth , wstjd nfsk ttsl mn hw\\'l ? wm l`lq@ bynhm ? wlknh lbdy@ fqT , `lyk \\'n tmsk blkhywT lbdy\\'y@ wb`d dhlk \\'trk nfsk tmman dwn ltfkyr fy shy , stjd \\'nl\\'Hdth lmttly@ wlmt`qb@ \\'d@ jdhb l ymknk ltGDy `nh ..l twjd shkhSy@ ry\\'ysy@ fy tlk lrwy@ , jmy` lshkhSyt lwrd@ lhm \\'dwr mHwry@ , rytshyl twm an skwt myGyn kml abdyk , wlkn ymkn lqwl \\'n lqTr hw syd lmwqf hn fy tlk lrwy@ , fmn khllh tbd\\' l\\'Hdth wttTwr wlwlh lm wjdt lHbk@ \\'w wsyl@ tS`dl\\'mwr ..rGm \\'n tslsl l\\'Hdth dkhl lrwy@ wlntql mn lmDy l~ lHDr wl`ks hw \\'mr mrhq l \\'n dhlk \\'Df~ lmzyd mn lthr@ wltshwyq dkhl tlk lrwy@ dwn l`ml `l~ Hrq l\\'Hdth , bl`ks tj`ll\\'mr wk\\'nh \\'shbh bfsyfs ttDH m`lmh klm tkmlt mkwnth ..ltrjm@ ? l Gbr `lyh , wlm \\'sh`r wk\\'nny \\'mm `ml mtrjm mn l\\'Sl ..\\'slwb lm`lj@ mmtz , lHbk@ ldrmy@ mmtz@ , lktb@ njHt fy stGll kf@ mqwmt lktb@ lnjH@ w\\'khrjt ln `mlan mmyzan khS@ w\\'nny l \\'myl l~ fy\\'@ lrwyt lbwlysy@ wlkn l\\'mr \\'khtlf hn ..mlHwZ@ \\'khyr@ : `ndm nZrt l~ Swr@ lktb@ fy nhy@ tlk lrwy@ sh`rt wk\\'nh qtl@ But was it a ride after! I loved all the characters mainly because everyone in the book has shades of gray and that is how real characters are supposed to be. The emotions were real and took their time to settle in and yet the story was fast paced. Definitely recommend. Hsnan Hn wqt \\'n \\'ltqT \\'nfsy w\\'`yd lhdw l~Wan b`d tlk lrHl@ lmt`b@ lmlyy\\'@ blGmwD wlbHth wlshkwk ..fy lbdy@ , `lyk `zyzy lqry lntbh qbl qr@ tlk lrwy@ l~ `d@ \\'shy , \\'wlh \\'n lrwy@ lyst rwmnsy@ \\'w drm `ks m ywHy smh , fh~ rwy@ tntmy l~ lfy\\'@ lbwlysy@ , fy\\'@ ljry\\'m wlkhywT lmtshbk@ .thny tlk l\\'shy hw \\'n tlk lrwy@ stj`lk mtsmran \\'mmh s`t Twyl@ dwn mll , wmn thm `lyk \\'n tnthy mn mshGlk wtj`l nfsk \\'syran lhdh l`ml dwn swh ..fy \\'wl~ lSfHt , stjd nfsk l tstw`bm yHdth , wstjd nfsk ttsl mn hw\\'l ? wm l`lq@ bynhm ? wlknh lbdy@ fqT , `lyk \\'n tmsk blkhywT lbdy\\'y@ wb`d dhlk \\'trk nfsk tmman dwn ltfkyr fy shy , stjd \\'nl\\'Hdth lmttly@ wlmt`qb@ \\'d@ jdhb l ymknk ltGDy `nh ..l twjd shkhSy@ ry\\'ysy@ fy tlk lrwy@ , jmy` lshkhSyt lwrd@ lhm \\'dwr mHwry@ , rytshyl twm an skwt myGyn kml abdyk , wlkn ymkn lqwl \\'n lqTr hw syd lmwqf hn fy tlk lrwy@ , fmn khllh tbd\\' l\\'Hdth wttTwr wlwlh lm wjdt lHbk@ \\'w wsyl@ tS`dl\\'mwr ..rGm \\'n tslsl l\\'Hdth dkhl lrwy@ wlntql mn lmDy l~ lHDr wl`ks hw \\'mr mrhq l \\'n dhlk \\'Df~ lmzyd mn lthr@ wltshwyq dkhl tlk lrwy@ dwn l`ml `l~ Hrq l\\'Hdth , bl`ks tj`ll\\'mr wk\\'nh \\'shbh bfsyfs ttDH m`lmh klm tkmlt mkwnth ..ltrjm@ ? l Gbr `lyh , wlm \\'sh`r wk\\'nny \\'mm `ml mtrjm mn l\\'Sl ..\\'slwb lm`lj@ mmtz , lHbk@ ldrmy@ mmtz@ , lktb@ njHt fy stGll kf@ mqwmt lktb@ lnjH@ w\\'khrjt ln `mlan mmyzan khS@ w\\'nny l \\'myl l~ fy\\'@ lrwyt lbwlysy@ wlkn l\\'mr \\'khtlf hn ..mlHwZ@ \\'khyr@ : `ndm nZrt l~ Swr@ lktb@ fy nhy@ tlk lrwy@ sh`rt wk\\'nh qtl@

I want to delete all special characters, punctuation, and leave only the [a-zA-Z0-9] characters.

Answer 1

\\W is the metacharacter signifying any non-alphanumeric character.

Create a list to store your lines that have been stripped of non-alpha/num characters, and then write these lines back to the same file.

import re

with open('document.txt') as hand:
    lines = []
    for line in hand:
        lines.append(re.sub("[\W]", "", line))

with open('document.txt', 'w') as hand:
    for line in lines:
        hand.write(line)

Output:

To keep the spaces:

re.sub("[^\s\w]+", "", line)

Answer 2

You can str.translate if you only have ascii in your file and write to a tempfile , then replace the original using shutil.move after writing.:

from tempfile import NamedTemporaryFile
from shutil import move


with open("document.txt") as f, NamedTemporaryFile(dir=".", delete=False) as tmp:
   _del =  "".join(filter(lambda x: not x.isalnum(), map(chr, range(256)))).replace(" ", "")
   for line in f:    
        tmp.write(line.translate(None, _del))

move(tmp.name, "document.txt")

An example using translate on a snippet of your data:

 In [31]: s = '''The emotions were real and took their time to settle
 in and yet the story was fast paced. Definitely recommend.
 Hsnan Hn wqt \'n \'ltqT \'nfsy w\'`yd lhdw l~Wan b`d tlk lrHl@ lmt`b@ 
lmlyy\'@ blGmwD wlbHth wlshkwk ..fy lbdy@ , `lyk `zyzy lqry lntbh qbl 
qr@ tlk lrwy@ l~ `d@ \'shy , \'wlh \'n lrwy@ lyst rwmnsy@ \'w drm 
`ks m ywHy smh , fh~ rwy@ tntmy l~ lfy\'@ lbwlysy@ , fy\'@ ljry\'m wlkhywT lmtshbk@ .thny tlk l\'shy hw \'n tlk lrwy@ stj`lk mtsmran \'mmh s`t Twyl@ dwn mll '''

In [32]: s.translate(None, _del)
Out[32]: 'The emotions were real and took their time to settle in and
 yet the story was fast paced Definitely recommend Hsnan Hn wqt n ltqT
 nfsy wyd lhdw lWan bd tlk lrHl lmtb lmlyy blGmwD wlbHth wlshkwk 
fy lbdy  lyk zyzy lqry lntbh qbl qr tlk lrwy l d shy  wlh n lrwy lyst
 rwmnsy w drm ks m ywHy smh  fh rwy tntmy l lfy lbwlysy  fy ljrym 
wlkhywT lmtshbk thny tlk lshy hw n tlk lrwy stjlk mtsmran mmh st Twyl 
dwn mll '

Answer 3

Are you looking for something like this:

re.sub(r'[^\w' + string.printable + ']', '', some_text_string)

File:

This is alpha numeric.
This is nøt alphanumeric.

Read into interpreter:

This is alpha numeric.\nThis is n\xc3\xb8t alphanumeric.

After re.sub:

This is alpha numeric.\nThis is nt alphanumeric.

EDIT: Saw your edit. How about:

re.sub(r'[^\wA-Za-z0-9 ]', '', text)

Your text is now:

But was it a ride after I loved all the characters mainly because everyone in the book has shades of gray and that is how real characters are supposed to be The emotions were real and took their time to settle in and yet the story was fast paced Definitely recommend Hsnan Hn wqt n ltqT nfsy wyd lhdw lWan bd tlk lrHl lmtb lmlyy blGmwD wlbHth wlshkwk fy lbdy lyk zyzy lqry lntbh qbl qr tlk lrwy ld shy wlh n lrwy lyst rwmnsy w drm ks m ywHy smh fh rwy tntmy l lfy lbwlysy fy ljrym wlkhywT lmtshbk thny tlk lshy hw n tlk lrwy stjlk mtsmran mmh st Twyl dwn mll wmn thm lyk n tnthy mn mshGlk wtjl nfsk syran lhdh lml dwn swh fy wl lSfHt stjd nfsk l tstwb m yHdth wstjd nfsk ttsl mn hwl wm llq bynhm wlknh lbdy fqT lyk n tmsk blkhywT lbdyy wbd dhlk trk nfsk tmman dwn ltfkyr fy shy stjd n lHdth lmttly wlmtqb d jdhb l ymknk ltGDy nh l twjd shkhSy ryysy fy tlk lrwy jmy lshkhSyt lwrd lhm dwr mHwry rytshyl twm an skwt myGyn kml abdyk wlkn ymkn lqwl n lqTr hw syd lmwqf hn fy tlk lrwy fmn khllh tbd lHdth wttTwr wlwlh lm wjdt lHbk w wsyl tSd lmwr rGm n tslsl lHdth dkhl lrwy wlntql mn lmDy l lHDr wlks hw mr mrhq ln dhlk Df lmzyd mn lthr wltshwyq dkhl tlk lrwy dwn lml l Hrq lHdth blks tjl lmr wknh shbh bfsyfs ttDH mlmh klm tkmlt mkwnth ltrjm l Gbr lyh wlm shr wknny mm ml mtrjm mn lSl slwb lmlj mmtz lHbk ldrmy mmtz lktb njHt fy stGll kf mqwmt lktb lnjH wkhrjt ln mlan mmyzan khS wnny l myl l fy lrwyt lbwlysy wlkn lmr khtlf hn mlHwZ khyr ndm nZrt l Swr lktb fy nhy tlk lrwy shrt wknh qtl

Answer 4

Using the re.sub function and the the \\W meta-character. You can open your file for reading and writing using the r+ option.

import re
import io
with open('document.txt', 'r+') as f:
    buf = io.StringIO(re.sub('\W', ' ', f.read()))
    f.seek(0)
    buf.seek(0)
    f.write(buf.getvalue())

After your file's content looks like this:

But was it a ride after I loved all the characters mainly because everyone in the book has shades of gray and that is how real characters are supposed to be The emotions were real and took their time to settle in and yet the story was fast paced Definitely recommend Hsnan Hn wqt n ltqT nfsy w yd lhdw l Wan bd tlk lrHl lmt b lmlyy blGmwD wlbHth wlshkwk fy lbdy lyk zyzy lqry lntbh qbl qr tlk lrwy ld shy wlh n lrwy lyst rwmnsy w drm ks m ywHy smh fh rwy tntmy l lfy lbwlysy fy ljry m wlkhywT lmtshbk thny tlk l shy hw n tlk lrwy stj lk mtsmran mmh st Twyl dwn mll wmn thm lyk n tnthy mn mshGlk wtj l nfsk syran lhdh l ml dwn swh fy wl lSfHt stjd nfsk l tstw bm yHdth wstjd nfsk ttsl mn hw l wm l lq bynhm wlknh lbdy fqT lyk n tmsk blkhywT lbdy y wb d dhlk trk nfsk tmman dwn