从Python字符串中去除数字和小数点以外的所有内容的最佳方法

Question

I'm reading an ASCII data stream with Python 2.7 that includes non-negative numbers with decimal places but also “garbage characters” that include nonprintables, letters, and punctuation. I can strip out the non-printables this way:

rawdata2 = filter(lambda x: x in string.printable, rawdata)

but that leaves a string like such:

Ri-G2015,2,20.23,9.13,273.1- ZW;w;K-;-A;B`R

What's a good way to strip out everything except numbers and decimal points (.) so I'm left with this:

2015,2,20.23,9.13,273.1

Answer 1

string.printable is just a string. You can use your own string in its place, like:

rawdata2 = filter(lambda x: x in ',.0123456789', rawdata)

Note that I included a comma, because your expected output also includes commas.

Answer 2

A faster approach is to use regular expressions:

import re

rawdata2 = re.sub('[^0-9,.]', '', rawdata)

This simply deletes any characters not in the set 0-9 , . (by replacing them with an empty string). This is over twice as fast as the filter approach on 100 repetitions of your input string, and is more concise.

---

The fastest approach (if you're processing a lot of text) is to use string.translate :

deltable = "".join(chr(c) for c in xrange(256) if chr(c) not in "0123456789,.")

rawdata2 = string.translate(rawdata, None, deltable)

This is over 100x faster than your original filter approach.

Answer 3

keepchars = string.digits + ",." #the characters you want to keep
rawdata2 = filter(lambda x: x in keepchars, rawdata)

I'd go with this since it seems like you want to whitelist chars. If instead you decide you want to blacklist chars, string.translate() might be a good place to look.

Answer 4

I love regular expressions. It's elegant,Since i don't know re...

In [45]: "".join([i for i in mystring if i=="." or i.isdigit() or i==','])
Out[45]: '2015,2,20.23,9.13,273.1'

Answer 5

Thanks everyone. My program doesn't need to be speedy since it's processing just one line every few minutes, but I appreciate learning about the efficiency of the different approaches. I ended up using the following two lines:

include = set('0' '1' '2' '3' '4' '5' '6' '7' '8' '9' '.' ',')

and then

cleandata1 = ''.join(ch for ch in rawdata if ch in include)

Later I inserted a third line to save the garbage characters for inspection:

garbage = ''.join(ch for ch in rawdata if ch not in include)