如何仅解析带有正则表达式的HTML文件中文本中的外来字符

Question

I'm trying to parse HTML and automatically change the font of any foreign characters, and I'm having some issues. There are a few different hackish ways I'm trying to accomplish this, but none work really well, and I'm wondering if anyone has any ideas. Is there any easy way with python to match all the foreign characters (specifically, Japanese Kanji/Hirigana/Katakana) with regular expressions? What I've been using is the complement of a set of non-foreign characters ([^A-Za-z0-9 <>'"=]), but this isn't working well, and I'm worried it will match things enclosed in <...>, which I don't want to do.

Answer 1

I wouldn't use just regular expressions for this. Down that path lies an angry Tony the Pony .

I'd use an HTML parser in conjuction with regular expressions, though. That way you can distinguish the markup from the non-markup.

Answer 2

Use BeautifulSoup to get the content that you need, then use a variation on this code to match your characters.

import re

kataLetters = range(0x30A0, 0x30FF)
hiraLetters = range(0x3040, 0x309F)
kataPunctuation = range(0x31F0,0x31FF)

myLetters = kataLetters+kataPunctuation+hiraLetters

myLetters = u''.join([unichr(aLetter) for aLetter in myLetters])


myRe = re.compile('['+myLetters+']+', re.UNICODE)

Use the code charts here to get the ranges for your characters.