如何使用正则表达式从文本文件中提取带标签的文本？

Question

For a class, I need to extract everything that comes between the element <seg> ... </seg> and I'm trying to do this via Python instead of wasting hours doing this by hand (it's well over 400 lines). What I have right now in ways of code is this (a code that I found online and changed a little so that it doesn't print the line number):

import re                           
err_occur = [] 
pattern = re.compile(r"<seg>(.*)</seg>")
try:
    with open ('corpus.txt', 'rt') as in_file:
        for linenum, line in enumerate(in_file):
            if pattern.search(line) != None:
                err_occur.append((linenum, line.rstrip('\n')))
        for linenum, line in err_occur:
            print(line, sep='')
except FileNotFoundError:
    print("Input file not found.")

The only problem I have with this is that it prints the <seg> and </seg> in the results, which I don't want to happen. I've tried to create groups (which you can see in my usage of parentheses in the pattern variable) but I have no idea how to manipulate the code to return just group 1 (I've tried many different ways).

Answer 1

You need to use positive lookbehind and a positive lookahead . The <seg> and <\\seg> in your regex consumes some text so you see them in your results, but the lookahead and lookbehind just checks if </seg> and <seg> are there respectively without consuming any characters. It only matches those strings.

Tl;dr: lookahead and lookbehind matches string in <seg>string</seg> and not the tags.

So your regex should be like (?<=<seg>).*(?=</seg>) , this should be fine.

There is some documantation here

Answer 2

Here's something that will print all the tagged text in each line without the tags:

The important modification was changing your regex from (r"<seg>(.*)</seg>" to r"<seg>(.*?)</seg>" —note the added ? after the * . This is called making it "non-greedy" so it doesn't match as much of the remaining text as possible (the default "greedy" mode). This is discussed in greater detail in the Regular Expression HOWTO section of Python's online documentation.

Another significant change, regex-wise, was to use pattern.findall() instead of pattern.search() .

I also removed all the parts of the code dealing with line numbers since you mentioned you weren't interest in that information.

import re

err_occur = []
pattern = re.compile(r"<seg>(.*?)</seg>")
input_filename = 'corpus.txt'

try:
    with open(input_filename, 'rt') as in_file:
        for line in in_file:
            matches = pattern.findall(line)
            if matches:
                for match in matches:
                    err_occur.append(match)
except FileNotFoundError:
    print("Input file %r not found." % input_filename)

for tagged in err_occur:
    print(tagged)

Answer 3

You can use BeautifulSoup for this.

soup = BeautifulSoup(your input)
print soup.findAll("seg")[0].renderContents()

Also the regex can be:

import re
print re.findall("<seg>(.*?)</seg>", your input)

http://tpcg.io/B6h2So