python中的正则表达式，从文件中读取文本

Question

I have regular expression that scans some data from html files the code is removing html tags using BeautifulSoup and return the following text (just a part from the text):

/Semester: 2011 / 1 Number : 20112222

Name : XXXX XXXX XXXX XXXX Advisor

This sample of my code:

import re,glob,os
from bs4 import BeautifulSoup
import nltk

path = 'C:\\xampp\\htdocs\\data_tools\\transcripts'
os.chdir(path)
delch=','

def scantext(text,snum) :
    re_semstudent = re.compile("Semester:\s*(\d*)\s*\/\s*(\d)\s*Number\s*:\s*(\d{8})\s*Name\s*:\s*(.*)\s*Advisor")
    semesters = text.split("Year")

    for ind in range(1,len(semesters)):
        s = semesters[ind]
        x = re.search(re_semstudent,s)
        if x :
            year=x.group(1)
            semester=x.group(2)
            studentid=x.group(3)
            studentname=x.group(4)

        print year+"#"+semester

    return 0

ii=1
for fname in glob.glob("*.html") :
    f = open (fname)        
    text = BeautifulSoup(f.read(), 'html.parser').getText()
    scantext(text,ii)

When I am trying the re.search with the text as fixed string, its work fine! But when I send the text in the scantext function and use the semesters = text.split("Year") . I can print the text of each split, but the regular expression can't match any value!

Answer 1

You need the re.U/re.Unicode flag:

  re_semstudent = re.compile("Semester:\s*(\d*)\s*\/\s*(\d)\s*Number\s*:\s*(\d{8})\s*Name\s*:\s*(.*)\s*Advisor",re.U)

Which if you run after will give you something like:

<_sre.SRE_Match object at 0x7fe9fb721df8>
2011#1
<_sre.SRE_Match object at 0x7fe9fb721d50>
2011#2
<_sre.SRE_Match object at 0x7fe9fb721df8>
2012#1
<_sre.SRE_Match object at 0x7fe9fb721d50>
2012#2

You might also need to open the file with encoding="utf-8" :

from io import open
for fname in glob.glob("*.html") :
    with open(fname, encoding="utf-8") as f:
        text = BeautifulSoup(f.read(), 'html.parser').getText()
        scantext(text, ii)