在python中具有匹配关键字的文件中搜索和返回行

Question

I have used regex to isolate a particular keyword in a line taken from a file. I want to search the entire file and return groups of lines which have the same keyword.

I am a little confused about this, and I was wondering if there is a direct regex way to do this in Python ?

eg - >

My file may look like this

1  0001    1   UG  science,ee;YEAR=onefour;standard->2;district->9
2  0002    1   UG  science,cs;YEAR=onefive;standard->1;district->9
3  0012    2   UG  science,eng;YEAR=onefour;standard->3;district->4
4  0021    2   UG  science,ee;YEAR=onetwo;standard->2;district->9
5  0056    4   UG  science,cs;YEAR=onefive;standard->1;district->8
6  0145    3   UG  science,eng;YEAR=onetwo;standard->4;district->2

I used regex to extract

"YEAR=****" 

and want to group the lines according to the value of

"****"

The output should look like this -

1  0001    1   UG  science,ee;YEAR=onefour;standard->2;district->9
3  0012    2   UG  science,eng;YEAR=onefour;standard->3;district->4

2  0002    1   UG  science,cs;YEAR=onefive;standard->1;district->9
5  0056    4   UG  science,cs;YEAR=onefive;standard->1;district->8

4  0021    2   UG  science,ee;YEAR=onetwo;standard->2;district->9
6  0145    3   UG  science,eng;YEAR=onetwo;standard->4;district->2

I believe I can do it the long way of opening file, storing in dictionaries and matching, etc, etc. But would like to know if there is short concise way of doing this.

as requested - a bit of code I tried to write and run -

#!/usr/bin/python

import re

##open file and read each line of file

dfile = open("datafile.txt","r")

##regex to find YEAR in entry and return YEAR

regex_unique = re.compile(r'(?<=\bYEAR=)[^;]+')

list_Name =[]

for line in dfile:
    match1 = re.search(regex_unique,line)
    if match1:
        if match1.group(0) not in list_Name:
        list_Name.append(match1.group(0))


## print (list_Name)

for item in list_Name:
for line in dfile:
    match2 = re.search(item,line)
    if match2:
        print (match2)

the last bit does not seem to work - I am assuming if I give

item

to

re.search

it should search for that word in the entire file - Now I think I might have to add some wildcard entries before and after the actual word to make it work.

Answer 1

I think I'm right in saying that regex only deals with matches on lines, not with how to aggregate matches -- so you'll need to do that yourself. You can keep things simple by writing your own utility function and keeping it separate from you application code.

Grouping operations in general must pass over all of the items to assemble the groups. Your problem can't be solved without making a pass over all of the data to collect the groups, and then another pass to output the groups.

A dictionary of lists is the natural datastructure to collect a each line by by a key (as you note). Doing this yourself set is a little kludgey, since you frequently need to test whether a key exists to know whether you should add to an existing list or create a new one. Fortunately, python provides defaultdict, that lets you:

from collections import defaultdict
>>> d = defaultdict(list)
>>> d[key].append(line)

So, you can do the following:

def groupLinesByMatch(filename,regex):
    import re
    from collections import defaultdict

    regex = re.compile(regex)
    result = defaultdict(list)

    for line in open(filename).readlines():
        matches = regex.match(line)
        if matches:    
            result[matches.group(1)].append( line )

    return result.values()


for lines in groupLinesByMatch(filename, regex):
    for line in lines:
        print line,
    print