如何从regex.findall的匹配中返回字典列表？

Question

I'm working on several hundreds of documents and I'm writing a function that will find specific words and its values and returns a list of dictionaries.

I'm looking specifically for a piece of specific information ('city' and the number that refers to it). However, in some documents, I have one city, and in others, I might have twenty or even one hundred, so I need something very generic.

A text example (the parenthesis are messed up like this):

text = 'The territory of modern Hungary was for centuries inhabited by a succession of peoples, including Celts, Romans, Germanic tribes, Huns, West Slavs and the Avars. The foundations of the Hungarian state was established in the late ninth century AD by the Hungarian grand prince Árpád following the conquest of the Carpathian Basin. According to previous census City: Budapest (population was: 1,590,316)Debrecen (population was: 115,399)Szeged (population was: 104,867)Miskolc (population was: 109,841). However etc etc'

or

text2 = 'About medium-sized cities such as City: Eger (population was: 32,352). However etc etc'

Using regex I found the string that I'm looking for:

p = regex.compile(r'(?<=City).(.*?)(?=However)')
m = p.findall(text)

Which returns the whole text as a list.

[' Budapest (population was: 1,590,316)Debrecen (population was: 115,399)Szeged (population was: 104,867)Miskolc (population was: 109,841). ']

Now, this is where I'm stuck and I don't know how to proceed. Should I use regex.findall or regex.finditer?

Considering that the amount of 'cities' varies in the documents, I would like to get a list of dictionaries back. If I run in text 2, I would get:

d = [{'cities': 'Eger', 'population': '32,352'}] 

If I run in text one:

d = [{'cities': 'Szeged', 'population': '104,867'}, {'cities': 'Miskolc': 'population': 109,841'}]

I really appreciate any help, guys!

Answer 1

You may use re.finditer with a regex having named capturing groups (named after your keys) on the matched text with x.groupdict() to get a dictionary of results:

import re
text = 'The territory of modern Hungary was for centuries inhabited by a succession of peoples, including Celts, Romans, Germanic tribes, Huns, West Slavs and the Avars. The foundations of the Hungarian state was established in the late ninth century AD by the Hungarian grand prince Árpád following the conquest of the Carpathian Basin. According to previous census City: Budapest (population was: 1,590,316)Debrecen (population was: 115,399)Szeged (population was: 104,867)Miskolc (population was: 109,841). However etc etc'
p = re.compile(r'City:\s*(.*?)However')
p2 = re.compile(r'(?P<city>\w+)\s*\([^()\d]*(?P<population>\d[\d,]*)')
m = p.search(text)
if m:
    print([x.groupdict() for x in p2.finditer(m.group(1))])

# => [{'population': '1,590,316', 'city': 'Budapest'}, {'population': '115,399', 'city': 'Debrecen'}, {'population': '104,867', 'city': 'Szeged'}, {'population': '109,841', 'city': 'Miskolc'}]

See the Python 3 demo online .

The second p2 regex is

(?P<city>\w+)\s*\([^()\d]*(?P<population>\d[\d,]*)

See the regex demo .

Here,

• (?P<city>\\w+) - Group "city": 1+ word chars
• \\s*\\( - 0+ whitespaces and (
• [^()\\d]* - any 0+ chars other than ( and ) and digits
• (?P<population>\\d[\\d,]*) - Group "population": a digit followed with 0+ digits or/and commas

You might try to run the p2 regex on the whole original string (see demo ), but it may overmatch.

Answer 2

A very good answer by @Wiktor. Since I spend some time on this, I am posting my answer.

d = [' Budapest (population was: 1,590,316)Debrecen (population was: 115,399)Szeged (population was: 104,867)Miskolc (population was: 109,841). ']
oo = []
import re
for i in d[0].split(")"):
    jj = re.search("[0-9,]+", i)
    kk, *xx = i.split()
    if jj:
        oo.append({"cities": kk , "population": jj.group()})
print (oo)

#Result--> [{'cities': 'Budapest', 'population': '1,590,316'}, {'cities': 'Debrecen', 'population': '115,399'}, {'cities': 'Szeged', 'population': '104,867'}, {'cities': 'Miskolc', 'population': '109,841'}]