计算 Python 中正则表达式匹配的出现次数

Question

I have a log checker that is using regex to find the IP from lines in a logfile. I want to find the lines, and count the total occurrences of those same lines, matched via IP

The goal is to generate statistics from the events, based on their IP. example:

WARNING - 192.168.1.1 TIMING OUT    
WARNING - 192.168.1.5 TIMING OUT    
WARNING - 192.168.1.1 TIMING OUT    
WARNING - 192.168.1.5 TIMING OUT    
WARNING - 192.168.1.1 TIMING OUT    
WARNING - 10.1.1.1 TIMING OUT    
WARNING - 10.72.3.1 TIMING OUT    

192.168.1.1 - 3 EVENTS    
192.168.1.5 - 2 EVENTS    
10.1.1.1 - 1 EVENT    
10.72.3.1 - 1 EVENT

So on and so forth. I'm a python novice so I'm still learning what is best suitable for this purpose. As of this moment, I have the log file open, perform a for loop using the regex pattern to find the IP's in each line but from there I'm a bit lost. Cheers.

Answer 1

You could use re.findall here to capture all IP address events, then use a map to tally the number of occurrences:

inp = """WARNING - 192.168.1.1 TIMING OUT    
WARNING - 192.168.1.5 TIMING OUT    
WARNING - 192.168.1.1 TIMING OUT    
WARNING - 192.168.1.5 TIMING OUT    
WARNING - 192.168.1.1 TIMING OUT    
WARNING - 10.1.1.1 TIMING OUT    
WARNING - 10.72.3.1 TIMING OUT"""

matches = re.findall(r'\bWARNING - (\b\d+\.\d+\.\d+\.\d+\b)', inp)
d = {}

for elem in matches:
    try:
        val = d.get(elem) or 0
        d[elem] = val + 1
    except KeyError:
        d[elem] = d[elem]

print(d)

This prints:

{'10.1.1.1': 1, '192.168.1.5': 2, '10.72.3.1': 1, '192.168.1.1': 3}

Answer 2

Below is a modified variant of my answer at https://stackoverflow.com/a/64220148/6632736 .

It is assumed that the log is in a file, which is read line by line.

#!/usr/bin/python
import os
import re

def increment(ips: dict, line: str):
    match = re.match(r'^.+?\s+-\s+(?P<ip>\d{1,3}(\.\d{1,3}){3})\s.*$', line)
    if match:
        ip = match.group('ip')
        if not ip in ips:
            ips[ip] = 0
        ips[ip] += 1

def parse_log_file(log: str) -> dict:
    ips = dict()
    with open(log, 'r') as file:
        for line in file:
            increment(ips, line)
    return ips

# log is the path to the log file:
for key, value in parse_log_file(log).items():
    print(key, ":", value)