Python 清理域名 - 正则表达式还是 lambda？

Question

I am not sure if this is the best way to approach this problem in python. In bash I would probably just use awk, sed and be done with it.

There are two suggestions based from this post but I am having trouble to implement. I would like to clean up the domain names.

code

import re

log = ["4/19/2020 11:59:09 PM 2604 PACKET  0000014DE1921330 UDP Rcv 192.168.1.28   f975   Q [0001   D   NOERROR] A      (7)pagead2(17)googlesyndication(3)com(0)",
       "4/19/2020 11:59:09 PM 0574 PACKET  0000014DE18C4720 UDP R cv 192.168.2.54    9c63   Q [0001   D   NOERROR] A      (2)pg(3)cdn(5)viber(3)com(0)"]

rx_dict = {
    'date': re.compile(r'(?P<date>(\d+)[\/](\d+)[\/](\d+))'),
    'time': re.compile(r'(?P<time>\d{2}:\d{2}:\d{2}.(?:AM|PM))'),
    'client': re.compile(r'(?P<client>(?:[0-9]{1,3}\.){3}[0-9]{1,3})'),
    'flags': re.compile(r'(?P<flags>(?<=\].)(.\S{0,}))'),
    'query': re.compile(r'(?P<query>[\S]*)$')
    }

for item in log:
    counter = 0 
    for key, r_exp in rx_dict.items():
        print(f"{r_exp.search(item).group(1)}", end='')
        if counter < 4:
            print(',', end='')
            counter = counter + 1
    print()

output

4/19/2020,11:59:09 PM,192.168.1.28,A,(7)pagead2(17)googlesyndication(3)com(0)
4/19/2020,11:59:09 PM,192.168.2.54,A,(2)pg(3)cdn(5)viber(3)com(0)

preferred output

4/19/2020,11:59:09 PM,192.168.1.28,A,pagead2.googlesyndication.com
4/19/2020,11:59:09 PM,192.168.2.54,A,pg.cdn.viber.com

Answer 1

I assume that you want to clean up the query results.You can do this by using re.sub .

>>> help(re.sub)
Help on function sub in module re:

sub(pattern, repl, string, count=0, flags=0)
   Return the string obtained by replacing the leftmost
   non-overlapping occurrences of the pattern in string by the
   replacement repl.  repl can be either a string or a callable;
   if a string, backslash escapes in it are processed.  If it is
   a callable, it's passed the Match object and must return
   a replacement string to be used.

The first argument is the pattern(here it is (AnyNumber) ). The second argument is repl (Here it is clean_up_query function). This function will be called for every non-overlapping occurrence of pattern.

>>> import re
>>>
>>> log = [
...     "4/19/2020 11:59:09 PM 2604 PACKET  0000014DE1921330 UDP Rcv 192.168.1.28   f975   Q [0001   D   NOERROR] A      (7)pagead2(17)googlesyndication(3)com(0)",
...     "4/19/2020 11:59:09 PM 0574 PACKET  0000014DE18C4720 UDP R cv 192.168.2.54    9c63   Q [0001   D   NOERROR] A      (2)pg(3)cdn(5)viber(3)com(0)",
... ]
>>>
>>> rx_dict = {
...     "date": re.compile(r"(?P<date>(\d+)[\/](\d+)[\/](\d+))"),
...     "time": re.compile(r"(?P<time>\d{2}:\d{2}:\d{2}.(?:AM|PM))"),
...     "client": re.compile(r"(?P<client>(?:[0-9]{1,3}\.){3}[0-9]{1,3})"),
...     "flags": re.compile(r"(?P<flags>(?<=\].)(.\S{0,}))"),
...     "query": re.compile(r"(?P<query>[\S]*)$"),
... }
>>>
>>> def clean_up_query(match):
...     match_start, match_stop = match.span()
...     if (match_start == 0) or (
...         match_stop == len(match.string)
...     ):  # we do not want "." to be appeared on the result if the match is at the beginning or at the end.
...         return ""
...     return "."
...
>>> for item in log:
...     counter = 0
...     for key, r_exp in rx_dict.items():
...         if key == "query":
...             print(
...                 re.sub(r"\(\d+\)", clean_up_query, r_exp.search(item).group(1)), end=""
...             )
...         else:
...             print(f"{r_exp.search(item).group(1)}", end="")
...         if counter < 4:
...             print(",", end="")
...             counter = counter + 1
...     print()
...
4/19/2020,11:59:09 PM,192.168.1.28,A,pagead2.googlesyndication.com
4/19/2020,11:59:09 PM,192.168.2.54,A,pg.cdn.viber.com