在Python中读取日志文件

Question

I have a log file with following content. I like to read Iteration value and detection_eval value

I0704 18:10:31.097334  2421 solver.cpp:433] Iteration 200, Testing net (#0)
I0704 18:10:31.149454  2421 net.cpp:693] Ignoring source layer mbox_loss
I0704 18:10:40.241916  2421 solver.cpp:546]     Test net output #0: detection_eval = 0.00273318

I did

accuracy_pattern = r"Iteration (?P<iter_num>\d+), Testing net \(#0\)\n.* detection_eval = (?P<accuracy>[+-]?(\d+(\.\d*)?|\.\d+)([eE][+-]?\d+)?)"

But nothing is read, what is the issue?

EDIT: Then I read accuracy_pattern to array as

for r in re.findall(accuracy_pattern, log):
        iteration = int(r[0])
        accuracy = float(r[1]) * 100

log has all file content and read as follow

with open(log_file, 'r') as log_file2:
        log = log_file2.read()

Answer 1

As far as I understand your data, the following regex should work:

pattern = "Iteration\s+(\d+)|detection_eval\s+=\s+(.+$)"
for it,de in re.findall(pattern, log, flags=re.M):
    if it: 
        print('Iteration', int(it))
    if de:
        print('detection_eval', float(de))
#Iteration 200
#detection_eval 0.00273318

However, reading the whole log file at once is usually a bad idea. Consider reading one line at a time:

with open(log_file, 'r') as log_file2:
    for line in log_file2:
        for it,de in re.findall(pattern, log):
            if it: 
                print('Iteration', int(it))
            if de:
                print('detection_eval', float(de))

Answer 2

Using re.search

Demo:

import re

with open(log_file, "r") as infile:
    for line in infile:
        iteration = re.search("Iteration (\d+)", line)
        if iteration:
            print iteration.group()

        detection_eval = re.search("detection_eval = (\d.*)", line)
        if detection_eval:
            print detection_eval.group()

Output:

Iteration 200
detection_eval = 0.00273318

Or using re.findall

iteration = re.findall(r"Iteration (\d+)", log )
detection_eval = re.findall(r"detection_eval = (\d.*)", log )