解析日志文件並使用 python 中的表創建 CVS 報告。 由於某些原因，代碼不斷返回錯誤

Question

我被困了一個多星期了，請幫忙
我不明白為什么它不起作用

那作業：
想象一下，您的公司使用一台運行名為 ticky 的服務的服務器，這是一個內部票務系統。 該服務在成功運行和遇到錯誤時將事件記錄到 syslog。 該服務的開發人員需要您幫助從這些日志中獲取一些信息，以便他們能夠更好地了解他們的軟件是如何使用的以及如何改進它。 因此，對於本實驗，您將編寫一些自動化腳本來處理系統日志並根據從日志文件中提取的信息生成報告。

你會做什么 使用正則表達式解析日志文件 Append 並修改字典中的值 寫入 CSV 格式的文件 將文件移動到適當的目錄以用於 CSV->HTML 轉換器

以下是日志文件中的示例：

Jan 31 16:35:46 ubuntu.local ticky: ERROR: Timeout while retrieving information (oren)
Jan 31 16:53:54 ubuntu.local ticky: INFO: Commented on ticket [#3813] (mcintosh)`
Jan 31 16:54:18 ubuntu.local ticky: ERROR: Connection to DB failed (bpacheco)
Jan 31 17:15:47 ubuntu.local ticky: ERROR: The ticket was modified while updating (mcintosh)
Jan 31 17:29:11 ubuntu.local ticky: ERROR: Connection to DB failed (oren)
Jan 31 17:51:52 ubuntu.local ticky: INFO: Closed ticket [#8604] (mcintosh)

我的代碼（不斷返回錯誤）：

#!/usr/bin/env python3

import re
import sys
import operator
import csv

error = {}
per_user = {}

with open('syslog.log') as file:
  lines = file.readlines()
  for line in lines:
    result = re.search(r"ticky: ([A-Z]*):? ([\w' ]*)[\[[#0-9]*\]?]? \(([a-z\.?]*)\)$", line)
    category, message, username = result.group(1), result.group(2), result.group(3)

    if category == "ERROR" and message not in error.keys():
      error[message] = 1
    elif category == "ERROR" and message in error.keys():
      error[message] += 1

    if category == "INFO":
      if username not in per_user.keys():
        per_user[username] = {}
        per_user[username]["INFO"] = 1
        per_user[username]["ERROR"] = 0
      else:
        per_user[username]["INFO"] += 1
    elif category == "ERROR":
      if username not in per_user.keys():
        per_user[username] = {}
        per_user[username]["INFO"] = 0
        per_user[username]["ERROR"] = 1
      else:
        per_user[username]["ERROR"] += 1

# sort the error dictionary by the number of errors from most common to least common
sorted_errors = sorted(error.items(), key=operator.itemgetter(1), reverse=True)

# sort the user dictionary by username
sorted_users = sorted(per_user.items(), key=operator.itemgetter(0))

updated_errors = dict(sorted_errors)
updated_users = dict(sorted_users)

file.close()

# create error_message.csv 
# insert column names as ("Error", "Count") at the zero index position of the sorted error dictionary
keys = ["Error", "Count"]
with open("error_message.csv", "w", newline='') as error_csv:
  ew = csv.DictWriter(error_csv, fieldnames=keys)
  ew.writeheader()
  ew.writerows(updated_errors)
 
# create user_statistics.csv
# insert column names as ("Username", "INFO", "ERROR")
keys = ["Username", "INFO", "ERROR"]
with open("user_statistics.csv", "w", newline='') as user_csv:
  uw = csv.DictWriter(user_csv, fieldnames=keys)
  uw.writeheader()
  uw.writerows(updated_users)

我真的不明白錯誤信息，所以....我真的迷路了。

錯誤：

Traceback (most recent call last):
  File "ticky_check.py", line 54, in <module>
    ew.writerows(updated_errors)
  File "/usr/lib/python3.8/csv.py", line 157, in writerows
    return self.writer.writerows(map(self._dict_to_list, rowdicts))
  File "/usr/lib/python3.8/csv.py", line 147, in _dict_to_list
    wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys'

Answer 1

我不確定是否理解預期的 output，但也許您可以使用pandas.read_fwf ？

import pandas as pd
from io import StringIO

t = """Jan 31 16:35:46 ubuntu.local ticky: ERROR Timeout while retrieving information (oren)
Jan 31 16:53:54 ubuntu.local ticky: INFO Commented on ticket [#3813] (mcintosh)`
Jan 31 16:54:18 ubuntu.local ticky: ERROR Connection to DB failed (bpacheco)
Jan 31 17:15:47 ubuntu.local ticky: ERROR The ticket was modified while updating (mcintosh)
Jan 31 17:29:11 ubuntu.local ticky: ERROR Connection to DB failed (oren)
Jan 31 17:51:52 ubuntu.local ticky: INFO Closed ticket [#8604] (mcintosh)"""

names=["SERVER", "INFO", "ERROR"]
df = pd.read_fwf(StringIO(t), header=None, names=names)
# or df = pd.read_fwf(r'path_to_your_logfile.txt', header=None, names=names)
df['USERNAME'] = df['ERROR'].str.extract('.*\((.*)\).*')

>>> display(df)