Reading and printing var log messages related to USB in Python

Question

I want to print the lines related to USB of the log messages created in Ubuntu. Here's my code:

>>> import re
>>> fd = open("/var/log/syslog.1", "r")
>>> for lines in fd:
...     if re.match("usb", lines):
...             print lines

(By the way, I'm not sure if the file syslog.1 is the right one. However, I do find a lot of messages in it and some related to usb)

Answer 1

@SidNoob: Adding to the topic of logs (may not be python related), I am assuming you are looking for logs from the usb daemon or driver. If thats the case you are probably looking at the right log file "/var/log/syslog" which logs kernel related messages (drivers inclusive). If it is an application (with a different log level) then you might have to look at "/var/logs/messages".

But coming to the POINT: sometimes opening a file like "/var/log/syslog" for reading might be highly memory intensive because you are loading the file into the RAM. In server type environments this could run into MBs long or even longer. There is a command "dmesg" which just prints out to the stdout whatever the contents of ""var/log/syslog" file is. So instead of opening this large file you could also store "dmesg"'s output into a string and parse it line by line.

You might think both ways would be the same and may occupy the same memory in running the python program. Yes, but not if you clear out the "dmesg" output. "dmesg" is a ring buffer so you can clear it out using "dmesg -c" before running the usb application and issue "dmesg" after you start the application. By this way you reduce the amount of lines you parse and hence speeding up your program.

Hope this adds to your knowledge and your program.

Answer 2

For the most recent messages, you want /var/log/syslog - /var/log/syslog.1 is basically a backup of older messages.

The thing stopping your code from working is that you're using re.match() instead of re.search() - as you can see from the documentation at those links, re.match() only matches at the beginning of the text being searched.

In fact, though, you don't need to use regular expressions at all. Here's an alternative to your code:

>>> with open("/var/log/syslog") as f:
...     for line in f:
...         if "usb" in line.lower():
...             print line
... 

A few things are different here:

1. We're using thewith context manager to open the file. This helps Python clean up and close the file even if something goes wrong, and it's a good habit to get into.

2. As is conventional in Python, we've called the file variable f . fd will make someone reading your code think you're talking about a file descriptor , which is not the same thing.

3. Since it holds a single line, we'll call the string variable line , not lines .

4. Rather than a regex, we can just check whether "usb" is in the line somewhere (and we're converting the line to lower case before checking so that we'll catch "USB" in the original too.