Java Log Coverage tool

Question

Are there any tools or strategies for generating a "Log coverage" report on (Java, log4j)? Like code coverage, but ensuring that there aren't large methods, classes, or packages which don't log anything.

When coding web services, me team doesn't write many log statements. When debugging a real time problem with running, production code, we always wish we had. Inevitably we try to reproduce the bug in our test environment with either a debugger attached or additional log statements added, which can be very difficult depending on the structures and inter-operation involved.

Does anyone use this as a code-quality metric?

Answer 1

Code coverage takes special instrumentation because you're trying to find out whether a piece of production code is exercised by any test. What you're asking is a little more vague and could be either much easier ("is any logging done for this large class?") or much harder to the point of impossible ("did we log the method that's going to break in production?").

For the first question, you could whip up a shell script pretty quickly to do the job. Here's a skeleton in Perl, for example. Here, I assume that we're using SLF4J and that seeing the import of "LoggerFactory" is enough evidence to assume there's a logger.

while ($filename = shift) {
    open my $in, "<$filename";
    my $loc = 0;
    my $log = "NO LOGGER";
    while (<$in>) {
        $loc++;
        if (m/import org.slf4j.LoggerFactory/) {
            $log = "has logger";
        }
    }
    print "$filename : $loc LOC $log\n";
    $total{$log} += $loc;
}
print "\n\nTOTAL LOGGED: $total{'has logger'}\nTOTAL UNLOGGED: $total{'NO LOGGER'}\n";

and I can run this from my shell to run over all the Java files in a little project with

$ find . -name \*.java -exec perl haslog.pm {} \+

This only works for small-sized projects, and it's fairly brittle but it wouldn't be a ton of work to make a more robust version of this.

Answer 2

Lots of logs can be noise and in my experience I always found tracing through logs painful. Having said that if the logs are managed well you can get good diagnostics/reporting. One of the reason for the code not being tested properly is because of having lots of logs in production code. What developers tend to do is add a log statement when they are developing to check the code works, consequently it encourages not writing a test with the right assertion. What you need is lots of little classes that are well tested composed together. The assertion should exactly tell you why the test is failing.

Lets say in your code path you are expecting something to happen which is its main responsibility (eg Created a DB entry to register user/or someone logging in), when I say its main responsibility I am not talking about a side effect that happens in your code path. If you have an error condition in the main code path the exception should be thrown all the way up the stack were you can log and convert that to a user friendly message. RuntimeExceptions are a good here because you dont want to be catching these exceptions until its all the way up to the view layer. Side effects can be logged as well because they are like info/warnings.