Monitoring and Parsing a Log file using Java WatchService

Question

I am trying to monitor a folder to get updates on a specific log file, using the Java WatchService in Java 8.

This log file is created using java.util.logging.Logger and has a regular FileHander, which rotates the log file after it reaches a specific size. It creates tracelog0.log initially and after log rotation, it creates tracelog1.log. All new entries still go to tracelog0.log.

I am getting the StandardWatchEventKinds.ENTRY_MODIFY events. But the issue is when the Log file is rotated I do not get any event and Log monitoring Thread seems to be killed. I have added a new StandardWatchEventKinds.ENTRY_DELETE event as well but still does not work. Here is my Thread that does the monitoring. Please let me know if this is the correct approach or not. Please suggest any alternative solutions for this case.

class WatcherThread implements Runnable {
    public static final String WORKING_DIRECTORY = "/var/log/data/";
    Path logDir = Paths.get(WORKING_DIRECTORY);
    WatchEvent.Kind<?> [] kinds = { StandardWatchEventKinds.ENTRY_CREATE, 
            StandardWatchEventKinds.ENTRY_DELETE,
            StandardWatchEventKinds.ENTRY_MODIFY};
    boolean runThread = true;
    private WatchService watcher;
    private boolean logRotated = false;

    public WatcherThread(){
        try {
            watcher = logDir.getFileSystem().newWatchService();
            logDir.register(watcher, kinds);
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

    @Override
    public void run() {

        while(runThread) {
            // Get next event or wait if none are yet present
            try {
                WatchKey key = watcher.take();
                for (WatchEvent<?> event : key.pollEvents()) {

                    // Retrieve event kind
                    WatchEvent.Kind<?> kind = event.kind();

                    if (StandardWatchEventKinds.OVERFLOW == kind) {
                        continue;
                    }else {
                        WatchEvent<Path> ev = (WatchEvent<Path>)event;
                        Path filename = ev.context();
                        Path child = logDir.resolve(filename);

                        System.out.println("New file created Filename: "+child.getFileName());
                        if(child.getFileName().startsWith("tracelog")) {
                            System.out.println("tracelog file rotated.");
                            //Possible tracelog0.log file is rotated. 
                            logRotated = true;
                        }
                    }
                }
                boolean isValid = key.reset();
                if(!isValid) {
                    System.out.println("Unable to reset the Key exiting!!!");
                    runThread = false;
                }
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }

    }

    public boolean isLogRotated() {
        return logRotated;
    }
}

Answer 1

I think I have figured the issue. The issue is due to the java.util.Logger framework. When the Logfile rotation happens it does not delete the actual Logging file.

The current Log is stored in a cache by the Logger. When it needs to rotate the log file based on the log size. It creates a new file and copies the content to the new file.

Example 
When the server starts the below files get created 
/var/log/data/tracelog0.log 
/var/log/data/tracelog0.log.lck
when log file rotates it creates a new file tracelog1.log
/var/log/data/tracelog0.log 
/var/log/data/tracelog0.log.lck
/var/log/data/tracelog1.log

So only MODIFY and CREATE event are triggered. The DELETE event does not get triggered.

I think this may vary based on the Logger that we use. It's better to look at how the logger rotates the log file.