Log4J-JsonLayout和RollingFileAppender生成无效的JSON

Question

I'm trying to store my logs in JSON files using Log4J (2.6.2). I'm using a JsonLayout in a RollingFileAppender , and it's working properly as long as I'm not trying to append to a file that has already been written previously.

Here's my code for setting up the Layout and the Appender:

Layout<?> layout = JsonLayout.createLayout(config, false, false, false, true, false, true, "[", "]", Charset.defaultCharset());

String appenderFileName = "mylogfile-latest.log.json";
String appenderFilePattern = "mylogfile-%i.log.json";
String appenderName = "MyAppender";
Appender appender = RollingFileAppender.createAppender(appenderFileName, appenderFilePattern, "true", appenderName, "true", "256", "true",
        SizeBasedTriggeringPolicy.createPolicy(this.configuration.getLogMaxSize().toString()),
        null, layout, null, "false", "false", null, config);

As I said, it's working fine the first time I'm writing in my log file:

[
  {
    "timeMillis" : 1469620840442,
    "thread" : "SimpleAsyncTaskExecutor-43",
    "level" : "ERROR",
    "loggerName" : "MyLogger",
    "message" : "my log message",
    "endOfBatch" : false,
    "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
    "threadId" : 243,
    "threadPriority" : 5
  }
  , {
    "timeMillis" : 1469620840442,
    "thread" : "SimpleAsyncTaskExecutor-43",
    "level" : "DEBUG",
    "loggerName" : "MyLogger",
    "message" : "my log message",
    "endOfBatch" : false,
    "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
    "threadId" : 243,
    "threadPriority" : 5
  }
]

I then close my appender, etc... and relaunch my application, and when I'm writing new logs to this existing file, here's what I'm getting:

[
{
  "timeMillis" : 1469620840442,
  "thread" : "SimpleAsyncTaskExecutor-43",
  "level" : "ERROR",
  "loggerName" : "MyLogger",
  "message" : "my log message",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId" : 243,
  "threadPriority" : 5
}
, {
  "timeMillis" : 1469620840442,
  "thread" : "SimpleAsyncTaskExecutor-43",
  "level" : "DEBUG",
  "loggerName" : "MyLogger",
  "message" : "my log message",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId" : 243,
  "threadPriority" : 5
}

]
{
    "timeMillis" : 1469620840490,
    "thread" : "SimpleAsyncTaskExecutor-43",
    "level" : "ERROR",
    "loggerName" : "MyLogger",
    "message" : "my log message",
    "endOfBatch" : false,
    "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
    "threadId" : 245,
    "threadPriority" : 5
  }
  , {
    "timeMillis" : 1469620840492,
    "thread" : "SimpleAsyncTaskExecutor-43",
    "level" : "DEBUG",
    "loggerName" : "MyLogger",
    "message" : "my log message",
    "endOfBatch" : false,
    "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
    "threadId" : 245,
    "threadPriority" : 5
  }

]

This issue makes it impossible to parse the logs as Gson will fail to create a List from this file.

Do you have any idea on how to fix this issue without doing a crappy hack removing the extra ] and replacing it with a comma?

Thanks!

Answer 1

The reason why the JSONLayout does this is to create a "well-formed" JSON document. You need to put each document in its own file (with start and end brackets) for this to be true, which isn't the case here.

You have two options:

1. For JSONLayout, set complete=false (you have it as true in your sample) which will print it out without the '[' and ']'. You can then add these chars yourself before calling GSON (as the file will not really have an array).
2. Create your own JSONLayout (you need to extend AbstractStringLayout) using something simple like org.json:json or GSON to get you the string for the file.

Answer 2

Since no other solution was satisfying, I ended up using a regex to replace the invalid characters...

It's pretty safe since it checks if it's not in between two double-quotes, but it's not the ideal solution obviously:

String content = new String(Files.readAllBytes(Paths.get(pathToLogFile)));
content = content.replaceAll("(\\}\\R{2}\\]\\R\\{)(?=(?:(?:\\.|[^\"\\\\])*\"(?:\\.|[^\"\\\\])*\")*(?:\\.|[^\"\\\\])*)", "}\n, {");
Files.write(Paths.get(pathToLogFile), content.getBytes());