将appengine java上的日志写入stackdriver-logging

Question

I have a java app running on appengine.

I'm logging my logs in json structure and then I can see my logs on stack driver (as in the docs )

package com.foo.bar;

public class MyClass {

    private static final Logger log = Logger.getLogger(MyClass.class.getName());

    public void myFunc() {
        log.info("{msg: 'hello', corId: '123'}");
    }

here is the message I get on stackdriver-logging:

com.foo.bar.MyClass myFunc: {msg: 'hello', corId: '123'}

and in the log-request object:

protoPayload.line[].logMessage = "com.foo.bar.MyClass myFunc: {msg: 'hello', corId: '123'}"

how can I make the log message to be only the message I am logging - without the class prefix: {msg: 'hello', corId: '123'}

protoPayload.line[].logMessage = "{msg: 'hello', corId: '123'}"

Answer 1

I ended up shipping the logs from stackdriver to alasticsearch via logstash

in logstash I parsed my logs, I also seperated my logs tol have each log as its own record and not a nested array see: How to ship logs from pods on Kubernetes running on top of GCP to elasticsearch/logstash?

my config in logstash for parsing the logs:

filter {
  if [resource][type] == "gae_app" {
    # split the protoPayload.line array, so each log message is a separate entry in Elasticsearch
    split {
      field => "[protoPayload][line]"
      target => "line"
      remove_field => [  "httpRequest", "operation", "protoPayload"]
    }
    # extract `line.logMessage` and `line.severity` fields
    mutate {
      add_field => {"logMessage" => "%{[line][logMessage]}"}
      replace => {"severity" => "%{[line][severity]}"}
      remove_field => ["line"]
    }
    # remove the `com.example.MyClass myFunc: ` prefix from log
    grok {
      match => { "logMessage" => "^%{DATA}: %{GREEDYDATA:parsedMessage}"}
    }
    # parse the log message into json, json fields will be located in root
    json {
      source => "parsedMessage"
      target => "jsonPayload"
      add_field => {"[jsonPayload][level]" => "%{severity}"}
      remove_field => ["parsedMessage", "logMessage"]
    }

    # uniform GAE logs to the structure of GKE logs
    grok {
      match => {  # check..
        "[resource][labels][version_id]" =>
        "^%{DATA:[resource][labels][container_name]}-%{GREEDYDATA:[resource][labels][namespace_id]}"}
    }


  }
}