使用“系统和工作负载日志记录和监控”(GKE) 的接收器中的对象名称问题

Question

Currently, I am using a GKE cluster 1.14.10-gke.50 version. This cluster is using “Legacy logging and monitoring”. In addition, I have two sinks: A Bucket and a BigQuery Dataset. My concern is that according to the Google documentation, this logging implementation will be decommissioned in March, 2021. I did a test upgrading the logging implementation to “System and workload logging and monitoring”, however I have noticed that the folder “structure” in the Bucket is being mess up, instead of using the container names as “folder” (Please keep in mind that the Buckets do not use real folders), all the log entries are forward into the “stdout” folder. Regarding the BigQuery dataset, the tables previously were named with the container name, but with the new implementation stdout table is created. I would want keeping the old structure, this means using the container names to name the objects to be created, these are my reasons: It is clearer because you can filter easily and I am using some scripts to check the archived log entries and I want to avoid a refactor. According the Google documentation this is the normal behavior of the new “System and workload logging and monitoring”. Is there any solution?,

Answer 1

If you want to keep tidy your bucket, the best way is to create a new bucket, and have the old one serve as a reservoir of the old logs.

Regarding the new structure, is the way that the new workload logging and monitoring schema. If you process the logs afterwards, to discriminate them you will need to group the lines by the following attributes:

"resource":{
  "labels":{
    "cluster_name":"cluster-1",
    "container_name":"workload-1",
...
}

However, there is no built-in solution to avoid this new logging schema.