Greengrass V2 generates extra `stdout` logs

Question

. 。 Answers to this question are eligible for a +100 reputation bounty. user-517752 is looking for a . 。

I deploy greengrass components into my EC2 instance. The deploy greengrass components have been generating logs which wraps around my python log.

what is causing the "wrapping" around it? how can I remove these wraps.

For example, the logs in bold are wraps the original python log. The log in emphasis is generated by my python log formatter.

2022-12-13T23:59:56.926Z [INFO] (Copier) com.bolt-data.iot.RulesEngineCore: stdout. [2022-12-13 23:59:56,925][DEBUG ][iot-ipc] checking redis pub-sub health (io_thread[140047617824320]:pub_sub_redis:_connection_health_nanny_task:61). {scriptName=services.com.bolt-data.iot.RulesEngineCore.lifecycle.Run, serviceName=com.bolt-data.iot.RulesEngineCore, currentState=RUNNING}

The following is my python log formatter.

    formatter = logging.Formatter(
        fmt="[%(asctime)s][%(levelname)-7s][%(name)s] %(message)s (%(threadName)s[%(thread)d]:%(module)s:%(funcName)s:%(lineno)d)"
    )

    # TODO: when we're running in the lambda function, don't stream to stdout
    _handler = logging.StreamHandler(stream=stdout)
    _handler.setLevel(get_level_from_environment())
    _handler.setFormatter(formatter)

Answer 1

It looks like the logs you are seeing are being generated by different components or processes and are being interleaved or "wrapped" around each other. This can happen when multiple processes or threads are writing to the same log file or output stream, or when the logs are being collected and processed by a central log management system.

To prevent the wrapping of logs, you may want to consider the following options:

1. Configure each component or process to write to a separate log file or output stream. This will allow you to separate the logs and avoid interleaving.
2. Use log message timestamps to order the logs. If the logs include timestamps, you can use them to order the logs and determine which log messages were generated first. This can help you to understand the order in which events occurred.
3. Use a log management system that can parse and order the logs. There are many log management tools available that can collect and parse logs from multiple sources, including EC2 instances. These tools can often automatically order the logs based on timestamps and other metadata, which can make it easier to understand and analyze the logs.

Answer 2

by default Greengrass Nucleus captures the stdoud and stderr streams from the processes it manages, including custom components. It then outputs each line of the logs with the prefix and suffix you have highlighted in bold . This cannot be changed. You can switch the format from TEXT to JSON which can make the log easier to parse by a machine (check greengrass-nucleus-component-configuration - logging.format )

import logging
from logging.handlers import RotatingFileHandler

logger = logging.Logger(__name__)
_handler = RotatingFileHandler("mylog.log")

# additional _handler configuration

logger.addHandler(_handler)

If you want to output a log containing only what your application generates, change the logger configuration output to file. You can write the file in the work folder of the component or in another location, such as /var/log

Cheers, Massimiliano