Where does YARN application logs get stored in EMR before sending to S3

Question

我需要将来自EMR的Yarn应用程序日志写入到S3以外的其他源中。.请问我在EMR主实例中保存应用程序日志的位置是什么

Answer 1

If the application is submitted to the emr as a step then the logs will reside in:

/var/log/hadoop/steps/<<step-id>>/<<log-file>>

most logs for emr can be found under the /var/logs directory in the master node

you could also use the yarn cli to get the application logs and redirect the returned log stream to a file to do whatever you want with.

yarn logs -applicationId <<application_id>> > application_log_file.log

Answer 2

Yarn logs are found at /var/log/hadoop-yarn/ , and yarn container logs are found at /var/log/hadoop-yarn/container