How logs printed directly onto the console in yarn-cluster mode using spark

Question

我是Spark的新手，我想在纱线簇模式下使用apache spark在控制台上打印日志。

Answer 1

You need to check the value in log4j.properties file. In my case i have this file in /etc/spark/conf.dist directory

log4j.rootCategory=INFO,console

INFO - prints the all the logs on the console. You can change the value to ERROR, WARN to limit the information you would like to see on the console as sparks logs can be overwhelming