在IntelliJ中启用Spark日志

Question

我正在intelliJ上运行我的spark作业，并收到以下错误：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/sanchay/brazil-pkg-cache/packages/Log4j-slf4j/Log4j-slf4j-2.x.698.1338/RHEL5_64/DEV.STD.PTHREAD/build/lib/log4j-slf4j-impl-2.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/sanchay/brazil-pkg-cache/packages/Slf4j_log4j/Slf4j_log4j-1.7.2463.17539/RHEL5_64/DEV.STD.PTHREAD/build/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/sanchay/.m2/repository/org/slf4j/slf4j-log4j12/1.7.16/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

但是，当我使用spark-submit运行相同的jar /类时，我能够获取所需的所有INFO日志：

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/11/08 00:17:27 INFO SparkContext: Running Spark version 2.2.0
17/11/08 00:17:27 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/08 00:17:28 INFO SparkContext: Submitted application: JavaSparkPi
17/11/08 00:17:28 INFO SecurityManager: Changing view acls to: sanchay
17/11/08 00:17:28 INFO SecurityManager: Changing modify acls to: sanchay
17/11/08 00:17:28 INFO SecurityManager: Changing view acls groups to: 
17/11/08 00:17:28 INFO SecurityManager: Changing modify acls groups to: 
17/11/08 00:17:28 INFO SecurityManager: SecurityManager: authentication dis

如何在IntelliJ中切换到默认日志文件，以便可以通过INFO日志获取有关Spark Ui等的信息？

Answer 1

• 假设您使用maven管理代码。 构建dependency：tree，并检查使用的日志库版本。 https://www.slf4j.org/legacy.html带有图表的很好的图表说明了什么是库。 所有其他应排除在外

• 你如何建立罐子提交火花？ 我假设您没有包括所有依赖项，因此您没有额外的日志记录库。