如何从应用程序日志中避免Spark和Hive日志

Question

I would like to learn how to avoid Spark and Hive log from Application log. I mean both spark and hive are using log4j property file.

I have configured log4j.property file as per my application as under but it's printing Spark and hive console logging also. I am running application in HDFS environment.

log4j.rootLogger=DEBUG,debugLog, SplLog

log4j.appender.debugLog=org.apache.log4j.RollingFileAppender
log4j.appender.debugLog.File=logs/Debug.log
log4j.appender.debugLog.MaxFileSize=10MB
log4j.appender.debugLog.MaxBackupIndex=10
log4j.appender.debugLog.layout=org.apache.log4j.PatternLayout
log4j.appender.debugLog.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n
log4j.appender.debugLog.filter.f1=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.debugLog.filter.f1.LevelMax=DEBUG
log4j.appender.debugLog.filter.f1.LevelMin=DEBUG

log4j.appender.SplLog=org.apache.log4j.RollingFileAppender
log4j.appender.SplLog.File=logs/AppSplCmd.log
log4j.appender.SplLog.MaxFileSize=10MB
log4j.appender.SplLog.MaxBackupIndex=10
log4j.appender.SplLog.layout=org.apache.log4j.PatternLayout
log4j.appender.SplLog.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n
log4j.appender.SplLog.filter.f1=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.SplLog.filter.f1.LevelMax=FATAL
log4j.appender.SplLog.filter.f1.LevelMin=INFO

log4j.logger.debugLogger=DEBUG, debugLog
log4j.additivity.debugLogger=false

log4j.logger.AppSplLogger=INFO, SplLog
log4j.additivity.AppSplLogger=false

Answer 1

Raise the rootLogger level:

log4j.rootLogger=WARN,debugLog,SplLog

Or at least, set the org.apache logger level (without this config Spark and Hive loggers use the rootLogger config):

log4j.logger.org.apache=WARN