How do I enable logging/tracing in Apache Calcite using Sqlline?

Question

Following https://calcite.apache.org/docs/tutorial.html , I ran Apache Calcite using SqlLine. I tried activating tracing as instructed in https://calcite.apache.org/docs/howto.html#tracing . However, I don't get any logging. Here is the content of my session (hopefully containing all relevant information):

root@3b8279cda4cd:~/calcite/example/csv# egrep "^[^#]"  ../../core/src/test/resources/log4j.properties
log4j.rootLogger=TRACE, A1
log4j.logger.org.apache.calcite.runtime.CalciteException=FATAL
log4j.logger.org.apache.calcite.sql.validate.SqlValidatorException=FATAL
log4j.logger.org.apache.calcite.plan.RexImplicationChecker=ERROR
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p - %m%n
log4j.logger.org.apache.calcite.plan.RelOptPlanner=DEBUG
log4j.logger.org.apache.calcite.plan.hep.HepPlanner=TRACE
root@3b8279cda4cd:~/calcite/example/csv# cat target/classpath.txt
/root/.m2/repository/org/apache/calcite/calcite-core/1.20.0-SNAPSHOT/calcite-core-1.20.0-SNAPSHOT.jar:/root/.m2/repository/commons-codec/commons-codec/1.12/commons-codec-1.12.jar:/root/.m2/repository/org/apache/commons/commons-dbcp2/2.6.0/commons-dbcp2-2.6.0.jar:/root/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/root/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/root/.m2/repository/com/esri/geometry/esri-geometry-api/2.2.0/esri-geometry-api-2.2.0.jar:/root/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.9.8/jackson-annotations-2.9.8.jar:/root/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-yaml/2.9.8/jackson-dataformat-yaml-2.9.8.jar:/root/.m2/repository/org/yaml/snakeyaml/1.23/snakeyaml-1.23.jar:/root/.m2/repository/com/yahoo/datasketches/sketches-core/0.9.0/sketches-core-0.9.0.jar:/root/.m2/repository/com/yahoo/datasketches/memory/0.9.0/memory-0.9.0.jar:/root/.m2/repository/com/jayway/jsonpath/json-path/2.4.0/json-path-2.4.0.jar:/root/.m2/repository/net/hydromatic/aggdesigner-algorithm/6.0/aggdesigner-algorithm-6.0.jar:/root/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/root/.m2/repository/org/codehaus/janino/janino/3.0.11/janino-3.0.11.jar:/root/.m2/repository/org/codehaus/janino/commons-compiler/3.0.11/commons-compiler-3.0.11.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/root/.m2/repository/org/apache/calcite/calcite-linq4j/1.20.0-SNAPSHOT/calcite-linq4j-1.20.0-SNAPSHOT.jar:/root/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.9.8/jackson-core-2.9.8.jar:/root/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.9.8/jackson-databind-2.9.8.jar:/root/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/root/.m2/repository/org/apache/calcite/calcite-core/1.20.0-SNAPSHOT/calcite-core-1.20.0-SNAPSHOT-tests.jar:/root/.m2/repository/junit/junit/4.12/junit-4.12.jar:/root/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/root/.m2/repository/org/apache/calcite/avatica/avatica-core/1.15.0/avatica-core-1.15.0.jar:/root/.m2/repository/org/apache/calcite/avatica/avatica-metrics/1.15.0/avatica-metrics-1.15.0.jar:/root/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/root/.m2/repository/org/apache/httpcomponents/httpclient/4.5.6/httpclient-4.5.6.jar:/root/.m2/repository/org/apache/httpcomponents/httpcore/4.4.10/httpcore-4.4.10.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.8/commons-lang3-3.8.jar:/root/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/root/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/root/.m2/repository/sqlline/sqlline/1.8.0/sqlline-1.8.0.jar:/root/.m2/repository/org/jline/jline-terminal/3.10.0/jline-terminal-3.10.0.jar:/root/.m2/repository/org/jline/jline-reader/3.10.0/jline-reader-3.10.0.jar:/root/.m2/repository/org/jline/jline-terminal-jansi/3.10.0/jline-terminal-jansi-3.10.0.jar:/root/.m2/repository/org/fusesource/jansi/jansi/1.17.1/jansi-1.17.1.jar:/root/.m2/repository/org/jline/jline-terminal-jna/3.10.0/jline-terminal-jna-3.10.0.jar:/root/.m2/repository/net/java/dev/jna/jna/4.2.2/jna-4.2.2.jar:/root/.m2/repository/org/jline/jline-builtins/3.10.0/jline-builtins-3.10.0.jar:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar:/root/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar
root@3b8279cda4cd:~/calcite/example/csv# _JAVA_OPTIONS="-Djava.util.logging.config.file=../../core/src/test/resources/log4j.properties" ./sqlline
Picked up _JAVA_OPTIONS: -Djava.util.logging.config.file=../../core/src/test/resources/log4j.properties
sqlline version 1.8.0
sqlline> !connect jdbc:calcite:model=target/test-classes/model.json admin admin
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/root/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Transaction isolation level TRANSACTION_REPEATABLE_READ is not supported. Default (TRANSACTION_NONE) will be used instead.
0: jdbc:calcite:model=target/test-classes/mod> SELECT * FROM emps where "AGE" < 50;
+-------+-------+--------+--------+------+-------+-----+---------+---------+------------+
| EMPNO | NAME  | DEPTNO | GENDER | CITY | EMPID | AGE | SLACKER | MANAGER |  JOINEDAT  |
+-------+-------+--------+--------+------+-------+-----+---------+---------+------------+
| 100   | Fred  | 10     |        |      | 30    | 25  | true    | false   | 1996-08-03 |
| 120   | Wilma | 20     | F      |      | 1     | 5   |         | true    | 2005-09-07 |
+-------+-------+--------+--------+------+-------+-----+---------+---------+------------+
2 rows selected (1.601 seconds)
0: jdbc:calcite:model=target/test-classes/mod>

So, now the question: how do I get SqlLine/Calcite to log anything (in particular, I am interested in the query optimization log).

Answer 1

I have the impression that problem lies to the underlying implementation of the logger.

I am not an expert on logging configurations but I think specifying the properties file through -Djava.util.logging.config.file does not have any effect since the logger that is used (according to the classpath you provided) is the Log4J implementation ( slf4j-log4j12-1.7.25.jar ) and not the one of the jdk ( https://mvnrepository.com/artifact/org.slf4j/slf4j-jdk14/1.7.26 ).

I think that the right property to use for the log4j implementation is the folowing: -Dlog4j.configuration=file:C:\\Users\\user0\\workspaces\\apache-projects\\apache-calcite\\core\\src\\test\\resources\\log4j.properties