Logstash in docker error SQL Server & JDBC not loaded

Question

Build an Elastic stack where I have to connect to a SQL Server database. So I build my own Logstash image where I import the JDBC driver to the container.

FROM docker.elastic.co/logstash/logstash:6.3.0
USER root
COPY mssql-jdbc-6.4.0.jre9.jar /opt/
# Add your logstash plugins setup here
RUN logstash-plugin install logstash-input-jdbc
# Example: RUN logstash-plugin install logstash-filter-json

Then I launch my docker-compose file.

# Elastic and Kibana config remove for clarity

logstash:
#    image: docker.elastic.co/logstash/logstash:${TAG}
    build: logstash/
    container_name: Logstash
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    env_file:
      - .env
    ports:
      - '5001:5001'
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    depends_on:
      - elasticsearch1
    networks:
      - esnet

Here my logstash.conf

input {
    jdbc {
           jdbc_driver_library => "/opt/mssql-jdbc-6.4.0.jre9.jar"
           jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
           jdbc_connection_string => "jdbc:sqlserver://****.database.windows.net"
           jdbc_user => "${DB_USERNAME}"
           jdbc_password => "${DB_PASSWORD}"
           statement => "select * from *****.dbo.association"
        }
}

output {
    elasticsearch {
        hosts => ["http://elasticsearch1:9200", "http://elasticsearch2:9200"] 
    }
    stdout {
        codec => rubydebug
    }
}

Then I got this error:

Error: com.microsoft.sqlserver.jdbc.SQLServerDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?

Answer 1

You can go the docker /opt/mssql-jdbc-6.4.0.jre9.jar and check whether the file is there or you have right permission.

or you can bind another folder just for the javalib and then the jdbc_driver_library should be a path in docker container jdbc_driver_library => "/usr/share/logstash/javalib/mssql-jdbc-6.4.0.jre9.jar"

docker run --rm -it --network="elasticsearch_esnet" -v /data/logstash/lib:/usr/share/logstash/javalib  -v /data/logstash/pipeline/:/usr/share/logstash/pipeline/ docker.elastic.co/logstash/logstash:6.6.2

Answer 2

Had the same issue, you have to copy the jdbc driver file to

LOGSTASH_FOLDER/logstash-core/lib/jars/

directory.