Configuring filebeat to graylog with TLS (connection reset)

Question

I have successfully created a graylog server (in docker container) that ingests logs from filebeat on a separate machine.

However I of course would like to have the messages encrypted. I am attempting to set this up however I cannot seem to get graylog to accept the connection, instead it is always being reset by peer:

{"log.level":"error","@timestamp":"2023-01-04T15:08:57.746+0100","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://<graylog_ip>:5044)): read tcp 192.168.178.99:54372-\><graylog_ip>:5044: read: connection reset by peer","service.name":"filebeat","ecs.version":"1.6.0"}

(Without tls the connection works as intended, a new line appears in graylog every time one is added to my test log file.)

Setup Details

I created a filebeat.crt and filebeat.key file with openssl. I confirmed that the hostname for the certificate was the same hostname for the server with graylog on it:

openssl genrsa -out filebeat.key 2048
openssl req -new -x509 -key filebeat.key -out filebeat.crt -days 3650

From my knowledge, a CA should not be required since I have copied the key myself, filebeat can just encrypt the data it sends with the filebeat.crt, then the server can decrypt with filebeat.key (perhaps this is not correct of me to imagine?)

I then copied both files to the server and local machine. In my compose file I mounted the key into the graylog container and restarted. Then I set up the input configuration that was working previously to have:

    bind_address: 0.0.0.0
    charset_name: UTF-8
    no_beats_prefix: false
    number_worker_threads: 12
    override_source: <empty>
    port: 5044
    recv_buffer_size: 1048576
    tcp_keepalive: false
    tls_cert_file: /etc/graylog/server/filebeat.crt
    tls_client_auth: disabled
    tls_client_auth_cert_file: <empty>
    tls_enable: true
    tls_key_file: /etc/graylog/server/filebeat.key
    tls_key_password: ********

Then in filebeat I have the following configuration (I also tried converting and using filebeat.pem for the certificate, but no change):

output.logstash:
  hosts: ["<graylog_ip>:5044"]
  ssl.certificate: '/etc/pki/tls/certs/filebeat.crt'
  ssl.key: '/etc/pki/tls/private/filebeat.key'

I really cannot see the issue, any help would be greatly appreciated!

Answer 1

First, try to debug filebeat using

 /usr/bin/filebeat -e -d '*' -c filebeat_5.1.2.conf

Probably you will discover that CA is needed or something like that.

But my best quess is that filebeat tries to verify hostname and certivicate name, your generated certificate could not have CN identical to hostname.

Proper solution is using:

ssl.verification_mode: none

Well, this solution works for me.