Distributed Java Application inside docker swarm

Question

My goal is to run a distributed NIFI cluster inside docker containers on a docker swarm. The configurations I made to the official NIFI docker image work in a way that I am able to run the cluster.

For the cluster I use a single service and each replica is a separate NIFI instance. Since this works I want to continue with security now. I started by applying a publically signed wildcard certificate to the java application using a secret (handing over the trust- and keystore). For my opinion this could be a feasable approach for most distributed java applications. But with NIFI I now got the issue that NIFI itself does not support wildcard certificates.

I am currently thinking about an approach how to run the cluster in a way that each container has its own certificate. My current idea is to run self signed certificates inside the container using a self managed internal CA which the NIFI JVMs can trust. Since I am not 100% sure if this would be the right approach to this issue I am thankful for hints and ideas.
NIFI uses some ports to do the communication and requests are issued using the HTTP/S protocol. NIFI itself runs as java application on the nodes/containers.

Answer 1

Apache NiFi provides a TLS Toolkit which automates much of this process for you. It can run an ephemeral or long-lived internal certificate authority (CA) which generates internal keys and uses them to create a self-signed CA certificate and sign incoming certificate signing requests (CSR). Each node that comes online can reach out to the CA service and install a properly-configured certificate in its keystore and truststore, and automatically populate the nifi.properties file with the locations and passwords for those files with a single command-line invocation. This can be configured to run at deployment time via Dockerfile, Ruby/Python/shell script, etc.

A HMAC/SHA-256 signature is calculated over the SPKI using a shared secret token value to ensure rogue/malicious services don't get issued certificates. All the certificates will be signed by the same CA cert, and this is already populated in the truststore, so each node in the cluster will trust the others. The requested CN is also populated in the SAN entries, and additional SAN entries are supported, so this is RFC 6125 compliant.

As noted above, wildcard certificates are not supported and not recommended, for a litany of reasons ( some enumerated here ) but also because a more secure deployment of unique certificates is made easy using the provided toolkit. There are ongoing efforts ( NIFI-5443 , NIFI-5398 ) to modify some internal cluster communications to be resilient in the face of wildcard certificates, but these are not currently available.