Kubernetes Minikube Secrets appear not mounted in Pod

Question

I have a "Deployment" in Kubernetes which works fine in GKE, but fails in MiniKube. I have a Pod with 2 containers:- (1) Nginx as reverse proxy ( reads secrets and configMap volumes at /etc/tls & /etc/nginx respectively ) (2) A JVM based service listening on localhost

The problem in the minikube deployment is that the Nginx container fails to read the TLS certs which appear not to be there - ie the volume mount of the secrets to the Pod appears to have failed.

nginx: [emerg] BIO_new_file("/etc/tls/server.crt") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/tls/server.crt','r') error:2006D080:BIO routines:BIO_new_file:no such file)

But if I do "minikube logs" I get a large amount of seemingly "successful" tls volume mounts...

 MountVolume.SetUp succeeded for volume "kubernetes.io/secret/61701667-eca7-11e6-ae16-080027187aca-scriptwriter-tls" (spec.Name: "scriptwriter-tls")

And the secret themselves are in the cluster okay ...

$ kubectl get secrets scriptwriter-tls
NAME               TYPE      DATA      AGE
scriptwriter-tls   Opaque    3         1h

So it would appear that as far as miniKube is concerned all is well from a secrets point of view. But on the other hand the nginx container can't see it.

I can't logon to the container either since it keeps terminating.

For completeness the relevant sections from the Deployment yaml ... Firstly the nginx config...

  - name: nginx
    image: nginx:1.7.9
    imagePullPolicy: Always
    ports:
    - containerPort: 443
    lifecycle:
      preStop:
        exec:
          command: ["/usr/sbin/nginx", "-s", "quit"]
    volumeMounts:
      - name: "nginx-scriptwriter-dev-proxf-conf"
        mountPath: "/etc/nginx/conf.d"
      - name: "scriptwriter-tls"
        mountPath: "/etc/tls"

And secondly the volumes themselves at the container level ...

  volumes:
    - name: "scriptwriter-tls"
      secret:
        secretName: "scriptwriter-tls"
    - name: "nginx-scriptwriter-dev-proxf-conf"
      configMap:
        name: "nginx-scriptwriter-dev-proxf-conf"
        items:
          - key: "nginx-scriptwriter.conf"
            path: "nginx-scriptwriter.conf"

Any pointers of help would be greatly appreciated.

Answer 1

I am a first class numpty! :-) Sometimes the error is just the error! So the problem was that the secrets are created using local $HOME/.ssh/* certs ... and if you are generating them from different computers with different certs then guess what?! So all fixed now :-)