Mounting a large file in Kubernetes

Question

We are running a pod in Kubernetes that needs to load a file during runtime. This file has the following properties:

• It is known at build time
• It should be mounted read-only by multiple pods (the same kind)
• It might change (externally to the cluster) and needs to be updated
• For various reasons (security being the main concern) the file cannot be inside the docker image
• It is potentially quite large, theoretically up to 100 MB, but in practice between 200kB - 10MB.

We have considered various options:

• Creating a persistent volume, mount the volume in a temporary pod to write (update) the file, unmount the volume, and then mount it in the service with ROX (Read-Only Multiple) claims. This solution means we need downtime during upgrade, and it is hard to automate (due to timings).
• Creating multiple secrets using the secrets management of Kubernetes, and then "assemble" the file before loading it in an init-container or something similar.

Both of these solutions feels a little bit hacked - is there a better solution out there that we could utilize for solving this?

Answer 1

You need to use a shared filesystem that supports Read/Write Multiple Pods. Here is a link to the CSI Drivers which can be used with Kubernetes and provide those access: https://kubernetes-csi.github.io/docs/drivers.html

Ideally, you need a solution that is not an appliance, and can run anywhere meaning it can run in the cloud or on-prem.

The platforms that could work for you are Ceph, GlusterFS, and Quobyte ( Disclaimer, I work for Quobyte )