通过 kubernetes/skaffold 将私有 npm 存储库拉到 docker 容器

Question

I am new to skaffold, k8s, docker set and I've been having trouble building my application on a cluster locally.

I have a code repository that is trying to pull a private NPM package but when building it loses the.npmrc file or the npm secret.

npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@sh1ba%2fcommon - Not found
npm ERR! 404 
npm ERR! 404  '@sh1ba/common@^1.0.3' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-06-02T06_08_57_246Z-debug.log
unable to stream build output: The command '/bin/sh -c npm install' returned a non-zero code: 1. Please fix the Dockerfile and try again..

Ideally I'd like to avoid hard coding the secret into the file and use a k8s environment variable to pass in the key to docker as a secret. I am able to (kind of) do it with the docker build command:

• with "--build-args" with npm secret (the not safe way)
• with "--secret" with npm secret (the better way)
• copying the.npmrc file directly, npm install ing and deleting it right after

The issue arises when I try to build it using kubernetes/skaffold. After running, it doesn't seem like any of the args, env variables, or even the.npmrc file is found. When checking in the dockerfile for clues I was able to identify that nothing was being passed over from the manifest (args defined, .npmrc file, etc) to the dockerfile.

Below is the manifest for the application:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-depl
spec:
  replicas: 1
  selector: 
    matchLabels:
      app: auth
  template:
    metadata:
      labels:
        app: auth
    spec:
      containers:
        - name: auth
          image: auth
          env:
            - name: NPM_SECRET
              valueFrom:
                secretKeyRef:
                  name: npm-secret
                  key: NPM_SECRET
          args: ["--no-cache", "--progress=plain", "--secret", "id=npmrc,src=.npmrc"]

Here's the code in the dockerfile:

# syntax=docker/dockerfile:1.2
# --------------> The build image
FROM node:alpine AS build
WORKDIR /app
COPY package*.json .
RUN --mount=type=secret,mode=0644,id=npmrc,target=/app/.npmrc \
  npm install

# --------------> The production image
FROM node:alpine

WORKDIR /app
COPY package.json .
COPY tsconfig.json .
COPY src .
COPY prisma .

COPY --chown=node:node --from=build /app/node_modules /app/node_modules
COPY --chown=node:node . /app
s
RUN npm run build

CMD ["npm", "start"]

And also the skaffold file:

apiVersion: skaffold/v2alpha3
kind: Config
deploy:
  kubectl:
    manifests:
      - ./infra/k8s/*
      - ./infra/k8s-dev/*
build:
  local:
    push: false
  artifacts:
    - image: auth
      context: auth
      docker:
        dockerfile: Dockerfile
      sync:
        manual:
          - src: 'src/**/*.ts'
            dest: .

A few notes:

• I can't locate the.npmrc file regardless of where I copy and paste it (in auth, in the manifest, in skaffold and in the ~/ directories)
• I would like to also make it semi-usable (pretty reusable) in production too so that I don't need to do a complete overhaul if possible (but if this is bad practice I'd like to hear more about it)
• I've been able to make it work with buildArgs in the skaffold.yaml file but I'm not sure how that would translate into a production environment as I can't pass build args from kubernetes to docker (and I read that it isn't safe and that secrets should be used)
• The args in the manifest are throwing this error too (server runs if the args are commented out):

 - deployment/auth-depl: container auth terminated with exit code 9
    - pod/auth-depl-85fb8975d8-4rh9r: container auth terminated with exit code 9
      > [auth-depl-85fb8975d8-4rh9r auth] node: bad option: --progress=plain
      > [auth-depl-85fb8975d8-4rh9r auth] node: bad option: --secret
 - deployment/auth-depl failed. Error: container auth terminated with exit code 9.

Any insight would be amazing, I've been fiddling with this for far too long now.

Thank you!

Answer 1

Building and deploying an image to Kubernetes is at three levels:

1. Your local system where you initiate the building of an image
2. The Docker build that populates the image and then stores that image somewhere
3. The Kubernetes cluster that loads and starts running that image

Docker is not involved in #3. (This is only partially true, since some clusters use Docker to run the containers too, but that's a hidden detail and is also changing.)

There are two places where you might communicate secrets:

• at image build time (steps #1 to #2): you can use Docker --build-args or mounting secrets with --secret (both require Buildkit)
• at deployment time (step #3): you use Kubernetes secrets or config maps, which are configured separately from the image build

Skaffold supports passing build-time secrets, like your npm password, with Docker's --build-args and --secret flags, though they are slightly renamed.

buildArgs supports Go-style templating, so you can reference environment variables like MYSECRET as {{.MYSECRET}} :

build:
  local:
    useBuildkit: true
  artifacts:
    - image: auth
      context: auth
      docker:
        buildArgs:
          MYSECRET: "{{.MYSECRET}}"

Then you can reference MYSECRET within your Dockerfile :

ARG MYSECRET
RUN echo MYSECRET=${MYSECRET}

Note that build-args are not propagated into your container unless you explicitly assign it via an ENV MYSECRET=${MYSECRET} .

If the secret is in a local file, you can use the secret field in the skaffold.yaml :

build:
  local:
    useBuildkit: true
  artifacts:
    - image: auth
      context: auth
      docker:
        secret:
          id:   npmrc
          src: /path/to/.npmrc

and you'd then reference the secret as you are in your Dockerfile :

RUN --mount=type=secret,mode=0644,id=npmrc,target=/app/.npmrc \
  npm install

---

Now in your Deployment , you're attempting to setting args for your container:

          args: ["--no-cache", "--progress=plain", "--secret", "id=npmrc,src=.npmrc"]

The args field overrides the CMD directive set in your image. This field is used to provide command-line arguments provided to your image's entrypoint, which is likely node . If you want to reference a secret in a running container on a cluster, you'd use a Secret or ConfigMap .