简体   繁体   English

将 env 变量从 Google 的 Secret Manager 加载到在 Google Cloud Run 上运行但未通过 Cloud Build 部署的 Docker 容器中?

[英]Load env variables from Google's Secret Manager into Docker container that runs on Google Cloud Run, but not deployed via Cloud Build?

I'm currently delivering a node + nestjs application via as a docker container using Google's Cloud Run.我目前正在使用 Google 的 Cloud Run 通过 docker 容器交付 node + nestjs 应用程序。

I'm using the secrets manager for the secrets & using projects for dev/staging/prod and I'm trying to make the secrets available to my container in the cloud.我正在将秘密管理器用于秘密并使用项目进行开发/登台/生产,并且我正在尝试使秘密可用于我在云中的容器。

Current process is triggered by "yarn docker:auth" which triggers a series of bash scripts:当前进程由“yarn docker:auth”触发,触发一系列 bash 脚本:

docker build -t gcr.io/my_project_id_dev/auth-service -f .docker/auth.DockerFile . &&
gcloud auth activate-service-account abc@my_project_id_dev.iam.gserviceaccount.com --key-file=gcloud-sa.json &&
gcloud auth configure-docker &&
docker push gcr.io/my_project_id_dev/auth-service &&
gcloud beta run services replace .gcp/cloud_run/auth.yaml &&
gcloud run deploy auth ... --allow-unauthenticated  --platform managed --region europe-west2

The arguments/flags on the last command also doesn't work and I'm prompted to pick a platform & region every time I run it.最后一个命令上的参数/标志也不起作用,每次运行时都会提示我选择一个平台和区域。

I've tried adding the flags to my auth.yaml file as well as the secrets as exemplified in the documentation for Google Cloud Build but it fails every time due to syntax error.我已经尝试将标志添加到我的 auth.yaml 文件以及 Google Cloud Build 文档中举例说明的秘密,但由于语法错误,它每次都失败。

In the yaml file, I added the following at the bottom of the yaml file nested under no property:在 yaml 文件中,我在嵌套在 no 属性下的 yaml 文件的底部添加了以下内容:

availableSecrets:
  secretManager:
    - versionName: projects/my_project_id/secrets/mongo_uri/versions/latest
    env: 'mongo_uri'

My questions is:我的问题是:

  • Is it possible to do this via YAML at all?是否可以通过 YAML 做到这一点?

I've also added a startup function to my nodejs app that tries to use @google-cloud/secret-manager from npmjs to load the secrets into environment.我还在我的 nodejs 应用程序中添加了一个启动 function 尝试使用 npmjs 中的@google-cloud/secret-manager将机密加载到环境中。 I've got no issues doing this locally using the default credentials, but:使用默认凭据在本地执行此操作没有问题,但是:

  • Will the docker container inside Cloud run have any kind of "Default" credentials? Cloud run 中的 docker 容器是否有任何类型的“默认”凭据? and If not, what would be the best way to inject it?如果没有,注入它的最佳方法是什么? It seems bad practice to either build or ship the container with a service account key-file.使用服务帐户密钥文件构建或运送容器似乎是一种不好的做法。

The fundamental issue I'm trying to solve is getting those secrets into the container environment.我试图解决的基本问题是将这些秘密带入容器环境。

Thank you.谢谢你。

EDIT:编辑:

Wanted to add the YAML part where I am assigning the service account to the cloud run container:想要添加 YAML 部分,我将在其中将服务帐户分配给云运行容器:

spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '2'
        run.googleapis.com/client-name: cloud-console
        run.googleapis.com/sandbox: gvisor
    spec:
      containerConcurrency: 2
      containers:
      - image:  gcr.io/my_project_id/auth-service
        ports:
        - containerPort: 3000
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
      serviceAccountName: abc@my_project_id.iam.gserviceaccount.com
      timeoutSeconds: 300

But sadly this still results in this generic error:但遗憾的是,这仍然会导致这个一般错误:

(gcloud.beta.run.services.replace) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable.

With absolutely no further details within the logs themselves, blank, The container doesn't start locally due to no service account injected into it, but without that piece of code that handles the secrets-loading.日志本身中绝对没有更多详细信息,空白,由于没有注入服务帐户,容器不会在本地启动,但没有处理秘密加载的那段代码。 the container starts locally just fine.容器在本地启动就好了。

There's a bit to unpack here.这里有一点要解压。 In short, I think you might be confusing build time with runtime secrets and how they are accessed.简而言之,我认为您可能会将构建时间运行时机密以及它们的访问方式混淆。

If you do not need access to secrets in your compilation or test phases, you can omit the availableSecrets stanza from your cloudbuild.yaml .如果您不需要在编译或测试阶段访问机密,则可以从cloudbuild.yaml中省略availableSecrets节。 That pulls in secrets at build time .这会在构建时引入秘密。 For example, suppose you wanted to run your tests in Cloud Build, and you needed an API key or database password to execute the tests.例如,假设您想在 Cloud Build 中运行测试,并且需要 API 密钥或数据库密码来执行测试。 That's when you'd use theCloud Build + Secret Manager integration .那时您将使用Cloud Build + Secret Manager 集成


 gcloud auth activate-service-account abc@my_project_id_dev.iam.gserviceaccount.com --key-file=gcloud-sa.json

If you are using gcloud locally, you shouldn't need a service account.如果您在本地使用 gcloud,则不需要服务帐户。 You can authenticate to gcloud as yourself via:您可以通过以下方式对 gcloud 进行身份验证:

gcloud auth login && gcloud auth application-default login

You generally perform these steps once, as part of installing gcloud.作为安装 gcloud 的一部分,您通常会执行一次这些步骤。 The same is true for configure-docker . configure-docker也是如此。 You generally only run this command once您通常只运行一次此命令


The Cloud Run + Secret Manager integration is currently in preview. Cloud Run + Secret Manager 集成目前处于预览阶段。 You can request access using this Google Form .您可以使用此 Google 表单请求访问权限 (For future readers, if that form no longer exists, it means the integration is public and no signup is required.) (对于未来的读者,如果该表格不再存在,则意味着集成是公开的,无需注册。)


Will the docker container inside Cloud run have any kind of "Default" credentials? Cloud run 中的 docker 容器是否有任何类型的“默认”凭据? and If not, what would be the best way to inject it?如果没有,注入它的最佳方法是什么? It seems bad practice to either build or ship the container with a service account key-file.使用服务帐户密钥文件构建或运送容器似乎是一种不好的做法。

Yes, the Cloud Run container will execute as the default Compute Engine service account by default.是的,默认情况下,Cloud Run 容器将作为默认的 Compute Engine 服务帐号执行。 This service account has broad permissions, but it does not have permission to access secrets.此服务帐户具有广泛的权限,但没有访问机密的权限。

We recommend that you create a new service account specific for this workload and grant access to only the specific secrets it needs .我们建议您为此工作负载创建一个特定的新服务帐户,并仅授予对它需要的特定机密的访问权限 Then, you can run the Cloud Run container using your custom service account with the --service-account flag:然后,您可以使用带有--service-account标志的自定义服务帐号运行 Cloud Run 容器:

gcloud run deploy auth --service-account "...@...iam.gserviceaccount.com"

The arguments/flags on the last command also doesn't work and I'm prompted to pick a platform & region every time I run it.最后一个命令上的参数/标志也不起作用,每次运行时都会提示我选择一个平台和区域。

We'd need to see the full command to understand what's failing, specifically what is the ... .我们需要查看完整的命令以了解失败的原因,特别是...


With absolutely no further details within the logs themselves, blank, The container doesn't start locally due to no service account injected into it, but without that piece of code that handles the secrets-loading.日志本身中绝对没有更多详细信息,空白,由于没有注入服务帐户,容器不会在本地启动,但没有处理秘密加载的那段代码。 the container starts locally just fine.容器在本地启动就好了。

You can view the logs for the container in Cloud Logging from the sidebar.您可以从侧边栏中查看 Cloud Logging 中容器的日志。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 Google Cloud 上的环境变量? - Env variables on Google Cloud? GCP - 无法在 Cloud Run 中使用 Google Secret Manager (@google-cloud/secret-manager) - GCP - Can't use Google Secret Manager (@google-cloud/secret-manager) inside Cloud Run 使用 Node.js 更新 Google Cloud Secret Manager 中的数据 - Update data in Google Cloud Secret Manager using Node.js 谷歌云构建、云运行、云 SQL Prisma 迁移 - Google Cloud Build, Cloud Run, Cloud SQL Prisma Migration 部署在 Cloud Run 上 Docker 容器中的节点 JS 未从 Mongo DB 获取数据 - Node JS deployed in Docker container on Cloud Run not fetching data from Mongo DB Google Cloud Secrets - 重用密钥 - Google Cloud Secrets - Reusing a secret 如何从 React App 访问 Google Cloud Run 环境变量 - How to access Google Cloud Run Environment Variables from React App 如何通过Google Cloud调试在Docker容器内运行的Nodejs应用程序 - How to debug Nodejs app running inside Docker container via Google Cloud 如何让 puppeteer 在 Google Cloud Run/Cloud Build 中工作? - How to get puppeteer working in Google Cloud Run/Cloud Build? 从 Bitbucket 管道访问存储在 Google Secret Manager 中的环境变量 - Access environment variables stored in Google Secret Manager from Bitbucket pipelines
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM