How to set environment variables/app secrets in Google App Engine

Question

The question is how can I set application secrets to make them available in application.yml ?

On heroku I was doing it simply, by setting environment variable for dyno, and acces it as:

server:
  port: ${PORT}
security:
  user:
    password: ${USERPASSWORD}

eureka:
  client:
    register-with-eureka: false
    fetch-registry: false
  instance:
    hostname: localhost
    securePortEnabled: true
  password: ${EUREKAPASSWORD}

How to achieve that in Google App Engine? I was trying with datastore :

Unfornately I don't know how to inject those values into my *.yml file.

EDIT:

One more important thing to add. I am using maven appengine plugin to deploy my app via CI pipeline, so there is no possibility for me to push app.yaml file to App Engine

Answer 1

If you want to store secrets that are available to the app at runtime, keeping them in the datastore isn't a bad idea. I know of many apps that do that.

Here's an app used by the Khan Academy that's a good example of storing secret credentials in the datastore. It's in Python, but you can get the general idea. Note that on first admin login , it prompts for secrets to store.

Answer 2

Google has also a tutorial on how to store encrypted secrets. https://cloud.google.com/kms/docs/store-secrets

TLDR: a separate bucket to store the encrypted secrets, instances download it when needed, decrypt using Google KMS ( https://cloud.google.com/kms/ ) and remove afterwards.

Answer 3

The best and secure way is to use GCP KMS or some third party secrets manager product like vault .

GCP KMS

1. We need to use a service account with encrypt and decrypt permission(role) to encrypt the credentials(secrets) file.
2. Upload the encrypted credential file to GCS
3. Fetch the encrypted credential from GCS and decrypt and parse it(Eg parse to plain java object) at runtime in your application code.

Datastore

Yes. We can store credentials/secrets environment variables into datastore and fetch them at runtime in application code.

Pros:

1. Simple
2. It can be used almost everywhere, GAE standard environment, GAE flexible environment, GCE, GCF, GKE, Cloud Run.

Cons:

1. Security is not as good as KMS.

GCE metadata

I used to use GCE metadata server to store my secret environment variables.

Pros:

1. It supports GAE, GCE, GKE.

2. Very simple. We just need to send HTTP requests to http://metadata.google.internal/computeMetadata/v1/ endpoint to fetch our custom metadatas(the secrets environment variables).

Cons:

1. Last year, GCE metadata doesn't support Cloud Function. (runtime: nodejs10).I can't fetch my custom secrets environment variables from GCE metadata within cloud function. But built-in metadatas can be fetched, like projectId .

2. security is not as good as KMS.

configmap and secrets(Only for GKE)

Simple base64 encryption is possible. Medium difficulty to use. Security is not as good as KMS.

Another hack way

I also create a post for this question here: How to pass system environment variables to app.yaml?

Yes, the Linux script way can do everything. But I don't like these hack way.