How to easily deploy sensitive config data to Amazon Elastic Beanstalk for Flask web server

Question

I have a Flask application that I want to deploy to Amazon Elastic Beanstalk, but have the following problem:

1. Our configuration contains a lot of secret API keys, so we don't have it checked into Git.
2. Elastic Beanstalk only deploys the files that are committed to Git, so our config doesn't get uploaded as part of the deployment process.
3. Elastic Beanstalk offers a way to set and read environment variables - but Flask typically expects environment variables to be set in Python files that are imported at runtime.

Is there a way we can get our configuration file automatically uploaded to AWS EB alongside the source code when we deploy it? I considered having it in a file that gets created via configuration in .ebextensions/... but that would just need to be in Git anyway, defeating the object.

Or do we have to (a) convert all our code to read configuration from environment variables, and (b) create some sort of pre-startup script that injects the values in the current config scripts into the environment? It wouldn't be ideal to have to maintain 2 totally different ways of reading configuration data into our app depending on whether it's running on AWS or not.

I have seen this question/answer - How to specify sensitive environment variables at deploy time with Elastic Beanstalk - and I don't think that adequately addresses the issue because it is not very scalable to large numbers of config options nor deals with the fact that Flask typically expects its config in a different format.

Answer 1

If you are doing a 12 factor application, the simplest way to handle this kind of configuration is to lookup environment variables by default, and fall back on your hard-coded configurations:

from os import environ

from flask.config import Config

class EnvConfiguration(Config):
    """Configuration sub-class that checks its environment

    If no environment variable is available,
    it falls back to the hard-coded value provided"""
    def __getitem__(self, key):
        environ.get(key, super(EnvConfig, self).__getitem__(key))

You can then override either Flask.config_class (v1+) or Flask.make_config to ensure that the configuration used is the one you want:

class CustomApp(Flask):
    # 1.0+ way
    config_class = EnvConfiguration

    # 0.8+ way (you only need one of these)
    def make_config(self, instance_relative=False):
        config = super(CustomApp, self).make_config(instance_relative=instance_relative)
        config = EnvConfig(**config)
        return config