How to perform os.environ join in python?

Question

I have a configuration of os.environ with default values (that cover 90% of my needs). I have a special application-framework-package, for example called SALOME , that does not provide package installation into system environment and tries to be self contained, it also requires use of special old technologies that rely on environmental variables thus sys.path and PYTHONPATH are not the only things it needs. I can get all variables it needs when it started calling os.environ inside an environment it creates. I can then serialize that os.environ dictionary.

I wonder how to apply a merge of os.environ I have on my currently running system with one I obtained by serializing?

Answer 1

Let's assume you have done something like the following to serialize the environment:

import json
import os

with open('environ.json', 'w') as f:
    json.dump(dict(**os.environ), f)

You can now read those back like this (in another program)

import json
import os

with open('environ.json', 'r') as f:
    os.environ.update(json.load(f))

This will only add or change the current environment variables to match the saved ones, but any additional variables will remain.

If you want to update only specific variables by adding them (so for instance to add extra paths), you can do that explicitly:

with open('environ.json', 'r') as f:
    loadedenv = json.load(f)

pathvars = ['PATH', 'PYTHONPATH']

for p in pathvars:
    os.environ[p] += ':' + loadedenv[p]

Answer 2

You can use the package environs to achieve exporting os.environ dictionary. It has inbuilt dumper/loader for exporting importing the environment variables.

from environs import Env

env = Env()
# reading an environment variable
gh_user = env('GITHUB_USER')  # => 'sloria'
secret = env('SECRET')  # => raises error if not set

# casting
api_key = env.str('API_KEY')  # => '123abc'
date = env.date('SHIP_DATE')  # => datetime.date(1984, 6, 25)

# serialize to a dictionary of simple types (numbers and strings)
env.dump()
# { 'API_KEY': '123abc',
# 'GITHUB_USER': 'sloria',    
# 'SECRET': 'AASJI93WSJD93DWW3X0912NS2',
# 'SHIP_DATE': '1984-06-25'}}

Answer 3

If you want to have multiple values for a dictionary which the standard python dictionary does not offer than you can use

werkzeug.datastructures.MultiDict
os.environ = MultiDict([('Key1', 'First Value'), ('Key1', 'Second Value')])

The update will also work the same way as I have mentioned below.

If you do not want to preserve the old key values before you merge with the new dictionary then you can do the following.

Since os.environ is a dictionary that you already have in memory the other dict is the one you are reading from would need to be converted into json. I generally use ujson since it is really fast.

 os.environ.update(new_dict)

If you want to save the json u can dump it to a file.

import ujson

with open('data.json', 'w') as file:
    ujson.dump(dict(**os.environ), file)

if you want to read the file and update the os.environ dictionary than you can use.

with open('environ.json', 'r') as f:
    os.environ.update(ujson.load(f))