Why are the variables inside my .env file resetting to the original file even after changing it from inside my Python script?

Question

I'm trying to implement a checkpoint mechanism for my original script with the following structure:

import os
import sys
from dotenv import load_dotenv

def main():
    load_dotenv()

    checkpoint = int(os.getenv('CHECKPOINT'))

    if checkpoint == 0:
        ...
        do something
        ...
        checkpoint += 1
        os.environ['CHECKPOINT'] = str(checkpoint)

    checkpoint = int(os.getenv('CHECKPOINT'))

    if checkpoint >= 1:
        ...
        do something
        ...
        checkpoint += 1
        os.environ['CHECKPOINT'] = str(checkpoint)

My .env file

CHECKPOINT=0

While this method is working inside a Windows machine (localhost), it is not working on my Ubuntu server.

That is, if my code fails somewhere on the checkpoint 2, I should be able to rerun it from that point only.

For example, if code fails at checkpoint=2:

Windows machine :

checkpoint=2

Ubuntu machine :

checkpoint=0

Also, I am using try and except to catch failures from the code, then exit system if there is any using:

sys.exit()

I can use a config.json to set these from time to time, but I really wanted to know why this is happening.

Answer 1

Here is a minimal working example:

import os


name = 'CHECKPOINT'


def getenv():
    return int(os.getenv(name) or 0)


def putenv(value):
    print(f'{name} (before): {getenv()}')
    os.environ[name] = str(value)
    print(f'{name} (after {value}): {getenv()}')
    return getenv()


checkpoint = getenv()
if checkpoint == 0:
    checkpoint = putenv(checkpoint + 1)
if checkpoint >= 1:
    checkpoint = putenv(checkpoint + 1)

and when you call it:

$ CHECKPOINT=1 python3 1.py 
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2
$ echo -n $CHECKPOINT
$ python3 1.py 
CHECKPOINT (before): 0
CHECKPOINT (after setting 1): 1
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2

The CHECKPOINT environment variable is not persisted as it's stored in the first process (python3 1.py) which exits.