pip install -r requirements.txt doesn't expand my environment variables

Question

I have private package in a Gitlab registry. In order to install it I had to create a token.

When I run the following command:

pip install my-package-name --index-url https://${GITLAB_TOKEN_USER}:${GITLAB_TOKEN}@gitlab.com/api/v4/projects/12345678/packages/pypi/simple

The package get installed properly. With following lines (we can see that the ENV var where expanded properly)

Looking in indexes: https://gitlab%2Bdeploy-token-564332:****@gitlab.com/api/v4/projects/1234567/packages/pypi/simple
Collecting my-package-name.
  Downloading https://gitlab.com/api/v4/projects/12345678/packages/ ...
....

But when i try to install through pip install -r requirements.txt . I get this message:

Looking in indexes: https://%24%7BGITLAB_TOKEN_USER%7D:****@gitlab.com/api/v4/projects/1234567/packages/pypi/simple
WARNING: 401 Error, Credentials not correct for https://gitlab.com/api/v4/projects/1234567/packages/pypi/simple/my-package-name/
ERROR: Could not find a version that satisfies the requirement my-package-name (from versions: none)
ERROR: No matching distribution found for my-package-name

And it seems that when I use this command the ENV var doesn't get expanded.

My requirements.txt looks like this:

--index-url https://${GITLAB_TOKEN_USER}:${GITLAB_TOKEN}@gitlab.com/api/v4/projects/1234567/packages/pypi/simple

my-package-name

I'm using pip version 22.3.1

Any why this is happenning? And if there is any other way to install multiple private packages without hardcoding my token in the requirements.

Answer 1

As mentioned by @Abdul Aziz Barkat my environment variable were not set.

I used source.env to load my variables. And when I was checking with echo $MY_ENV my var were displaying. So I though it was ok.

But when I used cat <(envsubst < requirements.txt) to check for my variables. They were replaced in my requirements.txt by an empty string.

As pointed by @Charles Duffy, in order to export my var from my .env file I should use:

set -a
source .env
set +a

this source helped me: https://ostechnix.com/difference-between-defining-bash-variables-with-and-without-export/