Intermittent pip install error in virtualenv?

Question

For the commands below:

virtualenv --system-site-packages `pwd`/.test

# load virtual environment
source `pwd`/.test/bin/activate

# install required python modules
# for some reason argparse is not included with credstash install but required at runtime :(
#PYPI_REPO=http://${PYPI_DNS}/aes/release/+simple/
#PYPI_DNS is internal company DNS 
pip install pip --upgrade -i ${PYPI_REPO} --quiet
pip install argparse==1.4.0 -i ${PYPI_REPO} --trusted-host ${PYPI_DNS} --quiet
pip install credstash==1.14.0 -i ${PYPI_REPO} --trusted-host ${PYPI_DNS} --quiet

This is sometimes work successfully.

1. However sometimes it will fail at the installation of argparse:

   Could not find a version that satisfies the requirement argparse==1.4.0 (from versions: ) No matching distribution found for argparse==1.4.0

2. Then at other times, it will fail installing credstash:

   Could not find a version that satisfies the requirement boto3>=1.1.1 (from credstash==1.14.0) (from versions: ) No matching distribution found for boto3>=1.1.1 (from credstash==1.14.0)

I cannot find the reason when it fails and when it succeeds.

Note that the commands above are from a xxx.sh file which is run as part of the EC2 UserData script - it works almost 90% during EC2 startup.

However, it works 10% of the time when I run the xxx.sh manually (the sh runs an ansible playbook after the credstash is installed - hence running manually is quicker way to test the playbook) while SSHing on the EC2 instance - the error switches between the 2 error messages I stated above. I am not python developer so it might be something I missing, please if you can point me to the right direction?

Answer 1

Most likely this intermittent failure can be due to slow response from your internal PYPI_REPO. To confirm it you can use -v option in your pip install command, to increase the verbosity of pip output. You can then analyze the pip output to check where the failure occurs.

You can also set a larger timeout value using --timeout 60 option in your pip command. The default timeout value is 15 seconds. This timeout value can also be set via pip configuration file (%VIRTUAL_ENV%\\pip.ini).

[global]
timeout = 60