python 3.10.5 pip3 SSLError behind firewall

Question

I am having trouble using pip3 with python 3.10.5 at my workplace due to the network firewall blocking SSL/HTTPS traffic.

With earlier versions of python3 I can just pass --trusted-host to pip3 install and it will work like a charm. However, for some reason this doesn't seem to work with the version of python in question. I wanted to mention that every version of Python I use at work were compiled from source.

Here is the error

zpietro@corp@phpgridzlogn003:~$ pip3 install --trusted-host pypi.org cget
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/cget/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/cget/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/cget/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/cget/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/cget/
Could not fetch URL https://pypi.org/simple/cget/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/cget/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement cget (from versions: none)
ERROR: No matching distribution found for cget
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
zpietro@corp@phpgridzlogn003:~$

Unfortunately SSL/HTTPS traffic won't be allowed no matter what, and so I am a bit stuck. Any help will be much appreciated.

Thanks

Answer 1

Debugging pip3 with strace showed that libssl was missing

stat("/tools/apps/software/lang/Python/3.10.5_1/bin", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
stat("/tools/apps/software/lang/Python/3.10.5_1/lib/python3.10", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
stat("/tools/apps/software/lang/Python/3.10.5_1/lib/python3.10/lib-dynload", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
stat("/tools/apps/software/lang/Python/3.10.5_1/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=591560, ...}) = 0
open("/tools/apps/software/lang/Python/3.10.5_1/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=591560, ...}) = 0
mmap(NULL, 2297144, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2aaab6fc1000
mprotect(0x2aaab6fea000, 2097152, PROT_NONE) = 0
mmap(0x2aaab71ea000, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x29000) = 0x2aaab71ea000
close(3)                                = 0
open("/tools/apps/software/lang/Python/3.10.5_1/lib/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/share/lsf/10.1/linux3.10-glibc2.17-x86_64/lib/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=70935, ...}) = 0
mmap(NULL, 70935, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2aaaaabe7000
close(3)                                = 0
open("/lib64/tls/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Looking at lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so confirmed OpenSSL was missing.

zpietro@corp@phpgridzlogn003:~$ ldd /tools/apps/software/lang/Python/3.10.5_1/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so
    linux-vdso.so.1 =>  (0x00002aaaaaacd000)
    libssl.so.1.1 => not found
    libcrypto.so.1.1 => not found
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaaaf00000)
    libc.so.6 => /lib64/libc.so.6 (0x00002aaaab11c000)
    /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)
zpietro@corp@phpgridzlogn003:~$

In a normal scenario installing OpenSSL should get pip3 working. But because we use environment modules, that meant I had to add the OpenSSL environment module as dependency of the Python 3.10.5 env module.

Thanks