Python：Ssl 证书验证失败

Question

I have installed dvc on my ubuntu-18.04-LTS system and while trying to download the data files from github using dvc, it fails with below error.

$ dvc get https://github.com/iterative/dataset-registry get-started/data.xml -o data/data.xml -v

2022-07-22 12:55:22,260 DEBUG: Creating external repo https://github.com/iterative/dataset-registry@None
2022-07-22 12:55:22,260 DEBUG: erepo: git clone 'https://github.com/iterative/dataset-registry' to a temporary dir
2022-07-22 12:55:23,683 DEBUG: Removing '/dvc/dvc_test/data/.UEeAzwmJCY3q85YQuCeahx'
2022-07-22 12:55:23,684 ERROR: failed to get 'get-started/data.xml' from 'https://github.com/iterative/dataset-registry' - Failed to clone repo 'https://github.com/iterative/dataset-registry' to '/tmp/tmpvmrmu9qsdvc-clone'
------------------------------------------------------------
Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 703, in urlopen
  File "urllib3/connectionpool.py", line 386, in _make_request
  File "urllib3/connectionpool.py", line 1042, in _validate_conn
  File "urllib3/connection.py", line 414, in connect
  File "urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
  File "urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
  File "ssl.py", line 500, in wrap_socket
  File "ssl.py", line 1040, in _create
  File "ssl.py", line 1309, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dvc/scm.py", line 145, in clone
  File "scmrepo/git/__init__.py", line 143, in clone
  File "scmrepo/git/backend/dulwich/__init__.py", line 199, in clone
scmrepo.exceptions.CloneError: Failed to clone repo 'https://github.com/iterative/dataset-registry' to '/tmp/tmpvmrmu9qsdvc-clone'

Already our corporate proxy certificate has been installed and traffic to github.com allowed I'm able to clone above repository separately on CLI. But with dvc the above errors are occurring, Even the below couldn't solve the issue.

$ python -c "import ssl; print(ssl.get_default_verify_paths())"

DefaultVerifyPaths(cafile=None, capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')

export SSL_CERT_DIR=/etc/ssl/certs/
export REQUESTS_CA_BUNDLE=/usr/local/lib/python2.7/dist-packages/certifi/cacert.pem
pip install --upgrade certifi
export PYTHONHTTPSVERIFY=0

sudo apt install ca-certificates
sudo update-ca-certificates --fresh

$ python --version
Python 2.7.17

$ dvc doctor
DVC version: 2.13.0 (deb)
---------------------------------
Platform: Python 3.8.3 on Linux-5.4.0-92-generic-x86_64-with-glibc2.14
Supports:
        azure (adlfs = 2022.7.0, knack = 0.9.0, azure-identity = 1.10.0),
        gdrive (pydrive2 = 1.10.1),
        gs (gcsfs = 2022.5.0),
        hdfs (fsspec = 2022.5.0, pyarrow = 8.0.0),
        webhdfs (fsspec = 2022.5.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.5.1),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.5.1),
        s3 (s3fs = 2022.5.0, boto3 = 1.21.21),
        ssh (sshfs = 2022.6.0),
        oss (ossfs = 2021.8.0),
        webdav (webdav4 = 0.9.7),
        webdavs (webdav4 = 0.9.7)

Tp bypass the ssl validation in git we have git config http.sslVerify "false" Similarly do we have option in dvc?

Further what should i update to resolve this issue?

Answer 1

The Python environment you're applying fixes to is not the one DVC is using. Your Python version is 2.7.17, while DVC is using Python 3.8.3.

It looks like you installed the Debian Linux package ; Please figure out what python binary that is using and fix the certs for that env.