[英]Is it possible to use pip to install a package from a private GitHub repository?
I am trying to install a Python package from a private GitHub repository.我正在尝试从私有 GitHub 存储库安装 Python 包。 For a public repository, I can issue the following command which works fine:
对于公共存储库,我可以发出以下运行良好的命令:
pip install git+git://github.com/django/django.git
However, if I try this for a private repository:但是,如果我尝试将其用于私有存储库:
pip install git+git://github.com/echweb/echweb-utils.git
I get the following output:我得到以下输出:
Downloading/unpacking git+git://github.com/echweb/echweb-utils.git
Cloning Git repository git://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build
Complete output from command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build:
fatal: The remote end hung up unexpectedly
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build...
----------------------------------------
Command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build failed with error code 128
I guess this is because I am trying to access a private repository without providing any authentication.我想这是因为我试图在不提供任何身份验证的情况下访问私有存储库。 I therefore tried to use Git +
ssh
hoping that pip would use my SSH public key to authenticate:因此,我尝试使用 Git +
ssh
希望 pip 使用我的 SSH 公钥进行身份验证:
pip install git+ssh://github.com/echweb/echweb-utils.git
This gives the following output:这给出了以下输出:
Downloading/unpacking git+ssh://github.com/echweb/echweb-utils.git
Cloning Git repository ssh://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build
Complete output from command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build:
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build...
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
----------------------------------------
Command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build failed with error code 128
Is what I am trying to achieve even possible?我正在努力实现的目标是可能的吗? If so, how can I do it?
如果是这样,我该怎么做?
You can use the git+ssh
URI scheme, but you must set a username.您可以使用
git+ssh
URI 方案,但您必须设置用户名。 Notice the git@
part in the URI:注意 URI 中的
git@
部分:
pip install git+ssh://git@github.com/echweb/echweb-utils.git
Also read about deploy keys .另请阅读部署密钥。
PS: In my installation, the "git+ssh" URI scheme works only with "editable" requirements: PS:在我的安装中,“git+ssh”URI 方案仅适用于“可编辑”要求:
pip install -e URI#egg=EggName
Remember : Change the :
character that git remote -v
prints to a /
character before using the remote's address in the pip
command:请记住:在
pip
命令中使用远程地址之前,将git remote -v
打印的:
字符更改为/
字符:
$ git remote -v
origin git@github.com:echweb/echweb-utils.git (fetch)
# ^ change this to a '/' character
If you forget, you will get this error:如果你忘记了,你会得到这个错误:
ssh: Could not resolve hostname github.com:echweb:
nodename nor servname provided, or not known
As an additional technique, if you have the private repository cloned locally, you can do:作为一项附加技术,如果您在本地克隆了私有存储库,您可以执行以下操作:
pip install git+file://c:/repo/directory
More modernly, you can just do this (and the -e
will mean you don't have to commit changes before they're reflected):更现代的是,您可以这样做(并且
-e
意味着您不必在更改反映之前提交更改):
pip install -e C:\repo\directory
I found it much easier to use tokens than SSH keys.我发现使用令牌比使用 SSH 密钥容易得多。 I couldn't find much good documentation on this, so I came across this solution mainly through trial and error.
我在这方面找不到太多好的文档,所以我主要通过反复试验找到了这个解决方案。 Further, installing from pip and setuptools have some subtle differences;
此外,从 pip 和 setuptools 安装有一些细微的差别; but this way should work for both.
但这种方式应该适用于两者。
GitHub don't (currently, as of August 2016) offer an easy way to get the zip / tarball of private repositories. GitHub(目前,截至 2016 年 8 月)不提供获取私有存储库的 zip/tarball 的简单方法。 So you need to tell setuptools that you're pointing to a Git repository:
所以你需要告诉 setuptools 你指向一个 Git 仓库:
from setuptools import setup
import os
# Get the deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']
setup(
# ...
install_requires='package',
dependency_links = [
'git+https://{github_token}@github.com/user/{package}.git/@{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]
A couple of notes here:这里有几点注意事项:
0
) at the end of the link, even if there's isn't any package on PyPI .0
),即使PyPI上没有任何包。 This has to be a actual number, not a word.git+
to tell setuptools it's to clone the repository, rather than pointing at a zip / tarballgit+
开头来告诉 setuptools 它是克隆存储库,而不是指向 zip / tarballversion
can be a branch, a tag, or a commit hash version
可以是分支、标签或提交哈希--process-dependency-links
if installing from pip--process-dependency-links
I figured out a way to automagically 'pip install' a GitLab private repository that requires no password prompt.我想出了一种自动“pip install”一个不需要密码提示的 GitLab 私有存储库的方法。 This approach uses GitLab "Deploy Keys" and an SSH configuration file, so you can deploy using keys other than your personal SSH keys (in my case, for use by a 'bot).
这种方法使用 GitLab“部署密钥”和 SSH 配置文件,因此您可以使用个人 SSH 密钥以外的密钥进行部署(在我的情况下,供“机器人”使用)。 Perhaps someone kind soul can verify using GitHub.
也许有好心人可以使用 GitHub 进行验证。
ssh-keygen -t rsa -C "GitLab_Robot_Deploy_Key"
The file should show up as ~/.ssh/GitLab_Robot_Deploy_Key
and ~/.ssh/GitLab_Robot_Deploy_Key.pub
.该文件应显示为
~/.ssh/GitLab_Robot_Deploy_Key
和~/.ssh/GitLab_Robot_Deploy_Key.pub
。
Copy and paste the contents of the ~/.ssh/GitLab_Robot_Deploy_Key.pub
file into the GitLab "Deploy Keys" dialog.将
~/.ssh/GitLab_Robot_Deploy_Key.pub
文件的内容复制并粘贴到 GitLab“部署密钥”对话框中。
The following command tells SSH to use your new deploy key to set up the connection.以下命令告诉 SSH 使用您的新部署密钥来设置连接。 On success, you should get the message: "Welcome to GitLab, UserName!"
成功后,您应该会收到消息:“欢迎来到 GitLab,用户名!”
ssh -T -i ~/.ssh/GitLab_Robot_Deploy_Key git@gitlab.mycorp.com
Next, use an editor to create a ~/.ssh/config
file.接下来,使用编辑器创建一个
~/.ssh/config
文件。 Add the following contents.添加以下内容。 The 'Host' value can be anything you want (just remember it, because you'll be using it later).
'Host' 值可以是您想要的任何值(记住它,因为稍后您将使用它)。 The HostName is the URL to your GitLab instance.
HostName 是 GitLab 实例的 URL。 The IdentifyFile is path to the SSH key file you created in the first step.
identifyFile 是您在第一步中创建的 SSH 密钥文件的路径。
Host GitLab
HostName gitlab.mycorp.com
IdentityFile ~/.ssh/GitLab_Robot_Deploy_Key
oxyum gave us the recipe for using pip with SSH: oxyum 为我们提供了在 SSH 中使用 pip 的秘诀:
pip install git+ssh://git@gitlab.mycorp.com/my_name/my_repo.git
We just need to modify it a bit to make SSH use our new Deploy Key.我们只需要稍微修改一下,让 SSH 使用我们的新部署密钥。 We do that by pointing SSH to the Host entry in the SSH configuration file.
我们通过将 SSH 指向 SSH 配置文件中的主机条目来做到这一点。 Just replace the 'gitlab.mycorp.com' in the command to the host name we used in the SSH configuration file:
只需将命令中的 'gitlab.mycorp.com' 替换为我们在 SSH 配置文件中使用的主机名即可:
pip install git+ssh://git@GitLab/my_name/my_repo.git
The package should now install without any password prompt.该软件包现在应该在没有任何密码提示的情况下安装。
If you want to install dependencies from a requirements file within a CI server or alike, you can do this:如果你想从CI服务器或类似的需求文件中安装依赖项,你可以这样做:
git config --global credential.helper 'cache'
echo "protocol=https
host=example.com
username=${GIT_USER}
password=${GIT_PASS}
" | git credential approve
pip install -r requirements.txt
In my case, I used GIT_USER=gitlab-ci-token
and GIT_PASS=${CI_JOB_TOKEN}
.就我而言,我使用
GIT_USER=gitlab-ci-token
和GIT_PASS=${CI_JOB_TOKEN}
。
This method has a clear advantage.这种方法有一个明显的优势。 You have a single requirements file which contains all of your dependencies.
您有一个包含所有依赖项的需求文件。
The syntax for the requirements file is given here:需求文件的语法在这里给出:
https://pip.pypa.io/en/latest/reference/pip_install.html#requirements-file-format https://pip.pypa.io/en/latest/reference/pip_install.html#requirements-file-format
So for example, use:例如,使用:
-e git+http://github.com/rwillmer/django-behave#egg=django-behave
if you want the source to stick around after installation.如果您希望源在安装后保留。
Or just要不就
git+http://github.com/rwillmer/django-behave#egg=django-behave
if you just want it to be installed.如果您只想安装它。
If you need to do this in, say, a command line one-liner, it's also possible.如果您需要在命令行单行中执行此操作,它也是可能的。 I was able to do this for deployment on Google Colab:
我能够在 Google Colab 上进行部署:
pip install git+https://<PERSONAL ACCESS TOKEN>@github.com/<USERNAME>/<REPOSITORY>.git
pip install git+https://<PERSONAL ACCESS TOKEN>@github.com/<USERNAME>/<REPOSITORY>.git
You can also install a private repository dependency via git+https://github.com/... URL by providing login credentials (login and password, or deploy token) for curl with the .netrc
file:您还可以通过git+https://github.com/... URL 安装私有存储库依赖项,方法是使用
.netrc
文件为curl提供登录凭据(登录名和密码,或部署令牌):
echo "machine github.com login ei-grad password mypasswordshouldbehere" > ~/.netrc
pip install "git+https://github.com/ei-grad/my_private_repo.git#egg=my_private_repo"
My case was kind of more complicated than most of the ones described in the answers.我的情况比答案中描述的大多数情况都复杂。 I was the owner of two private repositories
repo_A
and repo_B
in a Github organization and needed to pip install repo_A
during the python
unittests of repo_B
, as a Github action .我是 Github 组织中两个私有存储库
repo_A
和repo_B
的所有者,并且需要在 repo_B 的python
单元测试期间pip install repo_A
repo_B
Github 操作。
Steps I followed to solve this task:我为解决此任务而遵循的步骤:
repo_B
, pasted my Personal Access Token in there and named it PERSONAL_ACCESS_TOKEN
.repo_B
下创建了一个存储库机密,将我的个人访问令牌粘贴在那里并将其命名为PERSONAL_ACCESS_TOKEN
。 This was important because, unlike the solution proposed by Jamie , I didn't need to explicitly expose my precious raw Personal Access Token inside the github action .yml
file..yml
文件中显式公开我宝贵的原始个人访问令牌。pip install
the package from source via HTTPS ( pip install
包,如下所示:export PERSONAL_ACCESS_TOKEN=${{ secrets.PERSONAL_ACCESS_TOKEN }}
pip install git+https://${PERSONAL_ACCESS_TOKEN}@github.com/MY_ORG_NAME/repo_A.git
If you don't want to use SSH, you could add the username and password in the HTTPS URL.如果不想使用 SSH,可以在 HTTPS URL 中添加用户名和密码。
The code below assumes that you have a file called "pass" in the working directory that contains your password.下面的代码假定您在工作目录中有一个名为“pass”的文件,其中包含您的密码。
export PASS=$(cat pass)
pip install git+https://<username>:$PASS@github.com/echweb/echweb-utils.git
When I was installing from GitHub I was able to use:当我从 GitHub 安装时,我可以使用:
pip install git+ssh://git@github.com/<username>/<projectname>.git#egg=<eggname>
But, since I had to run pip as sudo
, the SSH keys were not working with GitHub any more, and "git clone" failed on "Permission denied (publickey)".但是,由于我必须将 pip 作为
sudo
运行,SSH 密钥不再适用于 GitHub,并且“git clone”在“Permission denied (publickey)”上失败。 Using git+https
allowed me to run the command as sudo, and have GitHub ask me for my user/password.使用
git+https
可以让我以 sudo 身份运行命令,并让 GitHub 询问我的用户名/密码。
sudo pip install git+https://github.com/<username>/<projectname>.git#egg=<eggname>
If you have your own library/package on GitHub, GitLab, etc., you have to add a tag to commit with a concrete version of the library, for example, v2.0, and then you can install your package:如果你在 GitHub、GitLab 等上有自己的库/包,你必须添加一个标签来提交库的具体版本,例如 v2.0,然后你可以安装你的包:
pip install git+ssh://link/name/repo.git@v2.0
This works for me.这对我有用。 Other solutions haven't worked for me.
其他解决方案对我不起作用。
oxyum's solution is OK for this answer. oxyum 的解决方案对于这个答案是可以的。 I just want to point out that you need to be careful if you are installing using
sudo
as the keys must be stored for root too (for example, /root/.ssh
).我只想指出,如果您使用
sudo
进行安装,则需要小心,因为密钥也必须为 root 存储(例如, /root/.ssh
)。
Then you can type然后你可以输入
sudo pip install git+ssh://git@github.com/echweb/echweb-utils.git
Just copy the remote from the original git clone
command (or from git remote -v
).只需从原始
git clone
命令(或从git remote -v
)复制远程。 You will get something like this:你会得到这样的东西:
Bitbucket: git+ssh://git@bitbucket.org:your_account/my_pro.git
比特桶:
git+ssh://git@bitbucket.org:your_account/my_pro.git
GitHub: git+ssh://git@github.com:your_account/my_pro.git
GitHub:
git+ssh://git@github.com:your_account/my_pro.git
Next, you need to replace :
with /
next to the domain name.接下来,您需要将
:
替换为域名旁边的/
。
So install using:所以安装使用:
pip install git+ssh://git@bitbucket.org/your_account/my_pro.git
I am trying to install a Python package from a private GitHub repository.我正在尝试从私有GitHub存储库安装Python软件包。 For a public repository, I can issue the following command which works fine:
对于公共存储库,我可以发出以下正常运行的命令:
pip install git+git://github.com/django/django.git
However, if I try this for a private repository:但是,如果我尝试将其用于私有存储库:
pip install git+git://github.com/echweb/echweb-utils.git
I get the following output:我得到以下输出:
Downloading/unpacking git+git://github.com/echweb/echweb-utils.git
Cloning Git repository git://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build
Complete output from command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build:
fatal: The remote end hung up unexpectedly
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build...
----------------------------------------
Command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build failed with error code 128
I guess this is because I am trying to access a private repository without providing any authentication.我猜这是因为我试图在不提供任何身份验证的情况下访问私有存储库。 I therefore tried to use Git +
ssh
hoping that pip would use my SSH public key to authenticate:因此,我尝试使用Git +
ssh
希望pip使用我的SSH公钥进行身份验证:
pip install git+ssh://github.com/echweb/echweb-utils.git
This gives the following output:这给出了以下输出:
Downloading/unpacking git+ssh://github.com/echweb/echweb-utils.git
Cloning Git repository ssh://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build
Complete output from command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build:
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build...
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
----------------------------------------
Command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build failed with error code 128
Is what I am trying to achieve even possible?我正在努力实现的目标是否可能? If so, how can I do it?
如果是这样,我该怎么办?
You may try你可以试试
pip install git+git@gitlab.mycorp.com/my_name/my_repo.git
without ssh:...
.没有
ssh:...
That works for me.这对我行得通。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.