How can I read the projects list from a GitLab group with GitPython?

Question

I am trying to find a way to clone all the git repositories from a remote GitLab group with Python ( gitPython for example).

Can you help me, please?

Answer 1

You should install python-gitlab (to iterate over the GitLab project structures) and GitPython (to clone the Git repos) packages first.

If you have a self-hosted GitLab instance, then you also need to get a personal token. Go to https://gitlab.your-company.com/-/profile/personal_access_tokens to access the token interface, like on the screenshot:

Save the obtained token (step 1).

You also need a deploy token for each repository that you are going to clone. To do so, go to https://gitlab.your-company.com/your_group/your_project/-/settings/repository -> "Deploy tokens", like on the screenshot below:

You'll get a token username (like gitlab+deploy-token-1488 ) and the token itself. Save this username/token pair (step 2) and repeat this step for each repo.

As the last step, you should get the ID of the group which contains your repos. It should be displayed on the web interface of your group at https://gitlab.your-company.com/your_group like on the screenshot:

Save this ID too (step 3).

Now you are all set to run this code snippet:

import git
import os
import gitlab

local_cloned_repos_dir = 'local_cloned_repos_dir'

gl = gitlab.Gitlab('https://gitlab.your-company.com', private_token='insert_token_from_step_1')

deploy_token_map = {
    'your_project_name': 'gitlab+deploy-token-1488:insert_token_from_step_2'
}

for project in gl.groups.get('insert_id_from_step_3').projects.list():
    local_cloned_project_dir = os.path.join(local_cloned_repos_dir, project.name)
    os.makedirs(local_cloned_project_dir, exist_ok=True)
    git.Repo.clone_from('https://{}@gitlab.your-company.com/{}'.format(deploy_token_map[project.name], project.path_with_namespace), local_cloned_project_dir)

Note that I've created a map to match the name of the project with the corresponding token username / deploy token pair, but you don't have to handle it exactly this way.

Hope it helps!