Using Github API to Get All Members with Python

Question

I am trying to grab all the members we have in a Github Organization. We have about ~4K.

Using the documentation here , I am trying to page through the results but it not iterating through the pages of results.

Here is the Code:

from dotenv import load_dotenv, find_dotenv
import json
import requests
import os

load_dotenv(find_dotenv())

headers = {
    "authorization": f"{os.getenv('github_token')}",
    "content-type": "application/json"
}


query_url = "https://api.github.com/orgs/<name of Org>/members?page="

members = [ ]

page_no = 1

loop_control = 0
while loop_control == 0:
    url = query_url + str(page_no)

    request = requests.get(url, headers=headers)

    print(url)
    print(request.status_code)

    response = request.json()

    print(len(response))

    for i in response:
        members.append(i)

    if len(response) == 30:
        page_no += 1
    elif len(response) < 30: 
        loop_control = 1
        with open('data/github/response.json', 'w') as file:
            print(len(members))
            json.dump(members, file)

With the code, it grabbing the first 30 results, then it grabs 7 for page 2 of the results.

Any Ideas?

Answer 1

Two things to check about your script:

• ensure the token is associated with an account that is a member of the organization
• ensure your token has the read:org scope set

If one of these conditions are not met the script will only see users who have public membership for the organization, which would explain the difference in numbers you're seeing.

To also improve the script performance, you can add a per_page=100 query string parameter to get 100 results per API call, instead of the default 30. This is documented in the Pagination section of the API docs.