What is the rate limit on Google Vault Api?

Question

I'm attempting to create an export request for all my users in my organization with Google Vault API in python

I have tried slowing my API requests with time.sleep(x). I have tried modifying the script to do one account at a time, with manual running of said script.

if not result:
        print('No matters found.')
    else:
        #error checking for user list
        if not emails:
            print("missing users list")
        else:
            for user in emails:
                req = {
                        "name":user,
                        "query":{
                            "corpus":'DRIVE',
                            "dataScope":'ALL_DATA',
                            "searchMethod":'ACCOUNT',
                            "accountInfo":{
                                "emails":[user]
                                }
                            }
                        }
            #create drive export request
                userReq = vaultservice.matters().exports().create(matterId=matters[0]['matterId'],body=req).execute()
                print ("completed request: {0}".format(user))
                time.sleep(10)

When I did an export of the entire org, it creates a gigantic zip file that's not useful because I don't know what belongs to who. So I then attempted to create individual export requests.

With current script, I see this error "Quota exceeded for quota metric 'vault.googleapis.com/export_writes' and limit 'ExportWritesPerMinutePerProject' of service 'vault.googleapis.com'"

Sometimes I get one export successfully done, sometimes two, but then it errors out. What is the rate limit for Google Vault, or how else should I accomplish my task?

Update I just got up and running on the API dashboard, and found out that the default quota limit for export writes is 20 per minute. However, I don't understand then why I am running into a limit problem. Even with a modified sleep(60) before the export create, it peaks at 20 requests and limits me. I guess I don't understand why my script is creating so many requests in a second, when I believe it should only create one.

Answer 1

It looks like Google has a problem in request measurement. They count per request sent 10 requests. So you have to stay below 2 requests/minute. For instance use a 31 seconds delay in your code.

Answer 2

This is indeed an issue with Google's API meter and the issue is easily reproduced. Using Google's own sample code and running it 3x will results in the limit of 20 ExportWritesPerMinutePerProject being reached after only 3 attempts.

Google Issue Tracker has been open for over 2 years - https://issuetracker.google.com/issues/131206384

Answer 3

This is because you have reached the limit. Export write operations are not counted as one API hit for each hit. It considers 10 export write hit when you create an export using API.So after 2 hits you use up the quota of 20 export write and for third request you will get quota exceed response You can check the the following link to check how Google counts each API hit for each Quota [1]: https://developers.google.com/vault/limits