error 403: Your client does not have permission to get URL in python google cloud module

Question

UPTADE: According to what our system expert told me, they fixed the problem as follows: v6 ips must be disabled in the operating systems they are in.

I have a python file on the server that is scheduled to run every week. like this:

from google.cloud import bigquery
gbq_credentials = create_gc_credentials(settings)
client = bigquery.Client(credentials=gbq_credentials,project=project_id)
df = client.query(sql_query).to_dataframe()

#data processing
....

But the file that works every week, gave an error today:

Error 403 (Forbidden)!!1
Your client does not have permission to get URL <code>/bigquery/v2/projects/xxxx/jobs</code> from this server.

when I run the same file on my local computer, I didn't get any errors.

I updated the outdated python libraries on the server . But it didn't work. Also I get the same error on all files using the google cloud module.

What could this problem be caused by?

Answer 1

As Bushmaster commented, disabling IPV6 solved the issue.

I'm writing down the steps for disabling it:

sudo echo "net.ipv6.conf.all.disable_ipv6=1" >> /etc/sysctl.conf
sudo echo "net.ipv6.conf.default.disable_ipv6=1" >> /etc/sysctl.conf
sudo echo "net.ipv6.conf.lo.disable_ipv6=1" >> /etc/sysctl.conf
sudo sysctl -p

Hope it helps!

Answer 2

How does the "create_gc_credentials" work? From the error message, the service account running on the server does not have the permission.

Answer 3

It could be for many reasons. You can consider these options:

• This error can be caused due to cache, and erasing the cache sometimes might help to fix it.
• You may just need to restart the server.
• You are not allowed to access the resource, or there's an error on the server side.
• The owners of the web server have improperly set up permissions, and you're getting denied access when you really shouldn't be.

Or

When using the BigQuery API, you need to create the client with OAuth credentials for the user. For access with an API, this is often a Service Account identity. When you create a Service Account, that account is not automatically added a membership role to your project. To update the users and service accounts that are members of your project, go to your project, select "Permissions" in the navigation panel, and make sure the user or service account identity you are calling with is a "Reader" on the project.

In case it is about permissions, you can see this documentation about it.

Answer 4

(this is supposed to be a comment, but I don't have enough reputation) Can you share how you confirmed that server has been banned by google and how did you solve the problem?

We have the same issue. On our server we receive "Error 403: Your client does not have permission to get URL", but locally requests work just fine. We tried moving to another server and everything worked at first, but now we are getting an error again. There are no errors or messages on google console and we only have basic support (= can't contact google directly). Do we have to buy a support plan?

Answer 5

Most times, the error is related to whether a user agent's identity can be verified and/or the user agent has the required permission to access a protected service.

I got this error after sending a post request to a cloud function without the authorisation header .

As an example, if a cloud function was created with Allow unauthenticated switched on, then the request goes through without this error.

A quick test was to generate a bearer token by running gcloud auth print-identity-token in cloud shell, and passing the generated token in the authorisation request header. No error!