BigQuery Client using Python | Timeout and Polling issues

Question

I am trying to execute a SQL statement on BQ Database by initializing the BQ Client. This has been running smoothly for a while but lately seeing an issue.

My code specifically fails when its trying to wait for the results from the DB

query_job = client.query(QUERY)  # API request to start the query    
db_rslt = query_job.result()     # Wait for the query to return results

Here is error message:

File "/opt/conda/default/lib/python3.7/site-packages/google/cloud/bigquery/job.py", line xxx, in result super(QueryJob, self).result(retry=retry, timeout=timeout) File "/opt/conda/default/lib/python3.7/site-packages/google/cloud/bigquery/job.py", line xxx, in result return super(_AsyncJob, self).result(timeout=timeout) File "/opt/conda/default/lib/python3.7/site-packages/google/api_core/future/polling.py", line xxx, in result self._blocking_poll(timeout=timeout, retry=retry, polling=polling) TypeError: _blocking_poll() got an unexpected keyword argument 'retry'

Added the timeout parameter to the result method but did not help

Answer 1

It depends on the version of your installed google-cloud-bigquery library, if you are using a 1.x version, make sure to set google-api-core to a version less than 1.34.0.

What is actually happening here is that when you install google-cloud-bigquery, a bunch of other google related libraries are installed automatically with it, google-api-core is one of these libraries. However, a higher version is being installed thus causing this problem.

In my case, setting my google-api-core==1.33.2 solves the issue. Other solution would be is to upgrade google-cloud-bigquery to a later version.

Answer 2

I solved it by changing the bigquery version to the latest: 'google-cloud-bigquery==3.4.0'.

Answer 3

7 days ago Google released an update to google-api-core==1.34.0 for bug fixes. I guess this introduced some breaking changes if you are not using the later versions of google-cloud-bigquery . You may either:

1. Pin google-api-core to the previous working version (eg. 1.33.2)
2. Upgrade google-cloud-bigquery to the latest versions.

Answer 4

This also happened to me. My python image is 3.8.13:bulleye . It was using the following stack of google library.

google-auth==1.33.0
google-cloud-bigquery==1.25.0
google-cloud-firestore==2.3.1

Changing them to

db-dtypes==1.0.0
google-auth==2.6.0
google-cloud-bigquery==3.0.1
google-cloud-firestore==2.5.3

solved the problems.

db-dtypes is required for new version of google cloud library.