Connect BigQuery to Python

Question

I have installed Google BigQuery and I'm trying to query from BigQuery inside of Python.

from google.cloud import bigquery 
client = bigquery.Client(project='aaa18')

But I'm getting this error and I don't know what this means :

C:\\Users\\udgtlvr\\AppData\\Local\\Continuum\\anaconda3\\python.exe C:/Users/udgtlvr/untitled4/bigquery

Traceback (most recent call last):

  File "C:/Users/udgtlvr/untitled4/bigquery", line 2, in <module>

    from google.cloud import bigquery

  File "C:\Users\udgtlvr\AppData\Local\Continuum\anaconda3\lib\site-packages\google\cloud\bigquery\__init__.py", line 35, in <module>

    from google.cloud.bigquery.client import Client

  File "C:\Users\udgtlvr\AppData\Local\Continuum\anaconda3\lib\site-packages\google\cloud\bigquery\client.py", line 43, in <module>

    from google import resumable_media

ImportError: cannot import name 'resumable_media' from 'google' (unknown location)

What are the steps I need to do in order to get this thing work?

Answer 1

Besides what @Hitobat mentions in his comment (reinstalling the google-cloud-bigquery module), you could also try only installing the module that is failing, the module google-resumable-media .

To verify which module is missing or might be outdated, you can run the following command:

pip freeze | grep google

This will list all the google's modules you have installed.

Answer 2

Just to make sure you did install the required official packages, you should have used this source first https://googleapis.dev/python/bigquery/latest/index.html

They specify the install considering you use a virtual environment

Windows
pip install virtualenv
virtualenv <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install google-cloud-bigquery

If you did right, installing the Google Cloud Python library usually installs the required dependencies (in your case the google-resumable-media)

pip install google-cloud

As I see your problem is more than an Anaconda/Local Python paths problem... So you may first verify where the Python Client for Google BigQuery is being installed.

If you definitely want to work with Anaconda environments, then try to install the packages from the Anaconda Navigator environments manager... That should solve the dependencies problem.

Answer 3

Create a service account with desired BigQuery roles and download JSON key file (example: data-lab.json). Use below code:

import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = "data-lab.json"
from google.cloud import bigquery
client = bigquery.Client()

Answer 4

This is what I do and works fine . I use Colab/Jupyter Notebooks to access Bigquery.

First Option is to authenticate using browser

from google.colab import auth
auth.authenticate_user()

The second option is to use service account for authentication which is explained above

Once you are done with authentication then the below code should work

from google.cloud import bigquery

client = bigquery.Client(project='project_id')
%load_ext google.cloud.bigquery

Run Bigquery commands using magic link

%%bigquery --project project_id
select * from table limit 1