PANDAS - Import data from GCP

Question

I am trying to import data from GCP but get the error below.

Code:

import pandas as pd
from pandas.io import gbq
DF = """SELECT * FROM `im-test-209122.PPM_UAT.SAP_PS_CONSOLIDATED` LIMIT 1000"""
DF = gbq.read_gbq(DF,project_id="im-test-209122")
DF.head(3)

Error:

ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Answer 1

Have you provided the BigQuery authentication to pandas_gbq library? I used a service account to provide the authentication following the info in this link . You have to download the json key and provide the file in the code using the google.oauth2 library. With the following code I could query data from my BigQuery project.

    from pandas.io import gbq
    from google.oauth2 import service_account

    sql = "SELECT * FROM PROJECT_ID.DATASET.TABLE"

    credentials = service_account.Credentials.from_service_account_file('path-to-key-file.json')
    df = gbq.read_gbq(sql, project_id="PROJECT_ID", credentials=credentials)

    df.info()
    print(df)

I'm not sure that this faces the issue you are getting because I couldn't get the same error as you, however I hope it will help.