What does this error mean and why does it happen?

Question

In this code:

import requests, pprint, re, gspread, time
from oauth2client.service_account import ServiceAccountCredentials
from datetime import datetime
import oauth2client, httplib2
from oauth2client.file import Storage



def temperature():
    scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
    storage = oauth2client.file.Storage('Singapore-Weather-84def2be176a.json')
    credentials = storage.get()
    http = httplib2.Http()
    http = credentials.authorize(http)
    credentials.refresh(http)
    gc = gspread.authorize(credentials)

    wks = gc.open('Singapore Weather').sheet1
    r = requests.get('https://api.darksky.net/forecast/b02b5107a2c9c27deaa3bc1876bcee81/1.312914,%20103.780257')
    json_object = r.json()


    regexCurrentTemp = re.compile(r'"temperature":(\d\d.\d\d)')
    moTemp = regexCurrentTemp.search(str(json_object))
    temperature = moTemp.group(1)

    regexApparentTemp = re.compile(r'"apparentTemperature":(\d\d.\d\d)')
    moApparent = regexApparentTemp.search(str(json_object))
    apparent = moApparent.group(1)

    current = json_object['currently']
    cloud = current['cloudCover']
    cloud *= 100

    timenow = datetime.now()
    wks.append_row([str(timenow), temperature, apparent, cloud])

while True:
    temperature()
    time.sleep(3597)

I am getting an error code related to one of the modules, which I do not know what it means. The error is:

Traceback (most recent call last):
  File "/Users/rosen59250/PycharmProjects/MorningWeather/main.py", line 39, in <module>
    temperature()
  File "/Users/rosen59250/PycharmProjects/MorningWeather/main.py", line 12, in temperature
    credentials = storage.get()
  File "/Users/rosen59250/EvenorOdd/lib/python3.7/site-packages/oauth2client/client.py", line 407, in get
    return self.locked_get()
  File "/Users/rosen59250/EvenorOdd/lib/python3.7/site-packages/oauth2client/file.py", line 54, in locked_get
    credentials = client.Credentials.new_from_json(content)
  File "/Users/rosen59250/EvenorOdd/lib/python3.7/site-packages/oauth2client/client.py", line 302, in new_from_json
    module_name = data['_module']
KeyError: '_module'

Why is this error code happening? Is there a way I can fix this, or is this a bug in the module?

If I can fix it, how could I?

Answer 1

there is no '_module' key in your data dict. Works for me using json authentication:

pass_url='/my/json/pass.json'
scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_name(pass_url, scope)
client = gspread.authorize(creds)

then client is able to open worksheets, append rows, etc. Hope this helps

Answer 2

you get this error cause you are not using the json format the oauth2client expects. The module documentation of new_from_json says:

class method new_from_json(json_data)[source] Utility class method to instantiate a Credentials subclass from JSON.

Expects the JSON string to have been produced by to_json().

Parameters: json_data – string or bytes, JSON from to_json().

Returns: An instance of the subclass of Credentials that was serialized with to_json().

The key here is that it expects the json object you give it to be one that it produced as a serialised instance of the class. you cant just give it any arrbitrary json object and expect it to know what to do.

When you produce the JSON serialised credential object from the modules to_json method it will set the _module and _class data

        # Add in information we will need later to reconstitute this instance.
        to_serialize['_class'] = curr_type.__name__
        to_serialize['_module'] = curr_type.__module__

the reason this is important is later when you try to do new_from_json it will execute this part of the code

        # Find and call the right classmethod from_json() to restore
        # the object.
        module_name = data['_module']

This fails in your case since the json you are using isnt json credential object produced by the module. The reson this is important is that later in the code it will need to know which service_account object to return to you

    def from_json(cls, json_data):
        # TODO(issue 388): eliminate the circularity that is the reason for
        #                  this non-top-level import.
        from oauth2client import service_account
        data = json.loads(_helpers._from_bytes(json_data))

        # We handle service_account.ServiceAccountCredentials since it is a
        # possible return type of GoogleCredentials.get_application_default()
        if (data['_module'] == 'oauth2client.service_account' and
                data['_class'] == 'ServiceAccountCredentials'):
            return service_account.ServiceAccountCredentials.from_json(data)
        elif (data['_module'] == 'oauth2client.service_account' and
                data['_class'] == '_JWTAccessCredentials'):
            return service_account._JWTAccessCredentials.from_json(data)

Long story short you cant give this any old json and expect it to work you will need to first create your credentials object in the class then call its to_json method to then serialise that object to be able to use it later to just load credentials from storage.

Answer 3

Using pydrive and oauth2client to authenticate, you can use the following:

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive    
from oauth2client.file import client


gauth = GoogleAuth()
cred = <content from `credentials.json`(string)>
gauth.credentials = client.Credentials.new_from_json(cred)
drive = GoogleDrive(gauth)

If you already have a credentials.json , there should not be a need for a client_secrets.json .