how to access google spreadsheet json file from s3 while using aws lambda with django

Question

I am using django and deployed my application on aws_lambda . Everything worked fine until i wanted to save the content of the database in a google spreadsheet

The problem is how to access/get the json.file (that would normally be located in the same folder as where i am using it) now that i am using aws_lambda in production

views.py

# how i would normally do it, locally

scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]

credentials = ServiceAccountCredentials.from_json_keyfile_name("secret-json-file.json", scope)

gc = gspread.authorize(credentials)

open_google_spreadsheet = gc.open('data').sheet1

but since i am using aws_lambda and i stored that json file on the main folder of my aws s3 bucket.

I am trying something like this:

s3_client = boto3.client("s3")

response = s3_client.get_object(Bucket=aws_bucket, Key="secret-json-file.json")

data = response["Body"].read()

# Google spreadsheet

scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]

credentials = ServiceAccountCredentials.from_json_keyfile_name(data, scope)

I also tried this approach:

AWS_ID = aws_access_key_id
AWS_KEY = aws_secret_access_key
AWS_S3_BUCKET = aws_bucket

session = boto3.session.Session(aws_access_key_id=AWS_ID, aws_secret_access_key=AWS_KEY) 

s3 = session.resource("s3")

my_s3_bucket = s3.Bucket(AWS_S3_BUCKET)

current_s3_file = []

for s3_file in my_s3_bucket.objects.all():
  if s3_file.key == "secret-json-file.json":
    current_s3_file.append(s3_file)

# Google spreadsheet

scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]

credentials = ServiceAccountCredentials.from_json_keyfile_name(current_s3_file, scope)

Unfortunately both approaches are not successful since i am not able to run the command zappa update production anymore, it crash with timeout

Below is the ouput:

Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 504 response code.

Any help would be much appreciated.

Answer 1

import gspread

credentials = { "type": "service_account", "project_id": "api-project-XXX", "private_key_id": "2cd … ba4", "private_key": "-----BEGIN PRIVATE KEY-----\nNrDyLw … jINQh/9\n-----END PRIVATE KEY-----\n", "client_email": "473000000000-yoursisdifferent@developer.gserviceaccount.com", "client_id": "473 … hd.apps.googleusercontent.com",... }

gc = gspread.service_account_from_dict(credentials)

sh = gc.open("Example spreadsheet")

wks=sh.worksheet("Example worksheet")

I saw this solution in another publication

Python - Make gspread.service_account take a python dictionary or string instead of file name