SQLAlchemy create_engine not working in Google Cloud Function

Question

I am trying to run a Google Cloud Function that collects data off a website then inserts it into a Cloud SQL (MySQL) database, but having problems with SQLAlchemy in Cloud which don't appear on my local machine. Any suggestions!?

When I run the function locally, against Py3.7 (on a Mac, not using virtualenv), using the Cloud SQL Proxy and SQLAlchemy, I successfully connect to the database.

When running the Cloud Function, I use connection string in this format mysql+pymysql://<username>:<password>/<dbname>?unix_socket=/cloudsql/<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME> .

The Cloud Function keeps throwing the following exception for SQLAlchemy.create_engine. It does not appear to be related to being able to connect, but due to instantiation.

Everything is in the same project.

I have also tried using the public IP and connection string in the format mysql+pymysql://<username>:<password>@<public ip address>:3306/<dbname> , which made no difference.

Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 449, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 268, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 265, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 14, in retrieve_and_log
    engine = create_engine(connection_string,echo=True)
  File "/env/local/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py", line 500, in create_engine
    return strategy.create(*args, **kwargs)
  File "/env/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 56, in create
    plugins = u._instantiate_plugins(kwargs)
AttributeError: 'Context' object has no attribute '_instantiate_plugins'

Here is a snippet of my code:

import requests
from bs4 import BeautifulSoup
from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

def retrieve_and_log(store_string, connection_string = 'mysql+pymysql://<username>:<password>/<dbname>?unix_socket=/cloudsql/<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>'):

    engine = create_engine(connection_string,echo=True)
    
    conn = engine.connect()
    # ....

Answer 1

If retrieve_and_log is the function you are trying to deploy as a background Cloud Function, it needs a signature like:

def retrieve_and_log(data, context):
    ...

It can't take arbitrary parameters.

See https://cloud.google.com/functions/docs/writing/background for more details.