无法通过云运行连接到云 sql 私有 IP

Question

我正在尝试通过简单的 python 应用程序代码连接到cloud sql(postgres) 。我正在尝试使用cloud shell在本地运行容器。一段时间后出现以下消息，然后最终出现错误。

vikrant@cloudshell:~/test/docker (test-project)$ docker run -ti --name=my-test-container test-docker-image
2022/06/12 03:56:09 current FDs rlimit set to 1048576, wanted limit is 8500. Nothing to do here.
2022/06/12 03:56:09 using credential file for authentication; email=service-gcf-transform@test-project.iam.gserviceaccount.com
2022/06/12 03:56:10 Listening on 127.0.0.1:5432 for test-project:europe-west1:test-uat-pgsql-65d01f80
2022/06/12 03:56:10 Ready for new connections
2022/06/12 03:56:11 Generated RSA key in 286.419278ms
connecting to postgres database
2022/06/12 03:56:19 New connection for "test-project:europe-west1:test-uat-pgsql-65d01f80"
2022/06/12 03:56:19 refreshing ephemeral certificate for instance test-projec:europe-west1:test-uat-pgsql-65d01f80
2022/06/12 03:56:22 Scheduling refresh of ephemeral certificate in 54m57s

  File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

下面是示例代码。 我也用云 sql IP 试过了..

import sqlalchemy
import os

db_user='test-user'
db_pass='test-pswd'
db_name='test-database'

db_engine = None

host = '127.0.0.1'
port ='5432'

drivername='postgresql+psycopg2'

def _create_engine():

    return sqlalchemy.create_engine(
        sqlalchemy.engine.url.URL(
            drivername=drivername,
            host=host,
            username=db_user,
            password=db_pass,
            database=db_name,
        )
    )


def get_engine():
    global db_engine
    if not db_engine:
        db_engine = _create_engine()
    return db_engine


def read_postgres(conn):
    sql = f"""
    select bucket_id,object_id from test_requests limit 5
    """
    result = conn.execute(sqlalchemy.text(sql)).fetchall()
    result_list=[]
    for r in result:
        d = dict(zip(r.keys(), r))
    result_list.append(d)

    return result_list

def connect_postgres():
    print('connecting to postgres database')
    with get_engine().connect() as conn:
        print('connected to postgres database')
        result = read_postgres(conn)
        result_list=[]
        for r in result:
            print(f"row data as:{r}")
            print(f"bucket name:{r['bucket_id']}")
            print(f"object name:{r['object_id']}")

connect_postgres()

gcloud run deploy run-postgres \
--image eu.gcr.io/test-project/test-docker-image \
--add-cloudsql-instances=test-project:europe-west1:test-uat-pgsql-65d01f80 \
--region=europe-west1 \
--service-account service-cce-transform@test-project.iam.gserviceaccount.com \
--vpc-connector test-uat-cf-connector

在本地运行它也不起作用

Dockerfile：

FROM python:3-alpine

WORKDIR usr/src/app
COPY main.py .
COPY requirements.txt .
COPY test_cf_SA_key.json .

ADD start.sh /
RUN chmod +x /start.sh

RUN wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy
RUN chmod +x cloud_sql_proxy

RUN cp ./cloud_sql_proxy /cloud_sql_proxy

RUN pip install --no-cache-dir -r requirements.txt

CMD ["/start.sh"]

使用 start.sh 运行应用程序代码以解决计时问题.. 阅读一些 stackoverflow 链接

#!/bin/sh

./cloud_sql_proxy -instances=test-project:europe-west1:test-uat-pgsql-65d01f80=tcp:5432 -credential_file=test_cf_SA_key.json &

chmod +x cloud_sql_proxy
sleep 10
python3 main.py

Answer 1

您无法连接到 Cloud SQL 实例的主要原因是您使用 Cloud SQL 身份验证代理的方式。 如果您从与私有 SQL 实例不在同一 VPC 内的环境中运行代理，则该代理将无法连接到私有实例。 您的本地计算机和Cloud Shell 都不属于您的 SQL 实例所在的 VPC。

例如，您可以对其进行测试的环境是与您的 SQL 实例位于同一 VPC 的 GCE 实例。 我测试了您的代码的简化版本，并且可以从 GCE 和 Cloud Run 服务查询实例。 以下是我使用的片段。

但是，使用 Cloud Run 容器中的身份验证代理连接到私有 CloudSQL 实例是多余的。 官方指南显示了从 Cloud Run 查询私有 Cloud SQL 实例的推荐设置。 无服务器 VPC 连接器将允许您的 Cloud Run 容器与您的 SQL 实例位于同一 VPC 中，并直接连接到它。 让我知道这是否有用。

Dockerfile

FROM python:3-alpine

WORKDIR /app
COPY main.py .
COPY requirements.txt .
COPY test_service_acctkey.json .

COPY start.sh .
RUN chmod +x ./start.sh

RUN wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy
RUN chmod +x cloud_sql_proxy

RUN pip install --no-cache-dir -r requirements.txt

CMD ["./start.sh"]

启动.sh

#!/bin/sh

./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:5432 -credential_file=<PATH_TO_SERVICE_ACCTKEY> -ip_address_types=PRIVATE &

chmod +x cloud_sql_proxy
sleep 10
gunicorn --bind :8080 --workers 1 --threads 8 --timeout 0 main:app

主文件

from flask import Flask
import sqlalchemy
import os

db_user='USER'
db_pass='PASS'
db_name='DATABASE'
db_engine = None

host = '127.0.0.1'
port ='5432'

drivername='postgresql+psycopg2'

app = Flask(__name__)

def _create_engine():

    return sqlalchemy.create_engine(
        sqlalchemy.engine.url.URL(
            drivername=drivername,
            host=host,
            username=db_user,
            password=db_pass,
            database=db_name,
        )
    )


def get_engine():
    global db_engine
    if not db_engine:
        db_engine = _create_engine()
    return db_engine


def read_postgres(conn):
    sql = f"""
    select * from testtable;
    """
    result = conn.execute(sqlalchemy.text(sql)).fetchall()
    return result

def connect_postgres():
    with get_engine().connect() as conn:
        result = read_postgres(conn)
        return result[0][1]

@app.route('/')
def entry():
    sql_data = connect_postgres()
    return f'TEST: {sql_data}'

if __name__ == '__main__':
    app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))

部署命令（用于测试）

gcloud run deploy SERVICE_NAME \
--source . \
--service-account=SERVICE_ACCT_ADDRESS \
--max-instances=3 \
--allow-unauthenticated \
--region=us-central1 \
--vpc-connector=SERVERLESS_CONNECTOR_NAME \
--no-cpu-throttling