![](/img/trans.png)
[英]Snowflake python connector not working on larger data set in AWS Lambda
[英]Python AWS Lambda signal: aborted (core dumped) during snowflake connection (snowflake.connector)
我是 lambda 功能的新手...我正在尝试测试 Lambda function 以连接到雪花。 经过一番研究,我终于把我需要的所有包都压缩并添加到了一个层中。
我在 lambda function 中添加了环境变量,并为密码、用户和帐户调用它们。
在我看来,它一直在循环尝试访问雪花......
我现在得到这个:
Response:
{
"errorType": "Runtime.ExitError",
"errorMessage": "RequestId: 76342539-d44a-4732-a027-6d133d3e4ea5 Error: Runtime exited with error: signal: aborted (core dumped)"
}
这是日志:
Function Logs:
in <listcomp>
File "/opt/requests/sessions.py", line 665 in send
File "/opt/requests/sessions.py", line 530 in request
File "/opt/requests/sessions.py", line 543 in get
File "/opt/snowflake/connector/ocsp_snowflake.py", line 298 in _download_ocsp_response_cache
File "/opt/snowflake/connector/ocsp_snowflake.py", line 258 in download_cache_from_server
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1205 in _check_ocsp_response_cache_server
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1178 in _validate_certificates_sequential
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1001 in _validate
File "/opt/snowflake/connector/ocsp_snowflake.py", line 996 in validate
File "/opt/snowflake/connector/ssl_wrap_socket.py", line 408 in ssl_wrap_socket_with_ocsp
File "/opt/urllib3/connection.py", line 361 in connect
File "/opt/urllib3/connectionpool.py", line 976 in _validate_conn
File "/opt/urllib3/connectionpool.py", line 381 in _make_request
File "/opt/urllib3/connectionpool.py", line 670 in urlopen
File "/opt/requests/adapters.py", line 439 in send
File "/opt/requests/sessions.py", line 643 in send
File "/opt/requests/sessions.py", line 237 in resolve_redirects
File "/opt/requests/sessions.py", line 665 in <listcomp>
File "/opt/requests/sessions.py", line 665 in send
File "/opt/requests/sessions.py", line 530 in request
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1383 in _fetch_ocsp_response
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1112 in validate_by_direct_connection
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1183 in _validate_certificates_sequential
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1001 in _validate
File "/opt/snowflake/connector/ocsp_snowflake.py", line 996 in validate
File "/opt/snowflake/connector/ssl_wrap_socket.py", line 408 in ssl_wrap_socket_with_ocsp
File "/opt/urllib3/connection.py", line 361 in connect
File "/opt/urllib3/connectionpool.py", line 976 in _validate_conn
File "/opt/urllib3/connectionpool.py", line 381 in _make_request
File "/opt/urllib3/connectionpool.py", line 670 in urlopen
File "/opt/requests/adapters.py", line 439 in send
File "/opt/requests/sessions.py", line 643 in send
File "/opt/requests/sessions.py", line 237 in resolve_redirects
File "/opt/requests/sessions.py", line 665 in <listcomp>
File "/opt/requests/sessions.py", line 665 in send
File "/opt/requests/sessions.py", line 530 in request
File "/opt/requests/sessions.py", line 543 in get
File "/opt/snowflake/connector/ocsp_snowflake.py", line 298 in _download_ocsp_response_cache
File "/opt/snowflake/connector/ocsp_snowflake.py", line 258 in download_cache_from_server
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1205 in _check_ocsp_response_cache_server
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1178 in _validate_certificates_sequential
File "/opt/snowflake/connector/ocsp_snowflake.py", line 1001 in _validate
File "/opt/snowflake/connector/ocsp_snowflake.py", line 996 in validate
File "/opt/snowflake/connector/ssl_wrap_socket.py", line 408 in ssl_wrap_socket_with_ocsp
File "/opt/urllib3/connection.py", line 361 in connect
File "/opt/urllib3/connectionpool.py", line 976 in _validate_conn
File "/opt/urllib3/connectionpool.py", line 381 in _make_request
File "/opt/urllib3/connectionpool.py", line 670 in urlopen
File "/opt/requests/adapters.py", line 439 in send
File "/opt/requests/sessions.py", line 643 in send
File "/opt/requests/sessions.py", line 237 in resolve_redirects
File "/opt/requests/sessions.py", line 665 in <listcomp>
File "/opt/requests/sessions.py", line 665 in send
...
END RequestId: 76342539-d44a-4732-a027-6d133d3e4ea5
REPORT RequestId: 76342539-d44a-4732-a027-6d133d3e4ea5 Duration: 23310.99 ms Billed Duration: 23400 ms Memory Size: 3008 MB Max Memory Used: 284 MB Init Duration: 1838.76 ms
RequestId: 76342539-d44a-4732-a027-6d133d3e4ea5 Error: Runtime exited with error: signal: aborted (core dumped)
Runtime.ExitError
这是 python 代码:
import os
import sys
sys.path.insert(0, '/opt')
import snowflake.connector as sc
import boto3
import json
acct = os.environ['sf']
user = os.environ['user']
pw = os.environ['pw']
def lambda_handler(event, context):
sfx = sc.connect(user=user, password=pw,account=acct)
sf_c = sfx.cursor()
sf_c.close()
return {
'statuscode': 200,
'body': 'I was here!'
}
sfx.close()
任何帮助将不胜感激!
我需要为此配置 VPC 吗?
要将 Lambda 服务与 Snowflake 一起使用,您需要确保您创建的 Python package 具有所需的所有依赖项。 下面的文章逐步解释了如何将 AWS Lambda 与 Snowflake 一起使用。 https://docs.snowflake.com/en/user-guide/data-load-snowpipe-rest-lambda.html
当您在 AWS 账户中启动 Lambda 服务时,默认情况下它会在默认 VPC 中启动。 如果默认 VPC 没有公共 Internet 连接,则连接可能会失败。 因此,请确保您使用可以访问您的雪花公共 URL 的 VPC。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.