[英]Connect Oracle from AWS Lambda
I have a Lambda function that needs to use pandas, sqlalchemy, and cx_Oracle.我有一个 Lambda function 需要使用 pandas、ZF3D386AADEEB6063D016E4_Oracle56026
Installing and packaging all these libraries together exceeds the 250MB deployment package limit of AWS Lambda .将所有这些库安装和打包在一起超过了AWS Lambda 的 250MB 部署 package 限制。
I would like to include only the.zip of the Oracle Basic Light Package , then extract and use it at runtime.我想只包括 Z3 0162ED78B6C10F731411F2FC440C24FZ Basic Light Package的 zip ,然后在运行时提取并使用它。
What I have tried我试过的
My project is structured as follows:我的项目结构如下:
cx_Oracle-7.2.3.dist-info/
dateutil/
numpy/
pandas/
pytz/six-1.12.0.dist-info/
sqlalchemy/
SQLAlchemy-1.3.8.egg-info/
cx_Oracle.cpython-36m-x86_64-linux-hnu.so
instantclient-basiclite-linux.x64-19.3.0.0.0dbru.zip
main.py
six.py
template.yml
In main.py
, I run the following:在
main.py
中,我运行以下命令:
import json, traceback, os
import sqlalchemy as sa
import pandas as pd
def main(event, context):
try:
unzip_oracle()
return {'statusCode': 200,
'body': json.dumps(run_query()),
'headers': {'Content-Type': 'application/json', 'Access-Control-Allow-Origin':'*'}}
except:
em = traceback.format_exc()
print("Error encountered. Error is: \n" + str(em))
return {'statusCode': 500,
'body': str(em),
'headers': {'Content-Type': 'application/json', 'Access-Control-Allow-Origin':'*'}}
def unzip_oracle():
print('extracting oracle drivers and copying results to /var/task/lib')
os.system('unzip /var/task/instantclient-basiclite-linux.x64-19.3.0.0.0dbru.zip -d /tmp')
print('extraction steps complete')
os.system('export ORACLE_HOME=/tmp/instantclient_19_3')
def get_db_connection():
return sa.engine.url.URL('oracle+cx_oracle',
username='do_not_worry', password='about_any',
host='of_these', port=1521,
query=dict(service_name='details')
)
def run_query():
query_text = """SELECT * FROM dont_worry_about_it"""
conn = sa.create_engine(get_db_connection())
print('Connected')
df = pd.read_sql(sa.text(query_text), conn)
print(df.shape)
return df.to_json(orient='records')
This returns the error:这将返回错误:
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory".
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) DPI-1047: 找不到 64 位 Oracle 客户端库:“libclntsh.so:无法打开共享的 ZA8CFDE6331BD59EB2ACZF 文件:” See https://oracle.github.io/odpi/doc/installation.html#linux for help (Background on this error at: http://sqlalche.me/e/4xp6 )
请参阅https://oracle.github.io/odpi/doc/installation.html#linux获取帮助(此错误的背景位于: http6://sql.cheme// )
What I have also tried我也尝试过的
I've tried:我试过了:
Environment:
Variables:
ORACLE_HOME: /tmp
LD_LIBRARY_PATH: /tmp
to template.yml
and redeploying.到
template.yml
并重新部署。 Same error as above.与上述相同的错误。
os.system('export LD_LIBRARY_PATH=/tmp/instantclient_19_3')
into the python script.os.system('export LD_LIBRARY_PATH=/tmp/instantclient_19_3')
添加到 python 脚本中。 Same error as above.cp
and ln
things that were forbidden in Lambda outside of the /tmp
folder.cp
和ln
在/tmp
文件夹之外的 Lambda 中被禁止。 Same error as above. One way that works, but is bad一种有效但不好的方法
If I make a folder called lib/
in the Lambda package, and include an odd assortment of libaio.so.1
, libclntsh.so
, etc. files, the function will work as expected, for some reason .如果我在 Lambda package 中创建一个名为
lib/
的文件夹,并包含一个奇怪的分类libaio.so.1
、 libclntsh.so
等文件, ZC1C425268E68385D1AB5074C1 将按预期工作。 I ended up with this:我最终得到了这个:
<all the other libraries and files as above>
lib/
-libaio.so.1
-libclntsh.so
-libclntsh.so.10.1
-libclntsh.so.11.1
-libclntsh.so.12.1
-libclntsh.so.18.1
-libclntsh.so.19.1
-libclntshcore.so.19.1
-libipc1.so
-libmql1.so
-libnnz19.so
-libocci.so
-libocci.so.10.1
-libocci.so.11.1
-libocci.so.12.1
-libocci.so.18.1
-libocci.so.19.1
-libociicus.so
-libons.so
However, I chose these files through trial and error and don't want to go through this again.但是,我通过反复试验选择了这些文件,不想再通过这个 go。
Is there a way to unzip instantclient-basiclite-linux.x64-19.3.0.0.0dbru.zip
in Lambda at runtime, and make Lambda see/use it to connect to an Oracle database? Is there a way to unzip
instantclient-basiclite-linux.x64-19.3.0.0.0dbru.zip
in Lambda at runtime, and make Lambda see/use it to connect to an Oracle database?
I am not by any means an expert at python but this line seems very strange我绝不是 python 的专家,但这条线似乎很奇怪
print('extracting oracle drivers and copying results to /var/task/lib')
os.system('unzip /var/task/instantclient-basiclite-linux.x64-19.3.0.0.0dbru.zip -d /tmp')
print('extraction steps complete')
os.system('export ORACLE_HOME=/tmp/instantclient_19_3')
Normally, you you will have very limited access to OS level API with Lambda.通常,您对操作系统级别 API 和 Lambda 的访问权限非常有限。 And even when you do, It can behave the way you do not expect It to do.
即使你这样做了,它也可以按照你不希望它做的方式行事。 ( Think as if: Who owns the "unzip" feature? File created by this command would be visible / invokable by who? )
(想像:谁拥有“解压缩”功能?此命令创建的文件将由谁可见/可调用?)
I see you mentioned that you have no issue extracting the files which is also a bit strange我看到您提到提取文件没有问题,这也有点奇怪
My only answer for you is that我给你的唯一答案是
1/ Try to "bring your own" tools ( Unzip, etc.. ) 1/ 尝试“自带”工具(解压缩等)
2/ Never try to do OS level calls. 2/ 永远不要尝试进行操作系统级别的调用。 Like os.system('export...'), Always use the full path
和 os.system('export...') 一样,总是使用完整路径
Looking again at your question, seems like the way you specify environment variable is conflicting再次查看您的问题,您指定环境变量的方式似乎有冲突
ORACLE_HOME: /tmp
should not it be不应该
Environment:
Variables:
ORACLE_HOME: /tmp/instantclient_19_3
LD_LIBRARY_PATH: /tmp/instantclient_19_3
Also, see: How to access an AWS Lambda environment variable from Python另请参阅: 如何从 Python 访问 AWS Lambda 环境变量
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.