[英]AWS Lambda Boto gensim model module initialization error: __exit__
Hosting a word2vec model with gensim on AWS lambda在 AWS lambda 上使用 gensim 托管 word2vec model
using python 2.7 boto==2.48.0 gensim==3.4.0使用 python 2.7 boto==2.48.0 gensim==3.4.0
and I have a few lines in my function.py file where I load the model directly from s3我的 function.py 文件中有几行,我直接从 s3 加载 model
print('################### connecting to s3...')
s3_conn = boto.s3.connect_to_region(
region,
aws_access_key_id = Aws_access_key_id,
aws_secret_access_key = Aws_secret_access_key,
is_secure = True,
calling_format = OrdinaryCallingFormat()
)
print('################### connected to s3...')
bucket = s3_conn.get_bucket(S3_BUCKET)
print('################### got bucket...')
key = bucket.get_key(S3_KEY)
print('################### got key...')
model = KeyedVectors.load_word2vec_format(key, binary=True)
print('################### loaded model...')
on the model loading line在 model 装载线上
model = KeyedVectors.load_word2vec_format(key, binary=True)
getting a mysterious error without much details:得到一个没有太多细节的神秘错误:
on the cloud watch can see all of my print messages til '################### got key...' inclusive, then I get:在云端手表上可以看到我所有的打印消息,直到 '################### got key...' 包含在内,然后我得到:
START RequestId: {req_id} Version: $LATEST
then right after it [no time delays between these two messages]然后就在它之后[这两条消息之间没有时间延迟]
module initialization error: __exit__
please, is there a way to get a detailed error or more info?请问,有没有办法获得详细的错误或更多信息?
More background details: I was able to download the model from s3 to /tmp/ and it did authorize and retrieve the model file, but it went out of space [file is ~2GB, /tmp/ is 512MB]更多背景细节:我能够将 model 从 s3 下载到 /tmp/,它确实授权并检索了 model 文件,但空间不足 [文件约为 2GB,/tmp/ 为 512MB]
so, switched to directly loading the model by gensim as above and now getting that mysterious error.所以,切换到上面的gensim直接加载model,现在得到那个神秘的错误。
running the function with python-lambda-local works without issues使用 python-lambda-local 运行 function 可以正常工作
so, this probably narrows it down to an issue with gensim's smart open or aws lambda, would appreciate any hints, thanks!因此,这可能将其缩小到 gensim 的 smart open 或 aws lambda 的问题,不胜感激任何提示,谢谢!
instead of connecting using boto, simply:而不是使用 boto 连接,只需:
model = KeyedVectors.load_word2vec_format('s3://{}:{}@{}/{}'.format(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, S3_BUCKET, S3_KEY), binary=True)
worked!工作!
but of course, unfortunately, it doesn't answer the question on why the mysterious exit error came up and how to get more info:/但当然,不幸的是,它没有回答为什么会出现神秘的退出错误以及如何获取更多信息的问题:/
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.