[英]AWS Lambda w/ Google Vision API throwing PEM_read_bio:no start line or Errno::ENAMETOOLONG
The Goal: User uploads to S3, Lambda is triggered to take the file and send to Google Vision API for analysis, returning the results. 目标:用户上传到S3,Lambda被触发以获取文件并发送到Google Vision API进行分析,并返回结果。
According to this , google-cloud
requires native libraries and must be compiled against the OS that lambda is running. 根据此 , google-cloud
需要本机库,必须针对拉姆达运行的操作系统进行编译。 Using lambda-packager
threw an error but some internet searching turned up using an EC2 with Node and NPM to run the install instead. 使用lambda-packager
引发了错误,但是一些Internet搜索使用带有Node和NPM的EC2来运行安装程序。 In the spirit of hacking through this, that's what I did to get it mostly working*. 本着黑客的精神,这就是我所做的,以使其大部分都能正常工作*。 At least lambda stopped giving me ELF header errors. 至少lambda停止给我ELF标头错误。
My current problem is that there are 2 ways to call the Vision API, neither work and both return a different error (mostly). 我当前的问题是,有两种方法可以调用Vision API,但两种方法均无效,并且两种方法都返回不同的错误(大多数情况下)。
The Common Code: This code is always the same, it's at the top of the function, and I'm separating it to keep the later code blocks focused on the issue. 通用代码:此代码始终相同,它位于函数的顶部,我将其分开以使以后的代码块专注于此问题。
'use strict';
const AWS = require('aws-sdk');
const S3 = new AWS.S3();
const Bucket = 'my-awesome-bucket';
const gCloudConfig = {
projectId: 'myCoolApp',
credentials: {
client_email: 'your.serviceapi@project.email.com',
private_key: 'yourServiceApiPrivateKey'
}
}
const gCloud = require('google-cloud')(gCloudConfig);
const gVision = gCloud.vision();
Using detect()
: This code always returns the error Error: error:0906D06C:PEM routines:PEM_read_bio:no start line
. 使用detect()
:此代码始终返回错误Error: error:0906D06C:PEM routines:PEM_read_bio:no start line
。 Theoretically it should work because the URL is public. 理论上,它应该起作用,因为URL是公共的。 From searching on the error, I considered it might be an HTTPS thing, so I've even tried a variation on this where I replaced HTTPS with HTTP but got the same error. 通过搜索错误,我认为这可能是HTTPS的事情,因此我什至尝试了一种变体,将HTTPS替换为HTTPS,但遇到了同样的错误。
exports.handler = (event, context, callback) => {
const params = {
Bucket,
Key: event.Records[0].s3.object.key
}
const img = S3.getSignedUrl('getObject', params);
gVision.detect(img, ['labels','text'], function(err, image){
if(err){
console.log('vision error', err);
}
console.log('vision result:', JSON.stringify(image, true, 2));
});
}
Using detectLabels()
: This code always returns Error: ENAMETOOLONG: name too long, open ....[the image in base64]...
. 使用detectLabels()
:此代码始终返回Error: ENAMETOOLONG: name too long, open ....[the image in base64]...
On a suggestion, it was believed that the method shouldn't be passed the base64 image, but instead the public path; 有人建议,该方法不应传递给base64映像,而应传递给公共路径。 which would explain why it says the name is too long (a base64 image is quite the URL). 这可以解释为什么它说名称太长(base64图像就是URL)。 Unfortunately, that gives the PEM error from above. 不幸的是,这从上面给出了PEM错误。 I've also tried not doing the base64 encoding and pass the object buffer directly from aws but that resulted in a PEM error too. 我也尝试过不进行base64编码,而是直接从aws传递对象缓冲区,但这也导致了PEM错误。
exports.handler = (event, context, callback) => {
const params = {
Bucket,
Key: event.Records[0].s3.object.key
}
S3.getObject(params, function(err, data){
const img = data.Body.toString('base64');
gVision.detectLabels(img, function(err, labels){
if(err){
console.log('vision error', err);
}
console.log('vision result:', labels);
});
});
}
According to Best Practices , the image should be base64 encoded. 根据最佳实践 ,该图像应使用base64编码。
From the API docs and examples and whatever else, it seems that I'm using these correctly. 从API文档和示例以及其他任何内容来看,我似乎都在正确使用它们。 I feel like I've read all those docs a million times. 我觉得我已经阅读了所有这些文档一百万次。
I'm not sure what to make of the NAMETOOLONG error if it's expecting base64 stuff. 我不确定如果要使用base64,该如何处理NAMETOOLONG错误。 These images aren't more than 1MB. 这些图像不超过1MB。
*The PEM error seems to be related to credentials, and because my understanding of how all these credentials work and how the modules are being compiled on EC2 (which doesn't have any kind of PEM files), that might be my problem. * PEM错误似乎与凭证有关,并且因为我对所有这些凭证的工作方式以及如何在EC2(没有任何PEM文件)上编译模块的理解,可能是我的问题。 Maybe I need to set up some credentials before running npm install
, kind of in the same vein as needing to be installed on a linux box? 也许我需要在运行npm install
之前设置一些凭据,就像需要在linux机器上安装一样? This is starting to be outside my range of understanding so I'm hoping someone here knows. 这开始超出我的理解范围,所以我希望这里的人知道。
Ideally, using detect
would be better because I can specify what I want detected, but just getting any valid response from Google would be awesome. 理想情况下,使用detect
会更好,因为我可以指定要检测的内容,但是仅从Google获得任何有效的响应就很棒。 Any clues you all can provide would be greatly appreciated. 大家都可以提供的任何线索将不胜感激。
So, a conversation with another colleague pointed me to consider abandoning the whole loading of the API and using the google-cloud
module. 因此,在与另一位同事的对话中,我建议我考虑放弃API的整个加载过程,而使用google-cloud
模块。 Instead, I should consider trying the Cloud REST API via curl
and seeing if it can work that way. 相反,我应该考虑通过curl
尝试使用Cloud REST API,看看它是否可以那样工作。
Long story short, making an HTTP request and using the REST API for Google Cloud was how I solved this issue. 简而言之,我提出了一个HTTP请求并针对Google Cloud使用REST API。
Here is the working lambda function I have now. 这是我现在可以使用的lambda函数。 Probably still needs tweaks but this is working. 可能仍然需要进行调整,但这是可行的。
'use strict';
const AWS = require('aws-sdk');
const S3 = new AWS.S3();
const Bucket = 'yourBucket';
const fs = require('fs');
const https = require('https');
const APIKey = 'AIza...your.api.key...kIVc';
const options = {
method: 'POST',
host: `vision.googleapis.com`,
path: `/v1/images:annotate?key=${APIKey}`,
headers: {
'Content-Type': 'application/json'
}
}
exports.handler = (event, context, callback) => {
const req = https.request(options, res => {
const body = [];
res.setEncoding('utf8');
res.on('data', chunk => {
body.push(chunk);
});
res.on('end', () => {
console.log('results', body.join(''));
callback(null, body.join(''));
});
});
req.on('error', err => {
console.log('problem with request:', err.message);
});
const params = {
Bucket,
Key: event.Records[0].s3.object.key
}
S3.getObject(params, function(err, data){
const payload = {
"requests": [{
"image": {
"content": data.Body.toString('base64')
},
"features": [{
"type": "LABEL_DETECTION",
"maxResults": 10
},{
"type": "TEXT_DETECTION",
"maxResults": 10
}]
}]
};
req.write(JSON.stringify(payload));
req.end();
});
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.