[英]AWS Lambda - Nodejs: Allocation failed - JavaScript heap out of memory
I am using Lambda (nodeJS) to query noSQL data (dynamo db). 我正在使用Lambda(nodeJS)查询noSQL数据(dynamo db)。
Let's say I have table "student" in DynamoDB, and I have an API which return list of students for a specific class (class_id). 假设我在DynamoDB中具有表“ student”,并且我有一个API,该API返回特定班级(class_id)的学生列表。 (I used "query" ) (我使用“查询”)
As I know, the dynamo paginates the result, so my API works like below: 据我所知,发电机对结果进行分页,因此我的API的工作方式如下:
{class_id : 'xxxx'}
=> return 1st list of students {class_id : 'xxxx'}
=>返回第一名学生名单 {class_id : 'xxxx', last_evaluated_key: { ....}
=> return next list of student (if LastEvaluatedKey
exists) {class_id : 'xxxx', last_evaluated_key: { ....}
=>返回学生的下一个列表(如果存在LastEvaluatedKey
) My lambda code : 我的lambda代码:
exports.handler = function(e, ctx, callback) {
var rp = require('request-promise');
var students = [];
var hasMore = true;
var params = {
class_id: e.class_id
}
while (hasMore) {
var options = {
method: 'POST',
uri: 'https://xxxxxx.eu-west-1.amazonaws.com/dynamodliblightdm-gobject-1-0:amd64liblightdm-gobject-1-0:amd64b/getStudents',
body: params,
json: true // Automatically stringifies the body to JSON
};
rp(options)
.then(function(repos) {
console.log('count: ' + repos.Count);
students.push(repos.Items);
if (repos.hasOwnProperty("LastEvaluatedKey")) {
params['last_evaluated_key'] = repos.LastEvaluatedKey;
} else {
hasMore = false;
}
})
.catch(function(err) {
console.log('Error', err);
});
}
callback(null, 'done.');
}
I got error: 我收到错误消息:
42676 ms: Mark-sweep 804.1 (954.3) -> 802.7 (954.3) MB, 1803.0 / 0.0 ms (+ 246.3 ms in 32 steps since start of marking, biggest step 35.7 ms) [allocation failure] [GC in old space requested]. 42676 ms:打标扫描804.1(954.3)-> 802.7(954.3)MB,1803.0 / 0.0 ms(自打标开始,分32步,+ 246.3 ms,最大步长35.7 ms)[分配失败] [要求在旧空间中使用GC] 。 44415 ms: Mark-sweep 802.7 (954.3) -> 802.7 (954.3) MB, 1738.6 / 0.0 ms [allocation failure] [GC in old space requested]. 44415毫秒:标记扫描802.7(954.3)-> 802.7(954.3)MB,1738.6 / 0.0毫秒[分配失败] [要求在旧空间中使用GC]。 46318 ms: Mark-sweep 802.7 (954.3) -> 809.5 (859.3) MB, 1902.9 / 0.0 ms [last resort gc]. 46318毫秒:标记扫描802.7(954.3)-> 809.5(859.3)MB,1902.9 / 0.0毫秒[最后手段gc]。 48184 ms: Mark-sweep 809.5 (859.3) -> 816.4 (858.3) MB, 1865.7 / 0.0 ms [last resort gc]. 48184毫秒:标记扫描809.5(859.3)-> 816.4(858.3)MB,1865.7 / 0.0毫秒[最后手段gc]。 <--- JS stacktrace ---> ==== JS stack trace ========================================= Security context: 0x322e8723fa99 2: new constructor(aka Multipart) [/var/task/lambda-func/node_modules/request/lib/multipart.js:~8] [pc=0x1b47df3f5f98] (this=0x1175e583149 ,request=0x1175e582fa9 ) 4: new constructor(aka Request) [/var/task/lambda-func/node_modules/request/request.js:125] [pc=0x1b47df4df3e6] (this=0x1175e... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [/var/lang/bin/node] 2: 0x55d79ff0b302 [/var/lang/bin/node] 3: v8::Utils::ReportApiFailure(char const*, char const*) [/var/lang/bin/node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/var/lang/bin/node] 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/var/lang/bin/node] 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/var/lang/bin/node] 7: 0x1b47df2062bf <--- JS堆栈跟踪---> ==== JS堆栈跟踪================================= ========安全上下文:0x322e8723fa99 2:新的构造函数(又名Multipart)[/var/task/lambda-func/node_modules/request/lib/multipart.js:~8] [pc = 0x1b47df3f5f98](this = 0x1175e583149,请求= 0x1175e582fa9)4:新的构造函数(aka请求)[/var/task/lambda-func/node_modules/request/request.js:125] [pc = 0x1b47df4df3e6](此= 0x1175e ...致命错误: CALL_AND_RETRY_LAST分配失败-JavaScript堆内存不足1:node :: Abort()[/ var / lang / bin / node] 2:0x55d79ff0b302 [/ var / lang / bin / node] 3:v8 :: Utils :: ReportApiFailure( char const *,char const *)[/ var / lang / bin / node] 4:v8 :: internal :: V8 :: FatalProcessOutOfMemory(char const *,bool)[/ var / lang / bin / node] 5:v8 :: internal :: Factory :: NewFillerObject(int,bool,v8 :: internal :: AllocationSpace)[/ var / lang / bin / node] 6:v8 :: internal :: Runtime_AllocateInTargetSpace(int,v8 :: internal :: Object **,v8 :: internal :: Isolate *)[/ var / lang / bin / node] 7:0x1b47df2062bf
Any suggestion is appreciated. 任何建议表示赞赏。
Suggestion to get all the students. 建议让所有的学生。
const fetch = (lastEvaluatedKey) => {
return rp().then((res) => {
students = students.concat(res.moreStudents);
if(res.shouldKeepFetching) {
return fetch(res.lastKey);
}
return Promise.resolve();
})
}
fetch().then(() => {
//you fetched them all
})
If there are a shitload of students this might get you with another out of memory problem. 如果有很多学生,这可能会给您带来另一个内存不足的问题。 Another thing you can do, but I don't know if lambda allows it yet, is to use a for loop with async/await 您可以做的另一件事,但我不知道lambda是否允许,是对async / await使用for循环
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.