堆棧內存溢出
  簡體   English   中英

Grisome 構建在本地成功，但在 AWS Amplify 上失敗

[英]Grisome build succeeds locally but fails on AWS Amplify

 原文  2020-04-23 14:53:40  5   1    aws-amplify/ aws-codebuild/ gridsome/ enomem

問題描述

我有一個 Gridsome 站點，我正在嘗試將其部署到 AWS Amplify。

該站點在本地使用gridsome build構建良好並且沒有錯誤，但是在 AWS Amplify 上，它失敗了。

我最初認為該錯誤與我正在使用所需的<ClientOnly>標記的非 SSR 組件有關，因為這會在我的本地構建中引發警告。 但是，我已經刪除了該組件並且問題仍然存在。

我還認為這個問題可能是由於與 webpack 相關的軟件包需要一些過時的依賴項，但在嘗試解決這些問題幾個小時后，我相對確信情況並非如此。

我不太了解錯誤日志，但我認為問題可能與我網站上的大量圖像有關，並且 AWS Amplify 在處理它們時用完了 memory ( ENOMEM )。 如果這甚至是原因，我還沒有找到增加 AWS Amplify 的 memory 分配的方法。

AWS Amplify 日志提供以下信息：

2020-04-23T14:21:23.331Z [WARNING]: Gridsome v0.7.14
2020-04-23T14:21:23.867Z [WARNING]: Initializing plugins...
2020-04-23T14:21:24.053Z [WARNING]: Load sources - 0.1s
2020-04-23T14:21:24.166Z [WARNING]: Create GraphQL schema - 0.11s
2020-04-23T14:21:24.200Z [WARNING]: Create pages and templates - 0.03s
2020-04-23T14:21:24.224Z [WARNING]: Generate temporary code - 0.02s
2020-04-23T14:21:24.224Z [WARNING]: Bootstrap finish - 0.89s
2020-04-23T14:21:24.226Z [WARNING]: Compiling assets...
2020-04-23T14:28:43.461Z [WARNING]: Compile assets - 439.23s
2020-04-23T14:28:43.487Z [WARNING]: Execute GraphQL (14 queries) - 0.03s
2020-04-23T14:28:43.492Z [WARNING]: Write out page data (14 files) - 0s
2020-04-23T14:28:45.196Z [WARNING]: Render HTML (14 files) - 1.7s
2020-04-23T14:28:45.198Z [WARNING]: Process files (0 files) - 0s
2020-04-23T14:28:45.563Z [INFO]: [2K
2020-04-23T14:28:45.563Z [INFO]: [1GProcessing images (7675 images) - 0%
2020-04-23T14:28:47.487Z [INFO]: [2K
2020-04-23T14:28:47.489Z [INFO]: [1GProcessing images (7675 images) - 0%
2020-04-23T14:28:47.579Z [INFO]: [2K[1G
2020-04-23T14:28:47.579Z [INFO]: Processing images (7675 images) - 1%
2020-04-23T14:28:48.834Z [INFO]: [2K[1GProcessing images (7675 images) - 1%
...
2020-04-23T14:33:25.168Z [INFO]: [2K[1GProcessing images (7675 images) - 41%
2020-04-23T14:33:26.901Z [WARNING]: internal/child_process.js:366
                                    throw errnoException(err, 'spawn');
                                    ^
                                    Error: spawn ENOMEM
                                    at ChildProcess.spawn (internal/child_process.js:366:11)
                                    at spawn (child_process.js:551:9)
                                    at Object.fork (child_process.js:113:10)
                                    at ChildProcessWorker.initialize (/codebuild/output/src926123643/src/website/node_modules/jest-worker/build/workers/ChildProcessWorker.js:105:44)
                                    at ChildProcessWorker.onExit (/codebuild/output/src926123643/src/website/node_modules/jest-worker/build/workers/ChildProcessWorker.js:204:12)
                                    at ChildProcess.emit (events.js:198:13)
                                    at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
2020-04-23T14:33:27.254Z [WARNING]: npm
2020-04-23T14:33:27.254Z [WARNING]: ERR! code ELIFECYCLE
                                    npm ERR! errno 1
2020-04-23T14:33:27.255Z [WARNING]: npm
2020-04-23T14:33:27.259Z [WARNING]: ERR! website@ build: `gridsome build`
                                    npm ERR! Exit status 1
                                    npm ERR!
                                    npm ERR! Failed at the website@ build script.
                                    npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2020-04-23T14:33:27.270Z [WARNING]: 
2020-04-23T14:33:27.270Z [WARNING]: npm ERR! A complete log of this run can be found in:
                                    npm ERR!     /root/.npm/_logs/2020-04-23T14_33_27_259Z-debug.log
2020-04-23T14:33:27.270Z [HELP]: Outputting the npm debug log
                                 0 info it worked if it ends with ok
                                 1 verbose cli [ '/root/.nvm/versions/node/v10.16.0/bin/node',
                                 1 verbose cli   '/root/.nvm/versions/node/v10.16.0/bin/npm',
                                 1 verbose cli   'run',
                                 1 verbose cli   'build' ]
                                 2 info using npm@6.9.0
                                 3 info using node@v10.16.0
                                 4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
                                 5 info lifecycle website@~prebuild: website@
                                 6 info lifecycle website@~build: website@
                                 7 verbose lifecycle website@~build: unsafe-perm in lifecycle true
                                 8 verbose lifecycle website@~build: PATH: /root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/codebuild/output/src926123643/src/website/node_modules/.bin:/usr/local/rvm/gems/ruby-2.4.6/bin:/usr/local/rvm/gems/ruby-2.4.6@global/bin:/usr/local/rvm/rubies/ruby-2.4.6/bin:/usr/local/rvm/bin:/root/.yarn/bin:/root/.config/yarn/global/node_modules/.bin:/root/.nvm/versions/node/v10.16.0/bin:/root/.local/bin:/usr/local/rvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                                 9 verbose lifecycle website@~build: CWD: /codebuild/output/src926123643/src/website
                                 10 silly lifecycle website@~build: Args: [ '-c', 'gridsome build' ]
                                 11 silly lifecycle website@~build: Returned: code: 1  signal: null
                                 12 info lifecycle website@~build: Failed to exec build script
                                 13 verbose stack Error: website@ build: `gridsome build`
                                 13 verbose stack Exit status 1
                                 13 verbose stack     at EventEmitter.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
                                 13 verbose stack     at EventEmitter.emit (events.js:198:13)
                                 13 verbose stack     at ChildProcess.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
                                 13 verbose stack     at ChildProcess.emit (events.js:198:13)
                                 13 verbose stack     at maybeClose (internal/child_process.js:982:16)
                                 13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
                                 14 verbose pkgid website@
                                 15 verbose cwd /codebuild/output/src926123643/src/website
                                 16 verbose Linux 4.14.165-103.209.amzn1.x86_64
                                 17 verbose argv "/root/.nvm/versions/node/v10.16.0/bin/node" "/root/.nvm/versions/node/v10.16.0/bin/npm" "run" "build"
                                 18 verbose node v10.16.0
                                 19 verbose npm  v6.9.0
                                 20 error code ELIFECYCLE
                                 21 error errno 1
                                 22 error website@ build: `gridsome build`
                                 22 error Exit status 1
                                 23 error Failed at the website@ build script.
                                 23 error This is probably not a problem with npm. There is likely additional logging output above.
                                 24 verbose exit [ 1, true ]
2020-04-23T14:33:27.291Z [ERROR]: !!! Build failed
2020-04-23T14:33:27.292Z [ERROR]: !!! Non-Zero Exit Code detected
2020-04-23T14:33:27.292Z [INFO]: # Starting environment caching...
2020-04-23T14:33:27.296Z [INFO]: # Environment caching completed
Terminating logging...

我的 package.json 包含以下內容：

{
  "name": "website",
  "private": true,
  "scripts": {
    "build": "gridsome build",
    "develop": "gridsome develop",
    "explore": "gridsome explore"
  },
  "dependencies": {
    "@gridsome/source-filesystem": "^0.6.2",
    "@gridsome/transformer-remark": "^0.5.0",
    "gridsome": "^0.7.14",
    "gridsome-plugin-gtm": "^0.1.1",
    "marked": "^0.8.0",
    "vue-burger-menu": "^2.0.4",
    "vue-easy-lightbox": "^0.11.0",
    "vue-slick-carousel": "^1.0.5"
  },
  "devDependencies": {
    "node-sass": "^4.13.1",
    "normalize.css": "^8.0.1",
    "sass-loader": "^8.0.2",
    "style-resources-loader": "^1.3.3"
  },
  "browserslist": [
    "defaults"
  ]
}

在嘗試解決此問題時，我將不勝感激。

先感謝您。

更新#1

我現在很確定這是由於我的網站上通過<g-image>處理了大量圖像。

我決定嘗試使用 AWS CodeBuild 進行構建，看看是否有更多信息。 它也失敗了，但確實給了我一些更有幫助的錯誤。

當我刪除了除少數圖像之外的所有圖像時，構建工作正常。 一旦我以不同尺寸顯示超過 300 張圖像，事情就會開始失敗。 錯誤消息：

Process files (0 files) - 0s
...
[2K·[1GProcessing images (4671 images) - 58%Error: Failed to process image src/assets/media/jobs/1.jpg. spawn ENOMEM

    at pMap.concurrency (/codebuild/output/src132267748/src/github.com/xxxxx/website/node_modules/gridsome/lib/workers/image-processor.js:113:13)

(sharp:31795): GLib-CRITICAL **: 04:50:22.507: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_lookup: assertion 'hash_table != NULL' failed

所以，是的，我猜這是一個 Gridsome 錯誤？

更新#2

在.npmrc文件中添加node-options=--max_old_space_size=4096會導致構建掛起並最終超時。

更新#3

批量調整圖像大小，使最大寬度/高度為 2000 像素（有效最大文件大小約為 2MB）。 構建現在成功。

希望有人可以提供解決方案，以便我們可以從源文件構建而無需預處理圖像。

1 個解決方案

解決方案1
 0  2020-04-26 11:09:57

這似乎是 Gridsome 在處理大量大文件時用完 memory 的錯誤。

此解決方法預處理所有圖像文件，以便在 Gridsome 嘗試處理它們之前減少每個圖像的負載。

使用 Sharp 預處理圖像

// image-preprocessor.js
const fs = require("fs");
const path = require("path");
const sharp = require("sharp");

const dirPath = "./src/assets/";
const maxWidth = 2000;
const maxHeight = 2000;

const getAllFiles = function(dirPath, arrayOfFiles) {
  files = fs.readdirSync(dirPath);

  arrayOfFiles = arrayOfFiles || [];

  files.forEach(function(file) {
    if (fs.statSync(`${dirPath}/${file}`).isDirectory()) {
      arrayOfFiles = getAllFiles(`${dirPath}/${file}`, arrayOfFiles);
    } else {
      arrayOfFiles.push(path.join(__dirname, dirPath, "/", file));
    }
  });
  return arrayOfFiles;
};

const result = getAllFiles(dirPath);

result.forEach(async (file) => {
  await sharp(`${file}`)
    .resize(maxWidth, maxHeight, {
      fit: "inside",
    })
    .toBuffer(function(err, buffer) {
      fs.writeFile(`${file}`, buffer, function(err) {
        if (err) {
          console.log(err);
        }
      });
    });
});

將圖像預處理器添加到構建規范

// buildspec.yml
  ...
  build:
    commands:
      - node image-preprocessor.js
      - npm run build
  ...

請注意，由於每次通過 buildpsec 構建 Gridsome 時都會進行預處理，因此源圖像在存儲庫中保持不變（並最終等待對該問題的適當修復）。

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 React 構建在本地運行但在 Amplify 上失敗 在 AWS Amplify 上構建 React 應用程序在 Amplify 控制台中失敗 如何在 AWS Amplify 中設置我的構建腳本？ 它們在本地成功運行，但在 AWS Amplify 的構建過程中失敗 AWS Amplify 構建設置 AWS Amplify 構建問題 - StackUpdateComplete [AWS][Amplify] Invoke function 本地崩潰無錯誤 從 React v16 更新到 v18 時，React 項目構建在 Aws amplify 上失敗 為什么添加 Auth 功能時 AWS-Amplify 構建失敗 NextJs SSR 構建在 Amplify 上部署失敗 放大構建失敗並顯示“請求失敗 401 未經授權”
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM