堆栈内存溢出
  繁体   English   中英

AWS Elastic Beanstalk 中使用 PlaywrightCrawler 时出错 package

[英]ERROR in AWS Elastic Beanstalk for using PlaywrightCrawler package

 原文  2022-12-06 03:23:18  4   1    node.js/ amazon-web-services/ amazon-elastic-beanstalk/ playwright

问题描述

在我的服务器环境中尝试使用 PlaywrightCrawler package 时，出现以下错误日志。

        [32mINFO[39m [33m PlaywrightCrawler:[39m Starting the crawl
        [33mWARN[39m [33m PlaywrightCrawler:[39m Reclaiming failed request back to the list or queue. browserType.launchPersistentContext: Failed to launch: Error: spawn /root/.cache/ms-playwright/chromium-1033/chrome-linux/chrome ENOENT
    
============================================================
        at async PlaywrightPlugin._launch (/usr/src/app/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)
        at async BrowserPool._launchBrowser (/usr/src/app/node_modules/@crawlee/browser-pool/browser-pool.js:465:29)
        at async /usr/src/app/node_modules/@crawlee/browser-pool/browser-pool.js:274:37[90m {"id":"erPiwuCRDH5Dzyr","url":"https://www.google.com.au/search?tbm=shop&hl=en-AU&psb=1&ved=2ahUKEwjP49q7gdb7AhUyn0sFHcYhAAQQu-kFegQIABAL&q=Coles+Extra Virgin Avocado Oil 250mL&oq=Coles+Extra Virgin Avocado Oil 250mL&gs_lcp=Cgtwcm9kdWN0cy1jYxADUABYAGAAaABwAHgAgAEAiAEAkgEAmAEA&sclient=products-cc&ts=658","method":"GET","uniqueKey":"https://www.google.com.au/search?gs_lcp=Cgtwcm9kdWN0cy1jYxADUABYAGAAaABwAHgAgAEAiAEAkgEAmAEA&hl=en-AU&oq=Coles+Extra Virgin Avocado Oil 250mL&psb=1&q=Coles+Extra Virgin Avocado Oil 250mL&sclient=products-cc&tbm=shop&ts=658&ved=2ahUKEwjP49q7gdb7AhUyn0sFHcYhAAQQu-kFegQIABAL"}[39m
    [32mINFO[39m [33m PlaywrightCrawler:[39m All the requests from request list and/or request queue have been processed, the crawler will shut down.
    [32mINFO[39m [33m PlaywrightCrawler:[39m Crawl finished. Final request statistics:[90m {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,1],"requestAvgFailedDurationMillis":107,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":5,"requestTotalDurationMillis":107,"requestsTotal":1,"crawlerRuntimeMillis":11078}[39m
            
    [32mINFO[39m [33m PlaywrightCrawler:[39m Error analysis:[90m {"totalErrors":1,"uniqueErrors":1,"mostCommonErrors":["1x: browserType.launchPersistentContext: Failed to launch: Error: spawn /root/.cache/ms-playwright/chromium-1033/chrome-linux/chrome ENOENT (/usr/src/app/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)"]}[39m

这是 my.ebextension/chromium.config：

packages:
  yum:
    cups-libs: []
    dbus-glib: []
    libXrandr: []
    libXcursor: []
    libXinerama: []
    cairo: []
    cairo-gobject: []
    pango: []
    libwayland-client: []
    libwayland-cursor: []
    gtk3: []
    gdk-pixbuf2: []
    libxkbcommon: []
    libXScrnSaver: []
    GConf2: []
    atk: []
    at-spi2-atk: []
    at-spi2-core: []
    alsa-lib: []

container_commands:
  install_playwright: 
    command: "npx playwright install --with-deps chromium"

下面是我的 yum.log 的 output：

Dec 08 10:06:18 Installed: libwayland-client-1.17.0-1.amzn2.x86_64
Dec 08 10:06:18 Installed: mesa-libglapi-18.3.4-5.amzn2.0.1.x86_64
Dec 08 10:06:18 Installed: atk-2.22.0-3.amzn2.0.2.x86_64
Dec 08 10:06:18 Installed: fontpackages-filesystem-1.44-8.amzn2.noarch
Dec 08 10:06:19 Installed: 1:liberation-fonts-common-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: libusbx-1.0.21-1.amzn2.x86_64
Dec 08 10:06:19 Installed: pixman-0.34.0-1.amzn2.0.2.x86_64
Dec 08 10:06:19 Installed: libxshmfence-1.2-1.amzn2.0.2.x86_64
Dec 08 10:06:19 Installed: 1:libglvnd-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
Dec 08 10:06:19 Installed: libwayland-server-1.17.0-1.amzn2.x86_64
Dec 08 10:06:19 Installed: mesa-libgbm-18.3.4-5.amzn2.0.1.x86_64
Dec 08 10:06:19 Installed: libgusb-0.2.9-1.amzn2.0.2.x86_64
Dec 08 10:06:19 Installed: 1:liberation-mono-fonts-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: 1:liberation-serif-fonts-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: 1:liberation-narrow-fonts-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: 1:liberation-sans-fonts-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: 1:liberation-fonts-1.07.2-16.amzn2.noarch
Dec 08 10:06:19 Installed: dejavu-fonts-common-2.33-6.amzn2.noarch
Dec 08 10:06:19 Installed: dejavu-sans-fonts-2.33-6.amzn2.noarch
Dec 08 10:06:20 Installed: fontconfig-2.13.0-4.3.amzn2.x86_64
Dec 08 10:06:21 Installed: libwayland-cursor-1.17.0-1.amzn2.x86_64
Dec 08 10:06:21 Installed: graphite2-1.3.10-1.amzn2.0.2.x86_64
Dec 08 10:06:21 Installed: harfbuzz-1.7.5-2.amzn2.x86_64
Dec 08 10:06:21 Installed: vulkan-filesystem-1.0.61.1-2.amzn2.noarch
Dec 08 10:06:21 Installed: jasper-libs-1.900.1-33.amzn2.x86_64
Dec 08 10:06:21 Installed: alsa-lib-1.1.4.1-2.amzn2.x86_64
Dec 08 10:06:21 Installed: fribidi-1.0.2-1.amzn2.1.x86_64
Dec 08 10:06:21 Installed: libepoxy-1.3.1-2.amzn2.x86_64
Dec 08 10:06:21 Installed: json-glib-1.4.2-2.amzn2.x86_64
Dec 08 10:06:21 Installed: hicolor-icon-theme-0.12-7.amzn2.noarch
Dec 08 10:06:21 Installed: xkeyboard-config-2.20-1.amzn2.noarch
Dec 08 10:06:21 Installed: libxkbcommon-0.7.1-3.amzn2.x86_64
Dec 08 10:06:22 Installed: dconf-0.28.0-4.amzn2.x86_64
Dec 08 10:06:22 Installed: gsettings-desktop-schemas-3.28.0-3.amzn2.0.1.x86_64
Dec 08 10:06:22 Installed: avahi-libs-0.6.31-20.amzn2.x86_64
Dec 08 10:06:22 Installed: 1:cups-libs-1.6.3-51.amzn2.x86_64
Dec 08 10:06:22 Installed: libthai-0.1.14-9.amzn2.0.2.x86_64
Dec 08 10:06:22 Installed: trousers-0.3.14-2.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: gnutls-3.3.29-9.amzn2.0.1.x86_64
Dec 08 10:06:23 Installed: libXau-1.0.8-2.1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libxcb-1.12-1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: vulkan-1.0.61.1-2.amzn2.x86_64
Dec 08 10:06:23 Installed: libX11-common-1.6.7-3.amzn2.0.2.noarch
Dec 08 10:06:23 Installed: libX11-1.6.7-3.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXext-1.3.3-3.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXrender-0.9.10-1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXfixes-5.0.3-1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXdamage-1.1.4-4.1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXrandr-1.5.1-2.amzn2.0.3.x86_64
Dec 08 10:06:23 Installed: libXi-1.7.9-1.amzn2.0.2.x86_64
Dec 08 10:06:23 Installed: libXcomposite-0.4.4-4.1.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: gdk-pixbuf2-2.36.12-3.amzn2.x86_64
Dec 08 10:06:24 Installed: gtk-update-icon-cache-3.22.30-3.amzn2.x86_64
Dec 08 10:06:24 Installed: libXtst-1.2.3-1.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: at-spi2-core-2.22.0-1.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: at-spi2-atk-2.22.0-2.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: libXcursor-1.1.15-1.amzn2.x86_64
Dec 08 10:06:24 Installed: libXft-2.3.2-2.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: libXinerama-1.1.3-2.1.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: libXxf86vm-1.1.4-1.amzn2.0.2.x86_64
Dec 08 10:06:24 Installed: mesa-libGL-18.3.4-5.amzn2.0.1.x86_64
Dec 08 10:06:24 Installed: 1:libglvnd-glx-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
Dec 08 10:06:24 Installed: 1:libglvnd-egl-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
Dec 08 10:06:24 Installed: mesa-libEGL-18.3.4-5.amzn2.0.1.x86_64
Dec 08 10:06:24 Installed: cairo-1.15.12-4.amzn2.x86_64
Dec 08 10:06:24 Installed: pango-1.42.4-4.amzn2.x86_64
Dec 08 10:06:24 Installed: cairo-gobject-1.15.12-4.amzn2.x86_64
Dec 08 10:06:24 Installed: libwayland-egl-1.17.0-1.amzn2.x86_64
Dec 08 10:06:24 Installed: 1:emacs-filesystem-27.2-4.amzn2.0.1.noarch
Dec 08 10:06:24 Installed: desktop-file-utils-0.23-2.amzn2.x86_64
Dec 08 10:06:25 Installed: xdg-utils-1.1.0-0.17.20120809git.amzn2.noarch
Dec 08 10:06:25 Installed: lcms2-2.6-3.amzn2.0.2.x86_64
Dec 08 10:06:25 Installed: colord-libs-1.3.4-1.amzn2.0.2.x86_64
Dec 08 10:06:25 Installed: adwaita-cursor-theme-3.26.0-1.amzn2.noarch
Dec 08 10:06:28 Installed: adwaita-icon-theme-3.26.0-1.amzn2.noarch
Dec 08 10:06:28 Installed: libmodman-2.0.1-8.amzn2.0.2.x86_64
Dec 08 10:06:28 Installed: libproxy-0.4.11-10.amzn2.0.3.x86_64
Dec 08 10:06:28 Installed: glib-networking-2.56.1-1.amzn2.x86_64
Dec 08 10:06:28 Installed: libsoup-2.56.0-6.amzn2.x86_64
Dec 08 10:06:28 Installed: rest-0.8.0-2.amzn2.x86_64
Dec 08 10:06:29 Installed: gtk3-3.22.30-3.amzn2.x86_64
Dec 08 10:06:43 Installed: google-chrome-stable-108.0.5359.98-1.x86_64

即使已经安装了所有的铬依赖项，仍然会出现问题。

在本地运行时没有这个问题。

希望有人可以建议是什么问题。

更新：

经过进一步研究发现问题可能与 Docker 图像有关。 因此，我用[zenika/alpine-chrome:with-playwright][1]替换了我现有的 Docker 图像，该图像具有 playwright 和 headless chrome，并包含用于npx playwright install的新 RUN 命令。 现在得到一个新的错误是：

[31mERROR[39m[33m PlaywrightCrawler:[39m Request failed and reached maximum retries. browserType.launchPersistentContext: Executable doesn't exist at /home/chrome/.cache/ms-playwright/chromium-1028/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at async PlaywrightPlugin._launch (/usr/src/app/node_modules/crawlee/node_modules/@crawlee/playwright/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)
    at async BrowserPool._launchBrowser (/usr/src/app/node_modules/@crawlee/browser/node_modules/@crawlee/browser-pool/browser-pool.js:465:29)
    at async /usr/src/app/node_modules/@crawlee/browser/node_modules/@crawlee/browser-pool/browser-pool.js:274:37[90m {"id":"tGOZOvUIBmhYpXW","url":"https://www.google.com.au/search?tbm=shop&hl=en-AU&psb=1&ved=2ahUKEwjP49q7gdb7AhUyn0sFHcYhAAQQu-kFegQIABAL&q=Coles+Extra Virgin Avocado Oil 250mL&oq=Coles+Extra Virgin Avocado Oil 250mL&gs_lcp=Cgtwcm9kdWN0cy1jYxADUABYAGAAaABwAHgAgAEAiAEAkgEAmAEA&sclient=products-cc&ts=782","method":"GET","uniqueKey":"https://www.google.com.au/search?gs_lcp=Cgtwcm9kdWN0cy1jYxADUABYAGAAaABwAHgAgAEAiAEAkgEAmAEA&hl=en-AU&oq=Coles+Extra Virgin Avocado Oil 250mL&psb=1&q=Coles+Extra Virgin Avocado Oil 250mL&sclient=products-cc&tbm=shop&ts=782&ved=2ahUKEwjP49q7gdb7AhUyn0sFHcYhAAQQu-kFegQIABAL"}[39m
[32mINFO[39m [33m PlaywrightCrawler:[39m All the requests from request list and/or request queue have been processed, the crawler will shut down.
[32mINFO[39m [33m PlaywrightCrawler:[39m Crawl finished. Final request statistics:[90m {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,1],"requestAvgFailedDurationMillis":18,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":5,"requestTotalDurationMillis":18,"requestsTotal":1,"crawlerRuntimeMillis":11089}[39m
[32mINFO[39m [33m PlaywrightCrawler:[39m Error analysis:[90m {"totalErrors":1,"uniqueErrors":1,"mostCommonErrors":["1x: browserType.launchPersistentContext: Executable doesn't exist at /home/chrome/.cache/ms-playwright/chromium-1028/chrome-linux/chrome (/usr/src/app/node_modules/crawlee/node_modules/@crawlee/playwright/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)"]}[39m

这是我的 Dockerfile：

# # This file is a template, and might need editing before it works on your project.
FROM zenika/alpine-chrome:with-playwright

WORKDIR /usr/src/app

ENV PORT 3000

COPY package.json /usr/src/app/
RUN npm install --force
RUN npx playwright install

COPY . /usr/src/app

# replace this with your application's default port
EXPOSE 3000
CMD [ "npm", "start" ]

1 个解决方案

解决方案1
 0 已采纳  2023-01-10 12:30:39

全部固定。

这是Dockerfile

# Get the base image of Node version 16
FROM node:16

# Get the latest version of Playwright
FROM mcr.microsoft.com/playwright:focal

# Set the work directory for the application
WORKDIR /app

# COPY the needed files to the app folder in Docker image
COPY package*.json /app/

# Get the needed libraries to run Playwright
RUN apt-get update && apt-get -y install libnss3 libatk-bridge2.0-0 libdrm-dev libxkbcommon-dev libgbm-dev libasound-dev libatspi2.0-0 libxshmfence-dev

# Install the dependencies in Node environment
RUN npm ci

# Start the main script
CMD ["node", "--inspect=0.0.0.0:9229", "src/main.js"]

这是EB配置

container:
  install_playwright: 
    command: PLAYWRIGHT_BROWSERS_PATH=$HOME/pw-browsers npx playwright install

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 AWS Elastic Beanstalk EFS 安装错误：未知文件系统类型“efs” 在 AWS Elastic Beanstalk 上重新请求 在 aws linux 2 个实例上使用弹性 beanstalk 运行 django-q 每当 Gem AWS 弹性 beanstalk 开发环境和AWS elastic beanstalk 无法安装弹性 beantalk AWS aws elastic beanstalk 部署了我的应用程序，但在下一次部署时抛出错误 如何解决 AWS Elastic Beanstalk 504 超时错误？ AWS Elastic Beanstalk CLI 使用错误的 Python 版本 将 django 部署到 AWS Elastic Beanstalk
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM