Running plotly dash selenium tests in gitlab CI

Question

So I'm attempting to implement selenium tests within a GitLab ci pipeline. Currently, I've run these tests on my local system mirroring the testing method that is going to happen within the pipeline ie I have my tests build on my local system then attempt to connect to the selenium/standalone-chrome instance to run tests with this command

python3 -m pytest --junitxml=report.xml --log-cli-level DEBUG --webdriver Chrome --headless --remote-url http://localhost:4444/wd/hub testing/

This functions correctly on my local machine. When attempting to use the gitlab-ci.yml

stages:
  - test

image: "python"

services:
  - selenium/standalone-chrome

test:
  stage: test
  tags:
    - docker
  services:
    - selenium/standalone-chrome:latest
  script:
    # Set environmental variables
    - DEV_DASHBOARD_CFG='./app/config.json'
    - apt-get update -qy
    - apt-get install -y xvfb zip wget ca-certificates
    - apt-get install -y libnss3-dev libasound2 libxss1 libappindicator3-1 libindicator7 gconf-service libgconf-2-4 libpango1.0-0 xdg-utils fonts-liberation libgbm1
    # Setup the application to run
    - apt-get install -y python-dev python-pip
    - pip install -r requirements.txt
    - cd app/
    - pwd
    - python3 -m pytest --junitxml=report.xml --log-cli-level DEBUG --webdriver Chrome -headless --remote-url http://selenium__standalone-chrome:4444/wd/hub testing/
  artifacts:
    when: always
    reports:
      junit: report.xml

This issue I'm facing is that it seems that there is an error with the chromium web driver.

---------------------------- Captured stdout call -----------------------------
 Dash is running on http://127.0.0.1:8050/
 Dash is running on http://127.0.0.1:8050/
 Dash is running on http://127.0.0.1:8050/
 Dash is running on http://127.0.0.1:8050/
 Dash is running on http://127.0.0.1:8050/
 Dash is running on http://127.0.0.1:8050/
 2021-04-23 14:27:21,034 | dash - INFO: Dash is running on http://127.0.0.1:8050/
  * Serving Flask app "app.app" (lazy loading)
  * Environment: production
    WARNING: This is a development server. Do not use it in a production deployment.
    Use a production WSGI server instead.
  * Debug mode: off
 2021-04-23 14:27:21,051 | _internal - INFO:  * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
 2021-04-23 14:27:21,164 | dashboard - INFO: Serving layout...
 2021-04-23 14:27:21,166 | app_cache - DEBUG: retrieving from cache: select distinct country from results
 2021-04-23 14:27:21,183 | _internal - INFO: 127.0.0.1 - - [23/Apr/2021 14:27:21] "GET / HTTP/1.1" 200 -
 ------------------------------ Captured log call -------------------------------
 INFO     app.app:dash.py:1714 Dash is running on http://127.0.0.1:8050/
 INFO     werkzeug:_internal.py:113  * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
 INFO     werkzeug:_internal.py:113 127.0.0.1 - - [23/Apr/2021 14:27:21] "GET / HTTP/1.1" 200 -
 --------------------------- Captured stdout teardown ---------------------------
 2021-04-23 14:27:23,110 | _internal - INFO: 127.0.0.1 - - [23/Apr/2021 14:27:23] "GET /_stop-adc0916c7cdb4b169e4eeb4c3f3f532e HTTP/1.1" 200 -
 ---------------------------- Captured log teardown -----------------------------
 INFO     werkzeug:_internal.py:113 127.0.0.1 - - [23/Apr/2021 14:27:23] "GET /_stop-adc0916c7cdb4b169e4eeb4c3f3f532e HTTP/1.1" 200 -
 _______________________________ test_tab_styles ________________________________
 dash_duo = <dash.testing.composite.DashComposite object at 0x7f14d260f610>
     def test_tab_styles(dash_duo):
 >       dash_duo.start_server(app)
 testing/test_db_tabs.py:28: 
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 /usr/local/lib/python3.9/site-packages/dash/testing/composite.py:16: in start_server
     self.server_url = self.server.url
 /usr/local/lib/python3.9/site-packages/dash/testing/browser.py:601: in server_url
     self.wait_for_page()
 /usr/local/lib/python3.9/site-packages/dash/testing/browser.py:341: in wait_for_page
     self.driver.get(self.server_url if url is None else url)
 /usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:333: in get
     self.execute(Command.GET, {'url': url})
 /usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:321: in execute
     self.error_handler.check_response(response)
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f14d260fdc0>
 response = {'status': 500, 'value': '{"value":{"error":"unknown error","message":"unknown error: net::ERR_CONNECTION_REFUSED\\n  (Session info: chrome=90.0.4430.85)","stacktrace":"#0 0x563f07a8ee89 \\u003Cunknown>\\n"}}'}

Answer 1

I've been experiencing the exact same problems and error messages above while trying to implement some plotly dash tests in gitlab CI. Finally got it working, and the main difference with the OP is the use of docker compose with these containers instead of scripting it directly in gitlab.ci.yml:

• dash-test container, with entrypoint: pytest -v --remote --log-cli-level DEBUG , and network_mode: host
• selenium-grid container, with image: selenium/standalone-chrome and network_mode: host

Previous to the above setup, I had created a docker.network and exposed port 8050 on the dash-test container and port 4444 on the selenium-grid container. That's when I got the ERR_CONNECTION_REFUSED error. I suspect that the issue with this setup (and might be what's happening in gitlab ci) is this:

• The dash-test container can access selenium-grid with --remote-url http://selenium-grid:4444/wd/hub so selenium tests start running
• However, the dash testing library has http://localhost:8050/ hardcoded (ps dash maintainers, this should definitely changed and made configurable) which is inaccessible to the selenium-grid container when the tests run. On modifying the docker-compose file to use network_mode: host , that made it accessible as the containers were now using the host's.network.

docker-compose.yaml:

version: '3.7'
services:
    dash-test:
        image: "dash-test"
        build:
            context: ..
            dockerfile: docker/Dockerfile
            target: test
        entrypoint: pytest -v --remote --log-cli-level DEBUG -k bsly001
        network_mode: host
        depends_on:
            selenium-grid:
                condition: service_healthy
        mem_limit: 4096m
        volumes:
            - ../config:/app/config
            - ../dash_playground:/app/dash_playground
            - ../qa_reports:/app/qa_reports
    selenium-grid:
        image: selenium/standalone-chrome
        shm_size: 2gb
        network_mode: host
        healthcheck:
            test: /opt/bin/check-grid.sh --host 0.0.0.0 --port 4444
            interval: 5s
            timeout: 30s
            retries: 30

Answer 2

Your Gitlab runner should have access to the application you are trying to run from the Gitlab CI. As per the error its clear that Gitlab runner its not able to connect to the application url.

So please check and confirm whether the Gitlab runner have access to the host machine.