在 Github 操作/工作流中使用 ddev

Question

I have acceptance tests set up with ddev. They get run locally on ddev composer cookieman:test . I would like to use the same setup with Github actions.

Did anybody have any luck with ddev in Github actions/workflow? I am getting until here where ddev's healthcheck fails:

...
Creating ddev-router ... done

Failed to start extension-cookieman-master: ddev-router failed to become ready: logOutput=2019/11/15 02:24:19 [emerg] 1630#1630: no servers are inside upstream in /etc/nginx/conf.d/default.conf:89
nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:89
nginx: configuration file /etc/nginx/nginx.conf test failed
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
ddev-router healthcheck endpoint not responding 
, err=container /ddev-router unhealthy: 2019/11/15 02:24:19 [emerg] 1630#1630: no servers are inside upstream in /etc/nginx/conf.d/default.conf:89
nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:89
nginx: configuration file /etc/nginx/nginx.conf test failed
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
ddev-router healthcheck endpoint not responding 

##[error]Process completed with exit code 1.

.github/workflows/tests.yml:

name: Tests
on: [push, pull_request]
jobs:
    tests-via-ddev:
        runs-on: ubuntu-latest
        steps:
            -   uses: actions/checkout@v1
            -   run: export DEBIAN_FRONTEND=noninteractive
            # update docker
            -   run: sudo -E apt-get purge -y docker docker-engine docker.io containerd runc nginx
            -   run: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
            -   run: sudo -E add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
            -   run: sudo -E apt-get update
            -   run: sudo -E apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
            # install linuxbrew
            -   run: sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"
            -   run: echo "::add-path::/home/linuxbrew/.linuxbrew/bin"
            # install ddev + docker-compose
            -   run: brew tap drud/ddev && brew install ddev docker-compose
            # Start ddev
            -   run: ddev start || exit 0
            # Debug
            -   run: ls -als .ddev/
            -   run: curl 127.0.0.1 || exit 0
            -   run: curl 127.0.0.1/healthcheck || exit 0
            -   run: docker ps || exit 0
            # we want Clover coverage
            -   run: ddev exec enable_xdebug
            # Run tests
            -   run: ddev composer cookieman:test

I tried

• using Ubuntu 16.04

• fully upgrading all packages on Ubuntu 16.04/18.04

• configuring ddev like that:

  • run: ddev config global --router-bind-all-interfaces=true
  • run: ddev config global --omit-containers=dba,ddev-ssh-agent

• changing to unprivileged router ports (settings router_http_port, router_https_port in config.yaml)

If I force it to continue with ddev start || exit 0 ddev start || exit 0 I can see containers up and running:

- run: docker ps || exit 0

CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS                      PORTS                                                                                                    NAMES
c36601a06fd6        drud/ddev-router:v1.11.0                "/app/docker-entrypo…"   27 seconds ago      Up 24 seconds (unhealthy)   0.0.0.0:4430->4430/tcp, 0.0.0.0:4444->4444/tcp, 0.0.0.0:8025->8025/tcp, 80/tcp, 0.0.0.0:8080->8080/tcp   ddev-router
18152602a054        drud/ddev-webserver:v1.11.0-built       "/start.sh"              30 seconds ago      Up 28 seconds (healthy)     8025/tcp, 127.0.0.1:32770->80/tcp, 127.0.0.1:32769->443/tcp                                              ddev-extension-cookieman-master-web
33aca55715f2        selenium/standalone-chrome:3.12         "/opt/bin/entry_poin…"   32 seconds ago      Up 30 seconds               4444/tcp                                                                                                 ddev-extension-cookieman-master-chrome
6c852ae62974        drud/ddev-dbserver:v1.11.0-10.2-built   "/docker-entrypoint.…"   32 seconds ago      Up 30 seconds (healthy)     127.0.0.1:32768->3306/tcp                                                                                ddev-extension-cookieman-master-db

curl 127.0.0.1 yields the default nginx start page (while I would expect '503: No ddev back-end site available')

curl 127.0.0.1/healthcheck yields a 404

So far my conclusion is: ddev-router is reachable but its nginx does not have the appropriate configuration (no servers are inside upstream in /etc/nginx/conf.d/default.conf). Thus ddev only runs the pre-start hook form config.yaml. post-start is not reached.

You can see the output of the last runs here https://github.com/dmind-gmbh/extension-cookieman/actions?query=branch%3Afeat%2Facceptance-tests

EDIT/AMEND: This is the (mal-)generated /etc/nginx/conf.d/default.conf from ddev-router :

# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
  default $http_x_forwarded_proto;
  ''      $scheme;
}
# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
  default $http_x_forwarded_port;
  ''      $server_port;
}
# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
  default upgrade;
  '' close;
}
# Apply fix for very long server names
server_names_hash_bucket_size 128;
# Default dhparam
# ssl_dhparam /etc/nginx/dhparam/dhparam.pem;
# Set appropriate X-Forwarded-Ssl header
map $scheme $proxy_x_forwarded_ssl {
  default off;
  https on;
}

rfay mentioned a miscommunication between the ddev-router and the underlying docker daemon via sockets.

Answer 1

EDIT: I put my findings into a Github action that can be included in other projects, too: https://github.com/marketplace/actions/setup-ddev

I came to the conclusion that the problem is with docker-gen.

In the first line of the template ( https://github.com/drud/ddev/blob/master/containers/ddev-router/nginx.tmpl or also jwilder's https://github.com/jwilder/nginx-proxy/blob/master/nginx.tmpl ) the.Docker.CurrentContainerID is empty which seemed to happen to some people in some contexts https://github.com/jwilder/docker-gen/issues/196#issuecomment-225412753 .

The suggested removal of '-only-exposed' did not work for me. I changed the template a bit instead to not rely on the container and that was it.

:)

This is still a bit dirty and only a Proof-of-concpet:

• this is the changed template https://github.com/jonaseberle/github-action-setup-ddev/blob/master/.ddev/patches/ddev-router/nginx.tmpl (compare the upstream {} section where I removed the check if containers are on the same network with the router)

• In the workflow I do

  • ddev start || exit 0 ddev start || exit 0 # this will fail and also not execute any post-start hooks
  • docker cp nginx-debug.tmpl ddev-router:/app/nginx-debug.tmpl
  • docker exec ddev-router sh -c "docker-gen -only-exposed -notify 'sleep 1 && nginx -s reload' /app/nginx-debug.tmpl /etc/nginx/conf.d/default.conf"
  • ... ddev is now up and healthy

Not decided how to move on from here. Maybe @rfay would have an idea how to change the nginx-template. Or I will use a custom Dockerfile for the ddev-router with a docker-compose.ddev-router.yaml to change the file just for the Github actions run...

EDIT/AMEND:

The shorter and tested version of this is:

• ddev start || docker cp.ddev/patches/ddev-router/nginx.tmpl ddev-router:/app/nginx.tmpl
• ddev start - this triggers a container restart and thus a docker-gen run