Jenkins 用于 Python 与边车 MongoDB 的单元测试

Question

I am not a huge DevOps expert, but I am trying to configure a Jenkins pipeline to run my Python Flask application's Unittests, that rely on MongoDB.

Jenkins 在 Docker 内部运行。

我尝试了以下方法，但它一直挂起而没有终止，我不明白它出了什么问题：

pipeline {
    agent {
        docker {
            image 'python:3.9'
            args '--user 0:0'
        }
    }
    stages {
        stage('build') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'pip install -e .'
            }
        }
        stage('test') {
            steps {
                script {
                    node {
                        docker.image('mongo:4').withRun('-p 27017:27017') { c ->
                            sh 'coverage run web/test/main.py'
                        }
                    }
                }
            }
            post {
                always {
                    sh 'coverage report'
                }
            }
        }
    }
}

这是完整的日志（大约 5 分钟后它挂起，它输出的process apparently never started日志）：

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/test-github-action-ci-pipeline
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . python:3.9
.
[Pipeline] withDockerContainer
Jenkins seems to be running inside container 76b85911bd303d0e6220f124d6d36f0c387e529f65aa845486eefb6fcd6687d9
but /var/jenkins_home/workspace/test-github-action-ci-pipeline could not be found among []
but /var/jenkins_home/workspace/test-github-action-ci-pipeline@tmp could not be found among []
$ docker run -t -d -u 1000:1000 --user 0:0 -w /var/jenkins_home/workspace/test-github-action-ci-pipeline -v /var/jenkins_home/workspace/test-github-action-ci-pipeline:/var/jenkins_home/workspace/test-github-action-ci-pipeline:rw,z -v /var/jenkins_home/workspace/test-github-action-ci-pipeline@tmp:/var/jenkins_home/workspace/test-github-action-ci-pipeline@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** python:3.9 cat
$ docker top cbdd5ceb2bb4b31c21b19bcc292a1de499e92ce04c12f815a302d62cdbdd423c -eo pid,comm
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] sh
+ pip install -r requirements.txt
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/include/python3.9/UNKNOWN
sysconfig: /usr/local/include/python3.9
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Collecting flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting pymongo==3.11.3
  Downloading pymongo-3.11.3-cp39-cp39-manylinux2014_x86_64.whl (518 kB)
Collecting coverage==5.5
  Downloading coverage-5.5-cp39-cp39-manylinux2010_x86_64.whl (243 kB)
Collecting python-dotenv==0.17
  Downloading python_dotenv-0.17.0-py2.py3-none-any.whl (18 kB)
Collecting click>=5.1
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting Werkzeug>=0.15
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting Jinja2>=2.10.1
  Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting itsdangerous>=0.24
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-1.1.1-cp39-cp39-manylinux2010_x86_64.whl (32 kB)
Installing collected packages: MarkupSafe, Werkzeug, Jinja2, itsdangerous, click, python-dotenv, pymongo, flask, coverage
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/include/python3.9/UNKNOWN
sysconfig: /usr/local/include/python3.9
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Successfully installed Jinja2-2.11.3 MarkupSafe-1.1.1 Werkzeug-1.0.1 click-7.1.2 coverage-5.5 flask-1.1.2 itsdangerous-1.1.0 pymongo-3.11.3 python-dotenv-0.17.0
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
[Pipeline] sh
+ pip install -e .
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/include/python3.9/UNKNOWN
sysconfig: /usr/local/include/python3.9
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Obtaining file:///var/jenkins_home/workspace/test-github-action-ci-pipeline
Requirement already satisfied: flask in /usr/local/lib/python3.9/site-packages (from web==0.0.0) (1.1.2)
Requirement already satisfied: pymongo in /usr/local/lib/python3.9/site-packages (from web==0.0.0) (3.11.3)
Requirement already satisfied: coverage in /usr/local/lib/python3.9/site-packages (from web==0.0.0) (5.5)
Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.9/site-packages (from flask->web==0.0.0) (1.0.1)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.9/site-packages (from flask->web==0.0.0) (1.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.9/site-packages (from flask->web==0.0.0) (2.11.3)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.9/site-packages (from flask->web==0.0.0) (7.1.2)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.9/site-packages (from Jinja2>=2.10.1->flask->web==0.0.0) (1.1.1)
Installing collected packages: web
  Running setup.py develop for web
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/include/python3.9/UNKNOWN
sysconfig: /usr/local/include/python3.9
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Successfully installed web-0.0.0
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] script
[Pipeline] {
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/test-github-action-ci-pipeline@2
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
process apparently never started in /var/jenkins_home/workspace/test-github-action-ci-pipeline@2@tmp/durable-d6462e0e
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
Cannot contact : java.io.FileNotFoundException: File '/var/jenkins_home/workspace/test-github-action-ci-pipeline@2@tmp/durable-d6462e0e/output.txt' does not exist

没有 MongoDB 容器，它就像一个魅力。 以下示例运行良好：

pipeline {
    agent {
        docker {
            image 'python:3.9'
            args '--user 0:0'
        }
    }
    stages {
        stage('build') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'pip install -e .'
            }
        }
        stage('test') {
            steps {
                sh 'coverage run web/test/main.py'
            }
            post {
                always {
                    sh 'coverage report'
                }
            }
        }
    }
}

Answer 1

好的，我做到了。 这工作正常，即使这可能不是最好的解决方案（它看起来有点“hacky”）。

pipeline {
    agent any
    stages {
        stage('MongoDB Setup') {
            steps {
                script {
                    try {
                        sh 'docker stop mongo_unittest'
                        sh 'docker rm mongo_unittest'
                    } catch (err) {
                        echo 'docker mongo_unittest is not already running' // err.getMessage()
                    }
                }
                sh 'docker pull mongo:4'
                sh 'docker run -d --name mongo_unittest -p 27017:27017 mongo:4'
            }
        }
        stage('Run Flask Unittests') {
            agent {
                docker {
                    image 'python:3.9'
                    args '--user 0:0 --net host'
                }
            }
            steps {
                checkout scm
                sh 'pip install -r requirements.txt'
                sh 'pip install -e .'
                sh 'echo "MONGO_HOST=127.0.0.1" >> web/.env'
                sh 'coverage run web/test/main.py'
            }
            post {
                always {
                    sh 'coverage report'
                }
            }
        }
    }
    post {
        always {
            sh 'docker stop mongo_unittest'
            sh 'docker rm mongo_unittest'
        }
    }
}