How to schedules jobs with specific parameters in a Jenkins multibranch pipeline

Question

We were having 2 FreeStyle projects on Jenkins:

One to generate builds(daily builds+manual builds), one other to execute tests.

We are moving to a Multibranch pipeline on jenkins, so my understanding is that we have one project per repository, and that we should use options to have different behavior.

So I can create parameters, to indicate if we want to run the tests, if we want to build the setups, that part I'm ok with it.

My issue is that I need that by default, the tests are NOT executed(because they take a lot of time to generate, and I don't want that developers can by mistake just let the "Execute tests" option checked.

And I need that this option is checked when executing the daily build in the night.

So 2 questions:

1. How to schedule?
2. How to provide the parameters value used for this schedule?

Answer 1

You can create a separate multi-branch job that will run by schedule and trigger your main job overriding all necessary parameters. It will look something like this

pipeline {
    agent any
    triggers {
        pollSCM('0 0 * * *')
    }
    stages {
        stage('Triggering the main job') {
            steps {
                build job: "main/${BRANCH_NAME.replace('/', '%2F')}", 
                      parameters: [string(name: 'RUN_TESTS', value: 'true')]
            }
        }
    }
}

You should put this file along with your main Jenkinsfile in the repository and configure a separate multi-branch pipeline job to use this file.

Answer 2

To keep this in the same job is going to require a little bit of groovy coding. Since your using MultiBranch pipeline this can all live in your Jenkinsfile

First, Set your cron as Vitalii mentions, this will kick of the job on schedule.

properties([
    pipelineTriggers([cron('0 0 * * *')])
])

Next, when this job is triggered by the schedule, we want to adjust the params its running with. So first we need to check what caused the build. This will probably require Security script approval.

List causes = currentBuild.rawBuild.getCauses().collect { it.getClass().getCanonicalName().tokenize('.').last() }

If this contains 'TimerTriggerCause' then we want to update the parameter.

if (causes.contains('TimerTriggerCause') { 
    setBooleanParam("EXECUTE_TESTS", true)
}

We wrote a function in a shared lib for this, you can put it in the same Jenkinsfile if you wish (At the bottom outside the pipeline logic):

/**
 * Change boolean param value during build
 *
 * @param paramName new or existing param name
 * @param paramValue param value
 * @return nothing
 */
Void setBooleanParam(String paramName, Boolean paramValue) {
    List<ParameterValue> newParams = new ArrayList<>();
    newParams.add(new BooleanParameterValue(paramName, paramValue))
    try {
        $build().addOrReplaceAction($build().getAction(ParametersAction.class).createUpdated(newParams))
    } catch (err) {
        $build().addOrReplaceAction(new ParametersAction(newParams))
    }
}

And let the job continue as normal. When it gets to evaluating params.EXECUTE_TESTS, this will now be true (instead of the default false).

Note: May need to import model for the value

import hudson.model.BooleanParameterValue

Putting that all together (Just cobbled the bits together quickly for an overall picture), Your jenkinsfile would end up something like this

#!groovy
import hudson.model.BooleanParameterValue


List paramsList = [
    choice(name: 'ACCOUNT_NAME', choices: ['account1', 'account2'].join('\n'), description: 'A choice param'),
    string(name: 'PARAM', defaultValue: 'something', description: 'A string param'),
    booleanParam(defaultValue: false, name: 'EXECUTE_TESTS', description: 'Checkbox'),
]

properties([
    buildDiscarder(logRotator(numToKeepStr: '20')),
    pipelineTriggers([cron('0 18 * * *')]), // 4am AEST/5am AEDT
    disableConcurrentBuilds(),
    parameters(paramList)
])


ansiColor {
    timestamps {
        node {
            try {
                causes = currentBuild.rawBuild.getCauses().collect { it.getClass().getCanonicalName().tokenize('.').last() }
                if (causes.contains('TimerTriggerCause') { 
                    setBooleanParam("EXECUTE_TESTS", true)
                }
                stage('Do the thing') {
                    // Normal do the things like build
                }
                stage('Execute tests if selected') {
                    if (params.EXECUTE_TESTS == true) {
                        // execute tests
                    } else {
                        echo('Tests not executed (Option was not selected/False)')
                    }
                }
            } catch (err) {
                throw err
            } finally {
                deleteDir()
            }
        }
    }
}

/**
 * Change boolean param value during build
 *
 * @param paramName new or existing param name
 * @param paramValue param value
 * @return nothing
 */
Void setBooleanParam(String paramName, Boolean paramValue) {
    List<ParameterValue> newParams = new ArrayList<>();
    newParams.add(new BooleanParameterValue(paramName, paramValue))
    try {
        $build().addOrReplaceAction($build().getAction(ParametersAction.class).createUpdated(newParams))
    } catch (err) {
        $build().addOrReplaceAction(new ParametersAction(newParams))
    }
}

Answer 3

Allow me to suggest a simpler approach. This is for declarative pipelines.

I would recommend keeping the tests (treated as first class code along with the application code) and application source code in same repository.

When Jenkins checks out from your SCM keeping them together in the one repository will allow you to apply a tag (label) when the test suite passes. Labels are your best friend and should be applied on success whenever possible. At time of writing multi-SCM checkouts don't seem to support application of labels, unfortunately.

I currently have a team of 5 developers and am currently using a multibranch pipeline to handle all the feature branches they are generating. We also have a 'master' and 'integration' branch. Master is for clean releases. Integration is my key branch.

Scheduling in a declarative pipeline is as simple as:

triggers { 
    cron('0 22 * * *') 
}

pollSCM doesn't seem to work as reliably for more complex crontabs.

One way you might like to consider introducing conditionality into your declarative pipeline is via branchname.

success {
    script {
        if( "${env.BRANCH_NAME}" == "integration" ) {
           //Create Package
           //Upload to Artifactory
           //Apply tag to Git
        }
    }
}

Feature branches in the above example are only executing the unit tests and providing feedback to the developers. It is only the integration branch that additionally on success generates an artifact (for later testing phases) and tags the repository.

If you don't want bifurcate behavior based on branch, I would suggest you have 2 different jobs (pipelines) defined in Jenkins: one to run every commit from the developers during the day and one that is scheduled to run nightly that executes your long running tests.

This is what I do with Unit tests and systems test.

Unit Test job is a multibranch pipeline that runs for every branch in the Enterprise GitHub repository. Polls every minute for changes with only the Integration branch creating the artifact and tagging. Unit tests take 10 minutes to run.

The System Test job is a simple pipeline that is scheduled to run nightly and it takes an hour or so to execute.

Answer 4

This doesn't map so well into multibranch pipeline. I've seen stages dependent on a branch, but not stages dependent on the parameter - it would break the stage view daily as well.

What I would recommend is to write two separate Jenkinsfiles, for example one called Jenkinsfile and second maybe Jenkinsnightlyfile . Then you can create two multibranch pipeline projects just using different jenkinsfile names on the same repository.

Put your regular stages in the first one, and all the tests in the other (you can divide the work here into multiple stages as well for clarity), be sure to have the other one use the appropriate triggers like for jenkins-pipeline:

properties([
    pipelineTriggers([cron('0 0 * * *')])
])

or for declarative pipeline:

triggers { 
    cron('0 0 * * *') 
}