Reusing stages of a jenkins pipeline in multiple jobs

Question

My team is moving to Jenkins 2 and I am using the pipeline plugin so that our build can live in our repository. Because getting repositories allocated has lots of overhead in our company we have a single respository with many sub-projects & sub-modules in it.

What I want is separate builds and reporting of Junit/checkstyle/etc reports for each sub-module as well as a final "build and deploy" step for each sub-project putting it all together.

My current plan is to create separate jobs for each sub-module so that they get their own junit/checkstyle/etc reports page. Then have a multi-job project to orchestrate the sub-module builds for the sub-projects. Since all of the sub-projects are simple jar builds, I want to put bulk of the logic in a common file, lets call it JenkinsfileForJars at the root of the sub-project. So the repo structure is

• sub-project
  • JenkinsfileForJars.groovy
  • sub-moduleA
    • Jenkinsfile
  • sub-moduleB
    • Jenkinsfile

My Jenkinsfile contains

def submoduleName = "submoduleA"
def pipeline
node {

    pipeline = load("${env.WORKSPACE}/subproject/JenkinsfileForJars.groovy")

}
pipeline.build()
pipeline.results()

And my JenkinsfileForJars contains

def build() {

    stage('Build') {
        // Run the maven build
        dir("subproject") {
            sh "./gradlew ${submoduleName}:build"
        }

    }
}
def results() {

    stage('Results') {
        dir("subproject/${submoduleName}") {
            junit 'build/test-results/TEST-*.xml'
            archive 'build/libs/*.jar'
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/cobertura/', reportFiles: 'frame-summary.html', reportName: 'Cobertura Report'])
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/findbugs/', reportFiles: 'main.html', reportName: 'Fidbugs Report'])
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/pmd/', reportFiles: 'main.html', reportName: 'PMD Report'])
            step([$class: 'CheckStylePublisher', pattern: 'build/reports/checkstyle/main.xml', unstableTotalAll: '200', usePreviousBuildAsReference: true])
        }
    }

}

return this;

When I run the Jenkinsfile above I get the following error:

Running on master in /var/lib/jenkins/workspace/jobA
[Pipeline] {
[Pipeline] load
[Pipeline] { (/var/lib/jenkins/workspace/jobA/subproject/JenkinsfileForJars.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.NullPointerException: Cannot invoke method build() on null object

As far as I can tell, I am following what is shown in the documents for loading manual scripts and the example given for a loaded script. I do not understand why my script is null after the load command.

How do I get my Jenkinsfile to load JenkinsfileForJars.groovy?

Answer 1

The problem is related to the SCM checkout as mentioned by Blake Mitchell in the comment above. Since you are loading your groovy functions from a submodule, you will need to checkout the submodule first, preferably on a build agent /slave, if you would like to keep only bare repos on the master.

def pipeline
node( 'myAgentLabel' ) {
    stage ( 'checkout SCM' ) {
        checkout([
            $class: 'GitSCM'
            ,branches: scm.branches
            ,extensions: scm.extensions 
                + [[ $class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: true, reference: '', trackingSubmodules: false]]
            ,doGenerateSubmoduleConfigurations: false
            ,userRemoteConfigs: scm.userRemoteConfigs
        ])
        pipeline = load( "${env.WORKSPACE}/path/to/submodule/myGroovyFunctions.grooovy" )
    }
    pipeline.build()
}

Note that in the checkout example, access to scm.* attributes also needs to be whitelisted by an administrator in Jenkins (In-process script approval)

Answer 2

There might be two possible problems:

• Why do you put the load in a node structure. This import does not need computational resources, so you do not need it there.
• The call to build should be put inside a node structure. And probably the call to result should also be inside (the same) node structure to make sure, that the correct results are archived (if you use more than one (slave) nodes).

(This should probably be a comment below your question but I do not have enough points to add a comment there.)