Jenkins throwing Serialization exception

Question

I am getting below exception in a Jenkins pipeline, the first line "writeFile" where I am updating template using SimpleTemplateEngine which is successful and after this I am running bat command to move some files. After the exception I have posted some part of code which is getting ivoked from Jenkins.

[Pipeline] writeFile
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
an exception which occurred:
in field delegate
in field delegate
in field delegate
in field delegate
in field delegate
in field delegate
in field locals
in field caller
in field e
in field program
in field threads
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@1ae5783
Caused: java.io.NotSerializableException: groovy.lang.Binding
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.HashMap.internalWriteEntries(HashMap.java:1785)
at java.util.HashMap.writeObject(HashMap.java:1362)
at sun.reflect.GeneratedMethodAccessor122.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.TreeMap.writeObject(TreeMap.java:2438)
at sun.reflect.GeneratedMethodAccessor151.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:140)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:458)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:434)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:422)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:362)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Finished: FAILURE

Part of source code -

   ........
       if (fileExists("${templatePath}")) {
            createInstallCmdFile(templatePath, finalTemplate, exeName)

            bat """ 
            copy /Y "${finalTemplate}" "${shareDir}\\" || exit /b 1 
            """
       }
    }

    def createInstallCmdFile(def template, def target, def exeName) {
    def installerTemplate=readFile "${template}"
    def binding=[
            'exeName': exeName
    ]
    def engine=new groovy.text.SimpleTemplateEngine()
    def tpl=engine.createTemplate(installerTemplate).make(binding)
    writeFile file: "${target}", text: "${tpl}".toString()
    }

    return this

Answer 1

Without showing your whole pipeline, it is difficult to say exactly where this is happening. Jenkins can pause pipelines and requires that any state, which includes local variables and other things, must be Serializable . The CPS engine underlying Jenkins pipelines requires this for a lot of reasons, such as picking up pipelines in the case of an agent/master disconnect. Some pipeline code can be wrapped in a @NonCPS to not be transformed by the CPS engine.

One way to detect exactly where it is happening is to add a sleep step (like sleep(time: 1, unit: 'MILLISECONDS') in different places and gradually move it earlier to detect which line of code is causing the serialization problem if the stacktrace does not point you to the correct place.

The most glaring object in your example is def engine=new groovy.text.SimpleTemplateEngine() , since groovy.text.SimpleTemplateEngine() is not Serializable .

Answer 2

As @mkobit says, groovy.text.SimpleTemplateEngine is not Serializable. For me, simply switching to groovy.text.StreamingTemplateEngine avoids the serialization problem and provides the functionality needed.

FWIW - I discovered this option in the Groovy docs: http://docs.groovy-lang.org/docs/next/html/documentation/template-engines.html#_template_framework and then experimenting with the StreamingTemplateEngine . Some of the other engines listing in those docs may also work - but I didn't try them after finding one that was suitable for my needs. Please add a comment if you conclusively determine that any of the others do, or do not, work.

As reference, here's the pipeline script I used to verify:

node {
    stage('template') {
        def raw      = 'b:"${b}"'
        def dict     = [:]
        def engine   = new groovy.text.StreamingTemplateEngine()
        def template = engine.createTemplate(raw).make(dict)

        dict.b = 'BBB'
        echo template.toString()

        dict.b = 'bbb'
        echo template.toString()
    }
}

And here's the output generated (with some "personal" values hidden):

Started by user USER-NAME
[Pipeline] node
Running on JENKINS-AGENT in /opt/jenkins-slave/workspace/JOB-PATH/zz-test@2
[Pipeline] {
[Pipeline] stage
[Pipeline] { (template)
[Pipeline] echo
b:"BBB"
[Pipeline] echo
b:"bbb"
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS