Gradle docker build hangs indefinitely

Question

My build works randomly and hangs randomly even when run with exactly the same parameters. It's always getting stuck after my copy task (some parts replaced with )

task createDocker(group: '<placeholder>', type: Docker) {
    push = true
    dockerfile = file(stageDir.getAbsolutePath()+"/Dockerfile")
    project.group = "<placeholder>"
    applicationName = "<placeholder>-clusterservice"
    tagVersion = "${project.version}"

    doFirst {
        println ("Copying Dockerfile to staging area:" + stageDir.getAbsolutePath())
        copy {
            from "${project.projectDir}/src/main/docker/"
            include "**/*"
            into stageDir
        }

        println ("Getting artifact")
        copy {
            from "$distDir/${war.baseName}-${war.version}.war"
            rename "${war.baseName}-${war.version}.war", "clusterservice.war"
            into stageDir
        }
    }
}

I ran it with debug flags and this is what it looks like when it works:

[16:26:15] :     [Step 1/1] 16:25:49.730 [QUIET] [system.out] Getting artifact
[16:26:15] :     [Step 1/1] 16:25:50.041 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Execute doFirst {} action for :clusterservice:createDocker'
[16:26:15] :     [Step 1/1] 16:25:50.041 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Execute doFirst {} action for :clusterservice:createDocker' completed
[16:26:15] :     [Step 1/1] 16:25:50.041 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Execute build for :clusterservice:createDocker' started
[16:26:15] :     [Step 1/1] 16:25:50.041 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Setting up staging directory.
[16:26:15] :     [Step 1/1] 16:25:50.041 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Creating Dockerfile from file /opt/teamcity/agent/work/7fe8da50bfc4c49f/clusterservice/build-gradle/docker/Dockerfile.
[16:26:15] :     [Step 1/1] 16:25:50.045 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Determining image tag: <placeholder>/<placeholder>-clusterservice:8.23.0-SNAPSHOT
[16:26:15] :     [Step 1/1] 16:25:50.045 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Using the native docker binary.
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out] Sending build context to Docker daemon    194MB
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out] Step 1/33 : FROM <placeholder>/tomcat-openjre8:8.5.40
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out]  ---> aa259d3bf785
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out] Step 2/33 : LABEL MAINTAINER 'dev@<placeholder>.com'
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out]  ---> Using cache
[16:26:15] :     [Step 1/1] 16:26:05.213 [QUIET] [system.out]  ---> 14bff4363c7f
... 

Then it continues to build normally with no errors!

Here's the a failing build run on the exact same machine and exact same parameters

[15:53:03] :     [Step 1/1] 15:52:37.030 [QUIET] [system.out] Getting artifact
[15:53:03] :     [Step 1/1] 15:52:38.608 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Execute doFirst {} action for :clusterservice:createDocker'
[15:53:03] :     [Step 1/1] 15:52:38.609 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Execute doFirst {} action for :clusterservice:createDocker' completed
[15:53:03] :     [Step 1/1] 15:52:38.609 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Execute build for :clusterservice:createDocker' started
[15:53:03] :     [Step 1/1] 15:52:38.609 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Setting up staging directory.
[15:53:03] :     [Step 1/1] 15:52:38.612 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Creating Dockerfile from file /opt/teamcity/agent/work/7fe8da50bfc4c49f/clusterservice/build-gradle/docker/Dockerfile.
[15:53:03] :     [Step 1/1] 15:52:38.645 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Determining image tag: <placeholder>/<placeholder>-clusterservice:8.23.0-SNAPSHOT
[15:53:03] :     [Step 1/1] 15:52:38.645 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Using the native docker binary.
[15:53:03] :     [Step 1/1] 15:53:03.859 [LIFECYCLE] [org.gradle.process.internal.health.memory.MemoryManager] 
[15:53:03] :     [Step 1/1] 15:53:03.859 [DEBUG] [org.gradle.process.internal.health.memory.MemoryManager] Emitting OS memory status event {Total: 33737138176, Free: 26021330944}
[15:53:03] :     [Step 1/1] 15:53:03.859 [DEBUG] [org.gradle.launcher.daemon.server.health.LowMemoryDaemonExpirationStrategy] Received memory status update: {Total: 33737138176, Free: 26021330944}
[15:53:03] :     [Step 1/1] 15:53:03.859 [DEBUG] [org.gradle.process.internal.health.memory.MemoryManager] Emitting JVM memory status event {Maximum: 1908932608, Committed: 1186463744}
[15:53:08] :     [Step 1/1] 15:53:08.859 [DEBUG] [org.gradle.process.internal.health.memory.MemoryManager] Emitting OS memory status event {Total: 33737138176, Free: 26022682624}
[15:53:08] :     [Step 1/1] 15:53:08.859 [DEBUG] [org.gradle.launcher.daemon.server.health.LowMemoryDaemonExpirationStrategy] Received memory status update: {Total: 33737138176, Free: 26022682624}
[15:53:08] :     [Step 1/1] 15:53:08.859 [DEBUG] [org.gradle.process.internal.health.memory.MemoryManager] Emitting JVM memory status event {Maximum: 1908932608, Committed: 1186463744}
[15:53:11] :     [Step 1/1] 15:53:11.450 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] DaemonExpirationPeriodicCheck running
[15:53:11] :     [Step 1/1] 15:53:11.450 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
[15:53:11] :     [Step 1/1] 15:53:11.450 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
...

It doesn't error out, and the health check repeats and runs indefinitely (I let it run over 2 hours) and it just continues to hang without any errors. I have no clue what's wrong with it, the build runs completely fine if I run the commands manually. If it works the build takes around 7 minutes or so. Please help.

Answer 1

It appears that development on the project for this specific docker gradle plugin has been abandoned.

If you check the issue on this github page here https://github.com/Transmode/gradle-docker/issues/119 and here https://github.com/Transmode/gradle-docker/issues/37 you will see that this issue was reported years ago but no fix has ever been merged in.

There are other gradle docker plugins that are available that are a bit more active in development which you could use. This one https://github.com/bmuschko/gradle-docker-plugin comes to mind.

Personally in the end after getting incredibly fed up with all of this I just rewrote everything in a bash script and used gradle to call said script in order to do my necessary docker configuration.