How to speed up maven-deploy-plugin to deploy artifacts to the Artifactroy?

Question

I'm using Maven version 3.5.4 with maven-deploy-plugin version 2.8.2 to deploy my artifacts to Artifactory version 6.3.2 .

When I execute the mvn deploy -DdeployAtEnd=true , it takes times for each module around 2+ minutes. I have 15 modules which takes overall times around 30+ minutes.

Do we have any way to speed up this process?

Here is my Maven information

Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 
    2018-06-18T01:33:14+07:00)
Maven home: C:\Java.Application\Apache\apache-maven-3.5.4\bin\..
Java version: 1.8.0_181, vendor: Oracle Corporation, 
    runtime: C:\Java.Application\Sun\Java\jdk1.8.0_181\jre
Default locale: en_US, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

The Artifactory artifactroy.log

2018-08-30 15:08:16,496 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:08:19,550 - Deploy to 'core-snapshot-local:/some/artifact/maven-metadata.xml' Content-Length: 416
2018-08-30 15:08:22,677 - Deploy to 'core-snapshot-local:/some/artifact/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:08:25,731 - Deploy to 'core-snapshot-local:/some/artifact/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:08:28,800 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-javadoc.jar' Content-Length: 110853
2018-08-30 15:08:32,016 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-javadoc.jar.sha1' Content-Length: 40
2018-08-30 15:08:35,100 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-javadoc.jar.md5' Content-Length: 32
2018-08-30 15:08:38,182 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml' Content-Length: 1868
2018-08-30 15:08:41,413 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:08:42,249 - Indexing archive: core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7.jar
2018-08-30 15:08:42,336 - Indexing archive: core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-javadoc.jar
2018-08-30 15:08:44,462 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:08:47,520 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-javadoc.jar' Content-Length: 221096
2018-08-30 15:08:50,731 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-javadoc.jar.sha1' Content-Length: 40
2018-08-30 15:08:53,816 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-javadoc.jar.md5' Content-Length: 32
2018-08-30 15:08:56,912 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml' Content-Length: 1868
2018-08-30 15:08:59,973 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:09:03,027 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:09:06,084 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-sources.jar' Content-Length: 19103
2018-08-30 15:09:09,293 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-sources.jar.sha1' Content-Length: 40
2018-08-30 15:09:12,886 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-sources.jar.md5' Content-Length: 32
2018-08-30 15:09:16,032 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml' Content-Length: 1868
2018-08-30 15:09:19,086 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:09:22,136 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:09:25,196 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-sources.jar' Content-Length: 66693
2018-08-30 15:09:28,455 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-sources.jar.sha1' Content-Length: 40
2018-08-30 15:09:31,573 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-sources.jar.md5' Content-Length: 32
2018-08-30 15:09:34,665 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml' Content-Length: 1868
2018-08-30 15:09:37,717 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:09:40,771 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32
2018-08-30 15:09:42,250 - Indexing archive: core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-javadoc.jar
2018-08-30 15:09:42,499 - Indexing archive: core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-sources.jar
2018-08-30 15:09:42,565 - Indexing archive: core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-sources.jar
2018-08-30 15:09:43,832 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-tests.jar' Content-Length: 60117
2018-08-30 15:09:47,060 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-tests.jar.sha1' Content-Length: 40
2018-08-30 15:09:50,309 - Deploy to 'core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-tests.jar.md5' Content-Length: 32
2018-08-30 15:09:53,402 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml' Content-Length: 1868
2018-08-30 15:09:56,458 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1' Content-Length: 40
2018-08-30 15:09:59,509 - Deploy to 'core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.md5' Content-Length: 32

The Artifactory access.log

2018-08-30 15:07:48,401 [ACCEPTED DOWNLOAD] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:07:48,456 [ACCEPTED DOWNLOAD] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml.sha1 for some-user/some-ip.
2018-08-30 15:07:51,641 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:07:51,642 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7.jar for some-user/some-ip.
2018-08-30 15:08:00,997 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:00,997 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7.pom for some-user/some-ip.
2018-08-30 15:08:01,079 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:07,271 [ACCEPTED DOWNLOAD] core-snapshot-local:/some/artifact/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:07,328 [ACCEPTED DOWNLOAD] core-snapshot-local:/some/artifact/maven-metadata.xml.sha1 for some-user/some-ip.
2018-08-30 15:08:19,594 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:28,919 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:28,920 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-javadoc.jar for some-user/some-ip.
2018-08-30 15:08:47,637 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:08:47,639 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-javadoc.jar for some-user/some-ip.
2018-08-30 15:09:06,192 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:09:06,193 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-sources.jar for some-user/some-ip.
2018-08-30 15:09:25,295 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:09:25,297 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-test-sources.jar for some-user/some-ip.
2018-08-30 15:09:43,950 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/maven-metadata.xml for some-user/some-ip.
2018-08-30 15:09:43,951 [ACCEPTED DEPLOY] core-snapshot-local:/some/artifact/some/version/my-artifact-1.0.2-20180830.080748-7-tests.jar for some-user/some-ip.

EDIT 1

Previously I've found some issue about Maven 3.5.2 here which give me some background about the different between Maven 3.3.9 and 3.5.x , the wagon version.

• Maven 3.3.9 use wagon version 2.10
• Maven 3.5.4 use wagon version 3.1.0

Again, I copy the maven_home/lib from Maven 3.3.9 to Maven 3.5.4 . Everything back to be fast, totally 2-3 minutes.

I also have a chance to copy maven_home/lib from Maven 3.5.4 to Maven 3.3.9 . The process is slow again.

EDIT 2

Even using wagon version 2.10 as an interim solution seems to work, I also open for the proper way to work with Maven 3.5.4 and hope that you may shed the light.

Answer 1

Since the wagon version 2.10 seems to be better, then the interim solution is adding those wagon as an extension at my pom.xml as the following: -

<properties>
    <my.wagon.vesion>2.10</my.wagon.vesion>
</properties>

<build>
    <extensions>
        <extension>
            <groupId>org.apache.maven.wagon</groupId>
            <artifactId>wagon-file</artifactId>
            <version>${my.wagon.vesion}</version>
        </extension>
        <extension>
            <groupId>org.apache.maven.wagon</groupId>
            <artifactId>wagon-http-shared</artifactId>
            <version>${my.wagon.vesion}</version>
        </extension>
        <extension>
            <groupId>org.apache.maven.wagon</groupId>
            <artifactId>wagon-http</artifactId>
            <version>${my.wagon.vesion}</version>
        </extension>
        <extension>
            <groupId>org.apache.maven.wagon</groupId>
            <artifactId>wagon-provider-api</artifactId>
            <version>${my.wagon.vesion}</version>
        </extension>
    </extensions>
</build>

Answer 2

Do not use -DdeployAtEnd with the regular maven deploy plugin.

It is slow because it will just upload the modules in series instead of in parallel.

Instead I recommend you two step your CI to emulate -DdeployAtEnd .

mvn install
mvn -T 4C deploy -Dmaven.install.skip=true -DskipTests=true #basically skip everything but jar.

You can put -T to whatever you think that will allow as much uploading to happen.

It is still not ideal as the maven deploy plugin will only parallelize so much of the above.

An alternate solution is to try the nexus deploy plugin which basically "stages" the deployment and has faster ways to upload the whole thing at once but this generally require a nexus repo IIRC.