How to reliably use a local Maven mirroring proxy with docker-compose?

Question

I am working to migrate our Maven build process into Docker for our existing Java projects. I am an experienced Java developer using Maven, but still learning Docker.

Our primary problem is that by default artifacts are not cached so we download half the Internet every five minutes, combined with we also have things on a local elderly Nexus server. As this is for automated integration testing we do not want to build the artifacts outside of docker.

After careful consideration I found that a good solution could be to have a Maven mirroring proxy running for each developer where the local configuration can be kept and the Docker instances then just refer to that. Manually running a Nexus3 container with

docker run -p 8081:8081 --name nexus sonatype/nexus3

(plus some manual configuration to proxy our internal repository too) has worked very well so far, but is not controlled by docker-compose.

Unfortunately I cannot see a way for the guests to access the host (where port 8081 is available) other than hardcoding the external IP-number of the machine, and I cannot see a way to make docker-compose responsible for ensuring that the nexus container is running when my docker builds needs it.

I am currently using this as the settings.xml file inside docker:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      https://maven.apache.org/xsd/settings-1.0.0.xsd">
    <mirrors>
        <mirror>
            <id>local-nexus</id>
            <name>Local Nexus</name>
            <!-- Host external IP number -->
            <url>http://1.2.3.4:8081/repository/sbforge_central/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>
    <localRepository>/usr/share/maven/ref/repository</localRepository>
</settings>

How should I approach this? I see two scenarios.

1. How can I get a reliable name for the host? Linux is required, other platforms will be nice.

2. How can I make docker-compose control this container so it is up when running docker-compose build so docker can ensure that the hostname of the nexus container resolves? It is not needed when bringing the finished containers up.

Or is there a smarter way?

Answer 1

I hope I got that right, but I see the following options to solve your problems:

1. [Updated] Use env parameter in settings.xml
Could you use an environment variable within your docker-compose.yml and set it in the settings.xml: ${env.EXTERNAL_NEXUS_HOST} ?

2. Nexus dependency

docker-compose build won't let you set conditions for other services, but you could use docker-compose run --rm and depends_on

version: '2'
services:
  # Whatever this might look like, it's just an example
  mvn:
    image: mvn
    depends_on:
      - nexus
    command: mvn deploy
  nexus:
    image: sonatype/nexus3

Does that solve anything or did I get you wrong?

Answer 2

As suggested offline by https://stackoverflow.com/users/4527948/asger-askov-blekinge the default network address of the host in a docker-compose controlled network is 172.17.0.1. The modified version of the pom.xml suggested for the maven:3-jdk-9-slim image (to move the local repository inside the image) then looks like:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      https://maven.apache.org/xsd/settings-1.0.0.xsd">
    <mirrors>
        <mirror>
            <id>local-nexus</id>
            <name>Local Nexus</name>
            <url>http://172.17.0.1:8081/repository/maven_central/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>
    <localRepository>/usr/share/maven/ref/repository</localRepository>
</settings>

This is the most useful to us when doing non-deployment builds, as we can then merge our inhouse repository with Maven Central easily in Nexus moving that complexity outside our builds.

Answer 3

Not an answer, but comments do not allow this size of comments:

Maybe I misunderstand a thing here: But what do you mean by `I am an experienced Maven developer?" ? Do you develop Maven plugins etc. ? Or are you a Java developer who is using Maven? Furthermore Why are the artifacts not cached if you build via Maven? Do you delete the local cache for each build ? Have you used a repository manager before ? And why don't you like to build artifacts outside Docker? Are you using a CI Server like Jenkins? Why using a repository manager in Docker but not mounted a volume for the respository data ? And where is the relationship to Docker-Compose? Apart from that docker-compose is not responsible for keeping a service up and available...

The configuration in your settings.xml looks like you are sharing the local cache somehow? I hope you don't do this...this will not work...

Update:

As I asked before why do you need to build inside a Docker Container ? It's much slower than outside and best is to use for such purposes a CI server.... docker-compose can't really orchestrate things cause it makes not sure that a service is up and running(That must be handled by your own service)

Furthermore you can't abstract the local cache aways cause you need it to build your jar/war/ear's...(BTW: I'm working with Docker for about 2 Years now;Working with Maven about 10 years; I'm a Apache Maven PMC Member)...

The point is you have to map the local cache of your Docker container on a volume which is not deleted after the Container is removed..So each time you start the "Build Container" you have to give the volume to to the Container...otherwise it is deleted (local cache inside the container) after you stop the container which is also true for your Nexus3 container...You have to map also a volume into the Nexus3 container otherwise the proxy idea is just smashed aways....

Some words about the idea with docke-compose. If you define things like a depends_on nexus this will start the Nexus3 container before the Maven execution but does not really wait until the Nexus3 is really usable..(this takes about 2-3 Minutes)...so in the end this kind of going will not work (cause usually a build faster than 2-3 minutes)...So you have to start the Nexus3 Container on a separate area (for example on a Mesos kind of environment) which is running always cause it's vital to your infrastructure to support Maven builds...I would also recommend to start using a CI solution like jenkins to handle all the builds...