Working with maven and multiple git repositories - reducing the pain

Question

We recently migrated from SVN, with most code in a single repo, to git, with most projects in their own repos (about 70 of them). We build about a dozen different apps from this java source. The apps all run on *nix servers. We use maven and nexus to build. Many of us are struggling with developing features when that feature touches more than one repo. Here are a few of the challenges:

• The developer has to branch each repo separately - we use the same name for all branches for one feature to make tracking less difficult.

• One must update poms of all repos to point to the updated versions of each repo's artifact. If multiple people are working on the same branch, there can be a lot of merging others pom changes. When I commit a change to a repo, then the artifact is renamed to "-SNAPSHOT" which means more pom updates.

• Changes need to be pushed in the right order or our automated builds will fail, eg: repo A depends on a change to repo B; if repo A is pushed before repo B is built and deployed, then repo A won't build.

• The person reviewing the feature has to look at changes in multiple repos.

• When the feature is merged from its branch to, say, master, One has to remember all the repos that were touched.

It looks like switching to a mostly monorepo approach might be best, tho there are some drawbacks there:

• Building the entire codebase with maven takes a looong time. (Why can't maven be more like make, only building things that have changed or whose dependencies have changed?)

• Each push kicks off a big set of builds and many unit tests rather than just one repo's artifact build and test.

• The developers who generally work in one or two repos prefer this new multi-repo world and will resist a change back.

I've looked into git submodules and sub trees, which don't seem to solve many of our issues (Not sure about Google Repo). Some of us use tools like "mu" to help. It would be sweet if there was a toolkit that would help developers maintain versions in poms, and track changes across repos.

Let me know if you have a set of procedures or tools you use to ease development in this kind of environment.

Answer 1

with most projects in their own repos (about 70 of them).`

For me this is where the problems start. My vote goes for minimising this number significantly.

If you really don't want a single repo (1 repo gets my vote) then you could separate the code base into n*change_often repos with 1*change_rarely repo. Keeping the n small is important. This way you would avoid rebuilding the bits that change rarely.

Also, even with the a single repo you don't need to reference everything by source and use binaries for base libraries. When a base library changes the person making the change could also update all the references in one go so that that all projects are up to date.