Canonical setup of Gradle Build

Question

I want to set up a large-ish project, and I'm told that gradle is the way to do it.

I'm very confused by Gradle, and the entire system seems like a lot of magic, hand-waving, and knowledge that I don't want to read all 60 chapters of the Gradle guide to grok.

I'm going to end up with the following components:

• ProtoBuf files defining a bunch of messages
• Java Library 1 and tests (dependent on the ProtoBufs)
• Java Library 2 and tests (dependent on Java Library 1)
• Java Applications (dependent on Java Library 2)
• Android Application (dependent on Java Library 2)
• iOS Application
• Duplicates of all of the Java stuff, but for Python, C++, and Objective-C)

I want to be able to build and test everything in one large shot. So normally I'd build a tree like this:

project/ proto/
         lib1/ java/   src/
                       test/
               python/ src/
                       test/
               ...
         lib2/ java/   src/
                       test/
               python/ src/
                       test/
               ...
         app1/ java/   src/
                       test/
               python/ src/
                       test/
               ...
         app2/ java/   src/
                       test/
               python/ src/
                       test/
               ...
         android/ src/
                  test/
         iOS/ src/
              test/

I get that build iOS from gradle might not be possible, so I'm happy to ignore it for now.

Is this an appropriate structure to use? How do I structure and place my gradle.build files so that libraries can be used by other teams properly? How do I make sure my dependencies are tight so that libraries include only the minimum set of what they need to include?

Gradle build files seem to leave a bunch of potentially unused tasks littered around. Do I just try to ignore these?

Answer 1

The structure I resided on was based on grouping similar projects into subproject structures.

project/client/core/common-android
project/client/core/features-android
project/client/core/ui-android
project/client/app/client-android
project/server/admin
project/server/base
project/server/util
project/server/api-deployment
project/bundles/bundle1
project/bundles/bundle2
project/bundles/bundle3
project/bundles/bundle4

This structure make using subproject much more intuitive then flat directory structures. Now we can apply configurations to the subproject that are similar. My final project structure looks like this

------------------------------------------------------------
Root project
------------------------------------------------------------

Root project 'platform'
+--- Project ':client' - Client: Android core projects
|    +--- Project ':client:common-android' - Client: Common library for Android aar
|    +--- Project ':client:features-android' - Client: Features library for Android aar
|    +--- Project ':client:ui-android' - Client: UI library for Android aar
|    +--- Project ':client:app-android' - Client: Apk client Android apk
+--- Project ':bundles' - bundles: OSGi bundles container project
|    +--- Project ':bundles:bundle1' - bundles: OSGi bundle jar
|    +--- Project ':bundles:bundle2' - bundles: OSGi bundle jar
|    +--- Project ':bundles:bundle3' - bundles: OSGi bundle jar
|    +--- Project ':bundles:bundle4' - bundles: OSGi bundle jar
\--- Project ':server' - Server: Coriolis root project
     +--- Project ':server:admin' - Server: admin jar
     +--- Project ':server:base' - Server: jar base
     +--- Project ':server:apiDeployment' - Server: platform deployment war
     \--- Project ':server:util' - Server: utils jar

Since we don't want the parent projects to know as little as possible about their children we can let each grouping of projects configure itself during configuration via the settings.gradle

settings.gradle:

rootProject.name = 'platform'

Map<String, String> projectProperties = startParameter.getProjectProperties()
projectProperties.put('platform', true.toString())

// TODO: Make project imports smarter by removing hardcoding of paths

def subprojects = settingsDir.listFiles(new FileFilter() {
    @Override
    boolean accept(File file) {
        return file.isDirectory() && (file.name == 'client' || file.name == 'server')
    }
})

for (File file : subprojects) {
    println "Found subproject directory: $file.absolutePath"
    switch (file.name) {
        case 'client':
            def androidHome = 'ANDROID_HOME'
            // any non-null value will add android modules to the build.
            // Assumption is only a valid SDK location will be set.
            if (System.getenv(androidHome)) {
                def clientFile = new File("$file.absolutePath/core/childProjectSettings.gradle")
                if (clientFile.exists()) {
                    println "Adding android client"
                    include ':client'
                    project(":client").projectDir = clientFile.parentFile
                    apply from: clientFile.absolutePath
                }
            } else {
                println "WARNING: Environment variable {$androidHome} not set.  Not adding Android modules as they are " +
                        "impossible to build without the Android SDK being installed.  To install the Android SDK " +
                        "please see: http://developer.android.com/sdk/installing/index.html"
            }
            break
        case 'server':
            def serverFile = new File("$file.absolutePath/server/childProjectSettings.gradle")
            if (serverFile.exists()) {
                println "Adding server"
                include ':server'
                project(':server').projectDir = serverFile.parentFile
                apply from: serverFile.absolutePath
            }

            def bundlesFile = new File("$file.absolutePath/bundles/childProjectSettings.gradle")
            if (bundlesFile.exists() && !projectProperties.containsKey('noBundles')) {
                println "Adding osgi bundles"
                include ':bundles'
                project(':bundles').projectDir = bundlesFile.parentFile
                apply from: bundlesFile.absolutePath
            }
            break
        default:
            println "Unknown subproject found: $file.absolutePath"
    }
}

Now only subprojects that exist on the disk will be included, we can remove the remaining hardcoding for a more dynamic example but this is simpler. Then we create a file (in this example) childProjectSettings.gradle for each of our project groupings (client, server, bundles). Your childProjectSettings.gradle should specify it's subprojects in a way that it doesn't need to be updated every time a new subproject is added.

childProjectSettings.gradle:

File moduleSettingsDir = new File("$settingsDir.absolutePath/server", "bundles")

println "Bundles sees settings dir as: $moduleSettingsDir.absolutePath"

def bundleDirectories = moduleSettingsDir.listFiles(new FileFilter() {
    @Override
    boolean accept(File pathname) {
        return pathname.isDirectory()
    }
})

// get a reference to this project's descriptor so we can add subprojects
ProjectDescriptor bundles = project(':bundles')

bundleDirectories.each { File bundleDir ->
    if (new File(bundleDir, "build.gradle").exists()) {
        // normalize project names (blah-blah -> blahBlah)
        def bundleName = bundleDir.name
        if (bundleName.contains("-")) {
            def names = bundleDir.name.split("-")
            bundleName = names[0] + names[1].capitalize()
        }

        // include a subproject in the build
        include ":bundles:$bundleName"
        // default location will be wrong lets update the project's directory
        project(":bundles:$bundleName").projectDir = bundleDir
        // add the project as a subproject giving us better grouping
        bundles.children.add(project(":bundles:$bundleName"))
    }
}

project(':bundles').children.each {
    println "Parent {$it.parent} found child {$it} in path {$it.path} using buildScript {$it.buildFile $it.path}"
}

Gradle build files seem to leave a bunch of potentially unused tasks littered around. Do I just try to ignore these?

It's not the build files that create the tasks it's the plugins that are applied in the build.gradle files. To keep the tasks as tight as possible then only declare plugins in the build.gradle that it's actually used. Tasks are not inherited from dependency projects but declared dependencies are inherited from dependency projects.

*There is an important note here all dependencies are transitive by default so if ui depends on core which declares the gson dependency then ui will by default have gson in it's classpath.

For your question about the source folders a more gradle structure would probably look more like the below. Where your groups of apps would be lib1:java|lib1:python|lib2:java|lib2:python|app1:java|app1:python|app2:java|app2:python each of theses would be a subproject of their containing group.

Then lib1 project contains two subproject lib1:java and lib1:python each compiled with the plugins in their own build.gradle files. Common code can be placed in a custom plugin in the buildSrc if needed.

project/ proto/
         lib1/ java/   src/main/java/
                       src/main/javaTest/
               python/ src/main/python/
                       src/main/pythonTest/
               ...
         lib2/ java/   src/main/java/
                       src/main/javaTest/
               python/ src/main/python/
                       src/main/pythonTest/
               ...
         app1/ java/   src/main/java/
                       src/main/javaTest/
               python/ src/main/python/
                       src/main/pythonTest/
               ...
         app2/ java/   src/main/java/
                       src/main/javaTest/
               python/ src/main/python/
                       src/main/pythonTest/
               ...
         android/ src/
                  test/
         iOS/ src/
              test/