Android Module specific libraries in Gradle

Question

I have an android module , core, which I use in a few projects. The core module has unit tests (as it should). The core module also has dependencies on a few external jars. The module itself builds fine and I get a nice shiny AAR, but I can't build the tests. Gradle (dex, actually) complains that certain libs are included multiple times. The tests use a custom test runner and that is their only jar dependency.

My directory hierarchy is

├── core
│   ├── libs
│   └── src
│       ├── instrumentTest
│       │   ├── assets
│       │   ├── java
│       │   └── res
│       └── main
│           ├── java
│           └── res
└── gradle
    └── wrapper

and my build.gradle is

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:0.6.+'
    }
}
apply plugin: 'android-library'

repositories {
    mavenCentral()
}

android {
    compileSdkVersion 18
    buildToolsVersion "18.1.0"

    defaultConfig {
        minSdkVersion 8
        targetSdkVersion 18
    }
}

dependencies {
    compile 'org.projectlombok:lombok:1.12.2'
    compile fileTree(dir: 'libs', include: '*.jar')
}

I believe that my solution is to have this directory hierarchy:

├── core
│   └── src
│       ├── instrumentTest
│       │   ├── assets
│       │   ├── java
│       │   ├── libs
│       │   └── res
│       └── main
│           ├── java
│           ├── libs
│           └── res
└── gradle
    └── wrapper

and include the jars separately in the main and instrumentTest modules. But I'm not sure what I have to do in my build.gradle file. I've tried adding a sourceSets block inside the android block with main and instrumentTest blocks inside, each with their own dependencies section:

sourceSets {
    main {
        dependencies {
            compile fileTree(dir: 'main/libs', include: '*.jar')
        }
    }
    instrumentTest {
        dependencies {
            compile fileTree(dir: 'instrumentTest/libs', include: '*.jar')
        }
    }
}

but gradle doesn't seem to pick it up at all (a lot of unresolved names).

The compileTest target works fine and completes successfully, but the connectedInstrumentTest producese the following errors:

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lorg/apache/http/entity/mime/MultipartEntity;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lcom/google/gson/JsonParseException;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Llombok/AccessLevel;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lorg/apache/commons/io/ByteOrderMark;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)
UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lorg/apache/log4j/Appender;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)
UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Llombok/AccessLevel;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lorg/apache/commons/lang3/CharRange$1;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lorg/apache/commons/collections/ArrayStack;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lde/mindpipe/android/logging/log4j/LogCatAppender;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lcom/google/i18n/phonenumbers/AlternateFormatsCountryCodeSet;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Lcom/zutubi/android/junitreport/JUnitReportListener$FilteringWriter;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:163)
    at com.android.dx.command.dexer.Main.processClass(Main.java:490)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:459)
    at com.android.dx.command.dexer.Main.access$400(Main.java:67)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:398)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)
    at com.android.dx.command.dexer.Main.processOne(Main.java:422)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:333)
    at com.android.dx.command.dexer.Main.run(Main.java:209)
    at com.android.dx.command.dexer.Main.main(Main.java:174)
    at com.android.dx.command.Main.main(Main.java:91)        

What can I do here to get rid of these errors?

Answer 1

The problem in this specific case was that there was already a lombok.jar in the libs folder, which conflicted with the declared dependency. I ended up deleting the jar and opting to use the downloaded version.

Another instance of this that hit me today was because the android plugin was finding the same file in multiple jar files. For example I have a few dependencies on Apache commons libs. These files share the same license, a notice file, etc. The android plugin was throwing a similar error.

as of android plugin 0.7.1 you can manually exclude these files:

packagingOptions {
    exclude 'META-INF/NOTICE.txt'
    exclude 'META-INF/LICENSE.txt'
    exclude 'META-INF/services/javax.annotation.processing.Processor'
}

Answer 2

When you are facing packages double inclusion, you can try the following:

compile ('org.roboguice:roboguice:2.0') {
    exclude module: 'guice'
}

In this case, RoboGuice tries to package Guice, so if you have already added it you will have this problem. Somehow, I think when you call connectedInstrumentTest the packaging is not done exactly as when you call just assemble or assembleRelease