How to prevent from C++ to recompile all files in Android NDK?

Question

Hello,

I'm using the Android Studio with NDK and JNI in a project with a large amount of C++ files. When I make changes in a single C++ file it won't apply in the code unless I rebuild the whole project and refresh the entire C++ files so they have to recompile. The compilation process takes more than 3 minutes for every small change, make it 20 times a day and you have lost an hour.

According to today, after I make a change to a file I go to

Build >> Refresh Linked C++ Projects,

and then run the project, resulting in a full, redundant compilation of all files.

I'm looking for way for the compiler to refresh only the changed file, and as a result shorten the build process.

NOTE: This problem only occurs in windows, when I run Android Studio on a Mac, the compiler recompiles only the relevant files.

This is my CMakeLists.txt file:

cmake_minimum_required(VERSION 3.4.1)

FILE(GLOB CPP_SRC
     "src/main/cpp/*.c"
     "src/main/cpp/*.h"
     "src/main/cpp/*.cpp"
     "src/main/cpp/*.hpp"
 )


add_library(MyLib

             SHARED
             ${CPP_SRC} )

find_library(
              log-lib
              log )

target_link_libraries(
                       MyLib

                       ${log-lib} )

target_link_libraries(MyLib
                      android
                      log
                      EGL
                      GLESv2)

And my gradle.build file:

apply plugin: 'com.android.library'

android {
    signingConfigs {
        config {
            keyAlias '*****'
            keyPassword '*****'
            storeFile file(*****)
            storePassword '*****'
        }
    }
    compileSdkVersion 27
    buildToolsVersion '27.0.3'
    defaultConfig {

    minSdkVersion 16
    targetSdkVersion 27
    versionCode 1

    versionName "1.0"
    testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

    externalNativeBuild {
        cmake {

            arguments "-DANDROID_ARM_NEON=TRUE", "-DANDROID_TOOLCHAIN=clang", "-DANDROID_STL=c++_shared", "-DCMAKE_BUILD_TYPE=Release", "-DANDROID_CPP_FEATURES=rtti exceptions"
            cppFlags "-D__STDC_FORMAT_MACROS", '-Ofast', '-fsigned-char', "-std=c++14", "-frtti", "-fexceptions", "-mtune=arm7", "-mfpu=vfpv3-d16", "-mfloat-abi=softfp", "-Wall",
                    "-DCOMPILE_EUROPE_ID_AND_FACE_OCR_MANAGER",
                    "-DCOMPILE_FRENCH_PASSPORT_SIGNATURE",
                    "-DCOMPILE_FRENCH_ID_BACK_OCR",
                    "-DCOMPILE_FRENCH_PASSPORT_SIGNATURE_MANAGER",
                    "-DCOMPILE_PASSPORT_AND_FACE_OCR_MANAGER",
                    "-DCOMPILE_MRZ_OCR",
                    "-DCOMPILE_FRENCH_ID_BACK_OCR_MANAGER"
        }

        ndk {
            abiFilters 'x86', 'armeabi-v7a'
        }
    }

    splits {
        abi {
            enable true
            reset()
            include 'x86', 'armeabi-v7a'
            universalApk true
        }
    }
}
    buildTypes {
       release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 
           'proguard-rules.pro'
            signingConfig signingConfigs.config
    }
}
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }

    libraryVariants.all { variant -> variant.outputs.all { output -> 
    outputFileName = "${"libScanovateImaging"}.aar" }
        }
    }

    allprojects {
            repositories {
            jcenter()
            maven {
                url "https://maven.google.com"
            }
        }    
    }
    dependencies {
        implementation 'com.google.android.gms:play-services-vision:15.0.0'
        implementation 'com.android.support:recyclerview-v7:27.1.1'
        implementation fileTree(dir: 'libs', include: ['*.jar'])
        implementation 'com.android.support.constraint:constraint-layout:1.1.0-beta6'
        implementation 'com.android.support:appcompat-v7:27.1.1'
        implementation 'com.android.support:design:27.1.1'

}

Thanks in advance!

Answer 1

Introduction

I think this has been a problem for a long time (judging by the complaints) in Android Studio (more specifically the NDK build system). Now, in the latest version, it is finally ready and working (hopefully).

Note: gradlew builds are significantly faster than using the Android Studio IDE , and can be built using a batch file script ( .bat ).

CMake possible problems(P = problem, C = cause, S = solution)

(P1) Android Studio rebuilds everything every time if multiple ABIs are supported See this SO Q/A .
(C1) Build variants all had the same output directory for the .o files.
(S1) You need to split out the build directory into release/debug and into the different ABIs.

My Setup (ndkBuild)

Android Studio 3.1 , NDK release 10 , gradle version 4.4 (plugin 3.1.0)

I have 2 libraies to build ( libhello_world.so with 1 c++ file, and libjni_photoeditor.so with 24 c++ files).

I am using externalNativeBuild -> ndkBuild to build my native code (you could use CMake also, I have not tested that).

app level build.gradle :

apply plugin: 'com.android.application'
        android {
            compileSdkVersion 25

            defaultConfig {
                applicationId "xxxxxxxxxxxxxxxxxxxxx"
                minSdkVersion 16
                targetSdkVersion 16
                versionCode 1
                versionName "1.0"
                ndk {
                         abiFilters 'armeabi-v7a' //,'x86'
                }//ndk
            }//defaultConfig

            buildTypes {
                release {
                    minifyEnabled true
                    proguardFiles.add(file('proguard-android-optimize.txt'))
                    proguardFiles.add(file('proguard-rules.pro'))
                }//release
                debug {
                    minifyEnabled false
                    jniDebuggable true
                    renderscriptDebuggable true
                }//debug
            }//buildTypes

            externalNativeBuild {
                // Encapsulates your CMake build configurations.
//              cmake {
//                // Provides a relative path to your CMake build script.
//                path "src/main/cpp/CMakeLists.txt"
//              }
                ndkBuild {
                //****this is the working build****
                    path 'src/main/cpp/Android.mk'
                }//ndkBuild
            }//externalNativeBuild
        }//android

        dependencies {
        }//dependencies

Application.mk :

APP_ABI := armeabi-v7a
APP_PLATFORM := android-19
APP_STL := stlport_static

Android.mk :

#================================================
LOCAL_PATH := $(call my-dir) #only call it ONCE !
#================================================
include $(CLEAR_VARS)
LOCAL_MODULE      := hello_world
LOCAL_MULTILIB := 32
LOCAL_SRC_FILES :=  hello_world.cpp
include $(BUILD_SHARED_LIBRARY)
#================================================
include $(CLEAR_VARS)
LOCAL_MODULE      := libjni_photoeditor
LOCAL_MODULE_TAGS := optional
LOCAL_SHARED_LIBRARIES := libm liblog libjnigraphics
LOCAL_LDLIBS := -lm -llog -ljnigraphics -lbcc
LOCAL_LDLIBS := -lm -llog -ljnigraphics
LOCAL_SRC_FILES := _jni.cpp utils.cpp quantize.cpp #etc.. 24 files
LOCAL_CFLAGS := -Werror \
    -I$(OUT)/../../../../frameworks/compile/libbcc/include
LOCAL_LDFLAGS := -L$(OUT)/system/lib
include $(BUILD_SHARED_LIBRARY)

Test 1. Android Studio Build -> Build APK(s) build

(A) Modify one of my projects 24 C++ files ( quantize.cpp ) and rebuild.

(B) Build -> Build APK(s)

(C) Only the C:\\Android\\PhotoRend1\\app\\build\\intermediates\\ndkBuild\\debug\\obj\\local\\armeabi-v7a\\objs-debug\\jni_photoeditor\\quantize.o timestamp changed (and the library).

Test 2. gradlew terminal window (command line) build

From the project root I use gradlew in a terminal window to assemble my apk .

(1) Every thing is up-to-date build:

C:\\Android\\PhotoRend1>gradlew assembleDebug

Starting a Gradle Daemon, 1 busy and 1 incompatible and 1 stopped Daemons could not be reused, use --status for details
> Configure project :app
> Task :app:externalNativeBuildDebug
Build hello_world armeabi-v7a
make.exe: `C:/Android/PhotoRend1/app/build/intermediates/ndkBuild/debug/obj/local/armeabi-v7a/libhello_world.so' is up to date.
Build jni_photoeditor armeabi-v7a
make.exe: `C:/Android/PhotoRend1/app/build/intermediates/ndkBuild/debug/obj/local/armeabi-v7a/libjni_photoeditor.so' is up to date.
BUILD SUCCESSFUL in 3m 46s

(2) Modify one of my projects 24 C++ files (quantize.cpp) and re-assemble.

(3) One file changes is up-to-date build:

C:\\Android\\PhotoRend1>gradlew assembleDebug

> Configure project :app
> Task :app:externalNativeBuildDebug
Build hello_world armeabi-v7a
make.exe: `C:/Android/PhotoRend1/app/build/intermediates/ndkBuild/debug/obj/local/armeabi-v7a/libhello_world.so' is up to date.
Build jni_photoeditor armeabi-v7a
[armeabi-v7a] Compile++ thumb: jni_photoeditor <= quantize.cpp
[armeabi-v7a] SharedLibrary  : libjni_photoeditor.so
BUILD SUCCESSFUL in 59s

(4) As you can see it did an incremental build.

Structure (with abiFilters 'x86', 'armeabi-v7a' ):

C:\Android\PhotoRend1\app\build\intermediates\ndkBuild
|
V
└───debug
    └───obj
        └───local
            ├───armeabi-v7a
            │   └───objs-debug
            │       ├───hello_world
            │       └───jni_photoeditor
            └───x86
                └───objs-debug
                    ├───hello_world
                    └───jni_photoeditor

Some links: Manual NDK updates

Answer 2

Finally I have found the cause to the problem.

In my project I have worked with symlink C++ files. The NDK cannot immediately apply the changes made in these files and they won't be expressed in your app unless you rebuild the whole project.

The solution is to work with hard copy files.

If I find some way to work correctly with symlink files it would be better for my needs. Until then I can at least edit single or multiple files without recompiling the whole project.

This is a relatively esoteric situation but might be helpful for someone in the future.