“符号表全局部分中的局部符号‘__bss_start’”仅在 Android NDK aarch64 构建中

Question

I'm creating an Android library with NDK (r19b) for arm , aarch64 , x86 , and x86_64 . All is OK, except that when I build an app for aarch64 architecture, I get the following error message.

ld.lld: error: found local symbol '_edata' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '_end' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '__bss_start' in global part of symbol table in file libmystuff.so

When I checked each build variant with readelf -s libmystuff.so , I noticed that only aarch64 is different.

[arm]
    4021: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    4022: 007c6b10     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    4023: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86]
    3848: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3849: 00ca4b28     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3850: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86_64]
    3874: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3875: 0000000000ce5f68     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3876: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[aarch64]
       3: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS _edata
       4: 0000000000b990e8     0 NOTYPE  LOCAL  DEFAULT  ABS _end
       5: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS __bss_start

     865: 0000000000b9e3e8     0 NOTYPE  GLOBAL DEFAULT  ABS __end__
    2468: 0000000000b54168     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start__

I can surely see _edata , _end , and __bss_start are in LOCAL instead of GLOBAL, but I don't (or at least I think I don't) do anything special for aarch64 ; they all use the same build configuration.

jni/Application.mk
    NDK_TOOLCHAIN_VERSION := clang
    APP_STL := c++_static
    APP_CFLAGS := -fstack-protector-all -fvisibility=hidden -ffunction-sections -fdata-sections
    APP_CPPFLAGS := -fstack-protector-all -std=c++11 -fvisibility=hidden -ffunction-sections -fdata-sections -frtti
    APP_LDFLAGS := -Wl,--gc-sections,-fvisibility=hidden,--strip-debug

So, why is aarch64 different? Better yet, how do I move those to GLOBAL?

[UPDATE] Thanks to the gentle folks at https://github.com/android-ndk/ndk/issues/927 , I found the solution works best; notice "-fuse-ld=lld" at the end.

APP_LDFLAGS := -Wl,--gc-sections,--strip-debug -fvisibility=hidden -fuse-ld=lld

This way, I still get to keep --gc-sections , and --no-fatal-warnings is not necessary either.

[MORE UPDATE] The example shown in this question is using ndk-build , and if you are using Android Studio with Gradle, you'll most likely have CMake . In that case, add your compiler flags as the following.

[app/build.gradle]
android {
    ...
    defaultConfig {
        externalNativeBuild {
            cmake {
                cppFlags '-fuse-ld=lld'
            }

Answer 1

APP_LDFLAGS := -Wl,-fvisibility=hidden

Does this do anything? This option doesn't appear in the help page for bfd, but those symbols are emitted by the linker so I'm wondering if that's what's causing this.

Failing that, try adding -fuse-ld=gold (or -fuse-ld=lld if you're feeling adventurous, but there are a number of Windows KIs with that if that's something you need to support) to your APP_LDFLAGS . The biggest difference between arm64 and the other architectures in the NDK is that we still use bfd for arm64.

Since these are symbols that are emitted by the linker, this seems like the most likely culprit.

Answer 2

I faced this problem when using an external libExternal.so file.

Since I did not have the source code, I couldn't rebuild it with LLD.

I eventually was able to use the libExternal.so file by disabling the linker warnings in my project. This link was very helpful https://github.com/android/ndk/issues/927

I have made a JNI android app in AOSP (Android 10) that uses a sample.cpp file that uses libExternal.so . My app was able to be built with mm command and worked properly on device.

Here is the Android.mk file that I used in my AOSP build (Android 10).

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE := External
LOCAL_SRC_FILES := libExternal.so
LOCAL_MODULE_TAGS := samples
#LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_SUFFIX := .so
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
include $(BUILD_PREBUILT)

include $(CLEAR_VARS)
LOCAL_MODULE := c++
LOCAL_SRC_FILES := libc++.so
LOCAL_MODULE_SUFFIX := .so
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
include $(BUILD_PREBUILT)

include $(CLEAR_VARS)
LOCAL_MODULE := libsample
LOCAL_C_INCLUDES := $(LOCAL_PATH)/include
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
#LOCAL_CFLAGS += -Iinclude
LOCAL_SRC_FILES := sample.cpp
LOCAL_REQUIRED_MODULES := External
#LOCAL_SHARED_LIBRARIES := External cdsprpc cutils c++ base log
LOCAL_SHARED_LIBRARIES := External
LOCAL_MODULE_TARGET_ARCH := arm64
#LOCAL_MODULE_TARGET_ARCH := arm64-v8a
#LOCAL_LDLIBS += $(LOCAL_PATH)/libExternal.so
#https://stackoverflow.com/questions/65597460/how-to-restrict-android-mk-to-compile-a-module-to-generate-only-64bit-library
LOCAL_CFLAGS += -Wall -std=c99 -g
LOCAL_CPPFLAGS += -Wall -std=c++03 -D_GLIBCXX_USE_CXX11_ABI=0 -g
# Does not build even after commenting below line
#LOCAL_LDLIBS := -lm -lstdc++ -ldl
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#include $(BUILD_EXECUTABLE)
LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
LOCAL_LDFLAGS := -Wl,--no-fatal-warnings
include $(BUILD_SHARED_LIBRARY)

include $(CLEAR_VARS)
res_dir := res $(LOCAL_PATH)/res
#LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_TAGS := samples
LOCAL_SRC_FILES := $(call all-java-files-under, java)
LOCAL_RESOURCE_DIR := $(addprefix $(LOCAL_PATH)/, $(res_dir))
LOCAL_USE_AAPT2 := true
LOCAL_JAVA_LIBRARIES := com.google.android.material_material
LOCAL_STATIC_ANDROID_LIBRARIES := \
        androidx.appcompat_appcompat \
        androidx-constraintlayout_constraintlayout \
        androidx.preference_preference \
        androidx.fragment_fragment \
        androidx.core_core
#LOCAL_REQUIRED_MODULES := sample External
LOCAL_REQUIRED_MODULES := External \
                           libsample \
                           libc++
LOCAL_SHARED_LIBRARIES := External \
                           libsample \
                           libc++
#LOCAL_LDLIBS += -lsample
LOCAL_JNI_SHARED_LIBRARIES := libsample libc++
#LOCAL_JNI_SHARED_LIBRARIES := libsample
#LOCAL_PREBUILT_JNI_LIBS := $(LOCAL_PATH)/libsample.so
LOCAL_PREBUILT_JNI_LIBS := libExternal.so
LOCAL_CERTIFICATE := platform
LOCAL_PRIVILEGED_MODULE := true
LOCAL_PACKAGE_NAME := NativeCPP
LOCAL_PRIVATE_PLATFORM_APIS := true
include $(BUILD_PACKAGE)

By the way I have also tried putting LOCAL_LDLIBS += -lstdc++ in the last block instead of using libc++.so and it works.