就性能而言，就共享库文件而言，“针对架构进行编译”与“在机器上进行编译”一样好吗？

Question

I am working on a project that is mostly Java. The application is extremely computationally intensive (it performs several mathematical operations such as exponentiations and multiplications on 2048-bit numbers). Performance is important, users want to get the results as quickly as possible.

Initially, all of the code was Java, however the most computationally intensive parts of the application have been re-implemented in C and in C++ for performance reasons (some parts in C and some parts in C++).

The application itself detects if certain shared library files are available or not on the path, if it finds these libraries then it uses them, if it does not find them, everything is done in Java.

The Java application uses JNI to make calls to the native code.

The C and in C++ use the GMP library ( https://gmplib.org ) to perform mathematical operations.

At the moment, the distribution file that is distributed to users includes a jar (containing the compiled Java code), C and C++ source code, and scripts to compile the C and C++ code, and to execute the application.

The build process that users have to perform is the following:

1. Download, compile and configure GMP.

2. Compile the C code to a shared library file (dll on windows, .so on linux), for example, on Linux, users execute:

   gcc -fPIC -shared -O2 -o ../lib/libconnectorgmp.so -I$$JAVA_HOME/include/ -I$$JAVA_HOME/include/linux -I include NativeAdapter.c ../include/libgmp.a

3. Compile the C++ code to a shared library file, for example, on Linux, user execute:

   g++ -fPIC -shared -O2 -o ../lib/libNativeLimMultiExpo.so -I$$JAVA_HOME/include/ -I$$JAVA_HOME/include/linux -I include NativeLimMultiExpo.cpp MultiExponentiation.cpp MultiExponentiation.h ../include/libgmp.a

Notes:

1. ../include/libgmp.aa is a file that was generated in step 1.

2. the .c, .cpp and .h files are application code.

Once the user has executed these steps, they will have two shared library files, and they then include these in the path when calling the Java application.

This build process works fine. When the shared libraries are generated and used the performance is normally around 5 times faster then purely Java.

The question is, would the same performance be achieved if compiled versions of the libraries files were distributed to users. In that case, 32-bit and 64-bit libraries for Windows and Linux would be distributed.

I guess that a compiled GMP object file would also be included in those libraries, so that users would not have to do step 1 of the current build process either.

One big advantage would be the users would have to perform less steps, but I don't know if the performance would be less.

I guess the question could be stated as: "Is compiled for an architecture" as good as "compiled on a machine", in terms of performance?

Answer 1

Compiling code with very specific compiler flags for a specific machine's hardware should in principle get the best performance.

However in practice I think that as you are using GMP under the hood, the bulk of optimization is already done simply by using GMP. The speed-up of using any GMP instead of Java's BigInteger is much greater than anything else, which will probably be close to immeasurable at best.

Modern C/C++ compilers optimize really well. Leave them to it.

Java and JNI are probably imposing an overhead that I'd be sorely tempted to get rid of if performance is so critical. JNI and Java come with overheads optimized C/C++ will not. In other words, skip Java entirely and go with pure C/C++.

I am dubious of distributing a build system and sources as your describe. This has many disadvantages, not least of which is that it forces user's to become developers. If, for example, GMP issue a new version with improvements ( which is quite possible ) your users now have to rebuild the application from new sources or simply put up with what they have. The same does if security patches fixes are issued.

I think it is generally better to install generic libraries like GMP in a way that permits easy upgrades and replacement in a manner the library distributors recommend.

So overall to gain performance :

• ditch Java entirely, use 100% C/C++

• leave the compiler to work out how best to optimize ( just -O3 ).

And don't distribute GMP yourself. It's making a problem you don't need.

A minor point :

some parts in C and some parts in C++

That's not something I generally recommend. Pick one or the other. You are effectively mixing three different languages ( with Java ) and it simply creates potential issues. If you're using C++, use it everywhere would be my suggestion.