Tips to make reverse engineering a C shared library more difficult

Question

I am interested in precautionary measures you could take to make reverse engineering a C shared library more difficult

It seems that it is impossible to prevent entirely but you can take steps that might make it take longer and therefore be less attractive/more costly to do

Things like code obfuscation, compiler optimisation options compiler debug flags etc. (I would be specifically interested in the gcc compiler)

Also interested in any thoughts on the relative difficulty of reverse engineering a ~5KLOC C shared library (.so size of ~200kb) in terms of $ or man hours etc.

Answer 1

Those are some tricks that I know of:

• Use strip with the --strip-unneeded option to remove all symbols except the ones that are needed for relocation (since we're talking about a shared library):

  strip --strip-unneeded libmylib.so

• Link with the standard libc statically , this results in a bigger library/binary which means more code to go through and also obfuscation since it will be more difficult to separate your functions from the library functions:

  gcc -static ...

• Force the compiler to inline small functions, this embeds small functions in your code instead of calling them, so it makes it almost impossible to distinguish between your code and the standard library code.

Having said that, I must add that I've had a binary that used the above measures and I was able to reverse engineer it in a few hours, I started by rebuilding the symtab if you're curious, and I'm no reverse engineering guru.

Answer 2

- No need of code obfuscation (as in C# or Java).

The name of your internal variables and functions do not exist in the object code.

However the name of your exported functions will still appear in clear in the shared object file. This sounds reasonable.

- Better to use gcc optimization flags -O2 or -O3 .

Using compiler optimization, object code can largely differ from source code. It is actually very difficult to go back to the original C source code.

However it is always possible to revert engineer at the assembly level. Usually, not all your program is valuable. It will be easier to reverse engineer the interesting portion in assembly rather than the whole program in C.

Answer 3

The problem with most common anti-reversing techniques is that they are well-known and there are tools to overcome them. Of course, stripping symbols and obfuscation, is a must have, but this is just about hiding names/symbols.

If you want to make the job for reversers even harder, invent some code manipulation yourself. It can be something really easy, like using a custom calling convention, but this will make all the disassemblers fail to recognize starts and ends of functions.

These kinds of simple tricks will be quite effective, because there won't be an already-made tool to revert the code to the original. You see, for almost every commercial packer or encrypter, there is already a one-click-unpacker.

Answer 4

Not that much. Once you've stripped debug symbols and obfuscated all internal symbols (not external ones because the clients of the library need a symbol to link to), theres not much you can do. Assembly disassembly is very easy because of the 1-1 mapping of instructions to opcodes. On this other hand, it is very difficult to understand compiler generated assembly code, as it is full of bizarre optimizations and efficiency tricks.

On that note, turning GCC up to highest optimization is probably one of the best obfuscation tricks you can use.

Considering modern computing power, the disassembly itself won't take that long, and at that point it takes a human to decipher it.

Just curious though: What is so important to not have reverse engineered? The encryption and security that keeps things like financial data safe are mostly open source. Or is this a proprietary software protection thing?

Answer 5

Mix your code with switch/case obfuscation (make it nonlinear).

Use C++ templates to encrypt variables/strings at the compilation time. It's possible to encapsulate integer types in double type to make the reversing a little bit harder.

Insert assembly obfuscation macros between the real code (jmp x, db 0E8h, x: etc.)

Use BOOST ;) it really makes reverse engineering a pain the the ... ;)