A virtual machine for C++ for optimizing performance

Question

An argument in favor of JITed languages such as C# and Java is that they can perform optimizations better since runtime profiling by the virtual machine can optimize code better than statically optimized code of C++.

However, I was wondering if we could also use a virtual machine to optimize code at runtime for C++, or rather any similar language. For example, we could take the IR generated by the LLVM compiler and make a virtual machine that interprets, JIT and optimize code, similarly as in case of Java and C#.

Ofcourse, there would be no garbage collection, but the optimization factor would be there. Has anyone worked on this. Are there any papers, tools on this? How good will this approach be?

Answer 1

This is a flawed argument. Yes, virtual machines have more information to work with- but they also have vastly less time and space compared to compilers.

Also, yes, absolutely you can do it if you really want to. But nobody does, so it generally isn't happening. At least, not for optimization reasons, you might do it for sandboxing.

Answer 2

LLVM includes a JIT compiler, lli, and Clang can already produce bitcode from C++. I haven't tried it, but I assume you can then use lli on the produced bitcode files (you may have to tell it where to find the C++ lib files to link in) and JIT C++ at runtime. You should even be able to build the libc++ as bitcode so that it can be JITed as well.

Google's Native Client has a sub project, Portable Native Client, where I think LLVM IR is sent to the client, instead of x86 binaries, to be jitted on the client.

Answer 3

Theoretically speaking, yes, a JIT can be made for C++. It could take advantage of some things in the underlying architecture to aggressively optimize the code. It would also come with the downside of making the application take longer to load at runtime.

Ofcourse, there would be no garbage collection and therefore the overhead due to it, but the optimization factor would be there. Has anyone worked on this. Are there any papers, tools on this? How good will this approach be?

Big misconception here. Imposing GC across the board for every user-defined type is major overhead. It's one of the reasons Android, iOS, and Windows mobile have all turned to C/C++ for high-performance applications in spite of starting off trying to use only managed VMs initially.

Of course the additional level of indirection means the GC is free to do things like compact memory, but an optimized C/C++ program would already be working with compacted memory from the beginning. It would also mean the memory is more fragmented initially, which is a performance killer for the kind of high-performance applications that C++ is good at (one's that deal with large, contiguous buffers, eg, like video processing, ray tracing, or audio processing).

Also turning every UDT instance into a reference means that everything is on the heap, which is effectively turning operations that are originally a few clock cycles into hundreds.

That said, to get to the heart of your question, sure, C++ code can be built using JIT, but you may find that there aren't really such compelling reasons to do so given things like the static nature in which people generally work with C++ code.