What makes a gem incompatible with JRuby?

Question

I ran bundle install and to my surprise, most of the gems that ship with Rails 4 installed just fine. byebug was one that did not, but no big deal.

What makes a gem incompatible with JRuby exactly?

Answer 1

In this particular case, the gem is written as a C extension for YARV, so it works only with YARV, not any other Ruby implementation, including, but not limited to, JRuby, Rubinius, MagLev, MRuby, IronRuby, etc.

The sad part is: as far as I can see, it only uses public Ruby APIs, it doesn't actually use any private internal information of the YARV VM, so it could just as well have been written in portable Ruby.

In general, there are several reasons why a gem may work only on one particular Ruby implementation:

• It uses some particular feature of the host platform that is not available to all Ruby implementations. For example, a gem written in Java which uses Java platform libraries, will most likely not work with YARV, only with JRuby.
• It accesses private internal implementation details of a particular Ruby implementation. This is obvious, I think. What's not so obvious is that there is no portable C API even amongst C-based Ruby implementations. What is usually called the "Ruby C extension API" is really just YARV laying bare its internals. Both Rubinius and JRuby have put in an enormous amount of work developing emulation layers for YARV so that at least some C extensions work, but it's hard. For example: YARV's garbage collector is extremely simple, it never moves objects in memory. Many C extensions rely on that, but this is simply not true for modern GCs such as Rubinius's or the JVM's. YARV doesn't allow multiple Ruby threads to run at the same time, many C extensions rely on that, but this is not true for pretty much all other Ruby implementations. And so on … Plus, those emulation layers are slow. Really slow. It is perfectly possible for a gem that has been "re-written in C for performance" to run slower than its pure-Ruby counterpart on JRuby or Rubinius. (After all, Rubinius can, in some cases, already compete with CEg the Hash class in Rubinius is written in Ruby, and performs comparable to YARV's Hash class which is written in C.)
• It uses a Ruby feature that is missing from a particular implementation. For example, Refinements have been widely criticized for being un-implementable in implementations that perform aggressive optimization of method lookup (or more precisely: Refinements, the way they are currently specified, make it impossible to apply those optimizations). The designers of Refinements all happen to be YARV developers, where it doesn't matter, because YARV doesn't optimize method lookup anyway, but eg the JRuby developers have decided to simply not implement Refinements. (10 years ago, they did the same thing with continuations.) So, if a gem uses Refinements, it won't work on JRuby.

The Rubinius and JRuby developers have developed an FFI API that can be used to make C APIs available from Ruby in a manner that is portable across many Ruby implementations. Rubinius, JRuby, MacRuby, and (I believe) MagLev support FFI natively, and for YARV, there is a gem which adds FFI support to it. Gems that use the implementation-independent FFI API instead of the YARV API should work on pretty much all implementations. However, the FFI API doesn't give access to the implementation internals (obviously), so for some gems it is not usable. For example, there are gems which try to give you access to the source code of a Proc , that's highly implementation-specific (and may not even work, eg when you ahead-of-time compile to a Java .class file with jrubyc , the source code doesn't even exist at runtime), so there has to be a different version of that gem for every Ruby implementation.