Why aren't binaries of different languages compatible with each other? How do you make them compatible?

Question

. 。 Answers to this question are eligible for a +50 reputation bounty. mfaani wants to to this question. 这个问题。

A swift app, will convert its dynamic frameworks into binaries. And once something is a binary, then it's no longer Swift/Ruby/Python, etc. It's machine code.

Same thing happens for a Python binary. So why aren't the machine codes compatible with each other out of the box?

Is it just that a simple mapping is required to bridge one language to the other?

Like if I needed to use a binary created from the Swift language — into a Python based app, then do I need to expose the Swift Headers to Python for it to work? Or something else is required?

Answer 1

I assume you're talking about making calls in one language to a library compiled in a different language.

At the assembly language level, there are standards ( ABI , for Application Binary Interface) that define how function parameters are passed in registers, how values are returned, the behavior of the stack, etc. ABIs are architecture and operating-system-dependent. Usually any function that is exported in a library will follow the ABI.

It is plain that ABIs basically expect a C language model for functions: a single return value, a well-defined data type for each function parameter as well as the return value, the possibility of using pointers, etc.

Problems start to arise once you move to a higher level language. C++ already introduces complications: whereas the name of a C function is the same in assembly (often a _ character is prepended), in C++ function names must encode data types due to the possibility of overloaded functions with the same name but different parameters. Thus, names must be mangled and demangled -- this is why a prototype for a C function must be declared as extern "C" in C++. Then there are issues of the classes ( this pointer, vtables), namespaces and so on, which complicate matters further.

Then you have dynamically typed languages like Python. In truth, there is no such thing as dynamic typing at the assembly language levels: the instruction encodings in machine language (ie binary codes as they're read by the CPU when executed) implicitly determine whether you're using an integer or floating-point or SIMD instruction (and the width of operands), which also determines which of the different register banks are accessed. Although the language makes dynamic typing transparent to you, at the assembly code level, the interpreter/JIT/compiler must resolve them somehow, because ultimately the CPU must be told exactly what data type to operate on.

This is why you can't directly call a C function (or in general any library function) from Python -- unlike a pure Python function which can disregard the types of its parameters, library functions must know the exact types of each parameter and the return type. Thus, you must use something like ctypes for Python, explicitly specifying the types in question for each function that needs to be called -- in a way, this is similar to function prototypes usually found in C headers. It is possible to write functions in C that are directly callable from Python (and, in that case, essentially from Python alone), but you'll have to jump through a few hoops .

As for the particular language pairing you're interested in (Python/Swift), a cursory search came up with this thread in the Swift forums ( this one , linked from there, may also be interesting. Reading the thread, there appears to be two feasible solutions at this time: first, use the @_cdecl attribute (which isn't officially supported) to make a C function, and then call it from Python using ctypes . But the second and apparently more promising one is to use the @objc attribute in Swift, and use PyObjC in Python. I assume this will allow using some of the higher-level features of Swift, at least those that intersect with what Objective-C offers.