Underscore prefix problem in x86: Calling NASM function from C++ function works in x64 but fails in x86

Question

I am using Visual studio 2019 in Windows 10, and I want to compile in x86 using MSVC(platform toolset 142) and NASM(version 2.14.02) the next code:

foo.asm

section .text

global foo
foo:
    mov eax, 123
    ret

main.cpp

extern "C" int foo(void);

int main()
{
    int x = foo();
    return 0;
}

But I got the error:

In x64 works well, in x86 the generated file main.obj adds a leading underscore to the function name foo, resulting in _foo. This does not happen in x64 but keeps the symbol as foo, not _foo.

So, is there any solution that works for both x86 and x64 platforms (preferably without modify source code, maybe some compiler/linker flag for MSVS compiler)?

I really appreciate any help.

Answer 1

The _ prefix is a result of name mangling, which depends on the target platform ABI (OS, bitness, calling convention).

According to Microsoft, the _ prefix is used in a 32-bit Windows cdecl calling convention, but not in 64-bit ( source ):

Format of a C decorated name

The form of decoration for a C function depends on the calling convention used in its declaration, as shown in the following table. This is also the decoration format that is used when C++ code is declared to have extern "C" linkage. The default calling convention is __cdecl. Note that in a 64-bit environment, functions are not decorated .

Calling convention — Decoration
__cdecl Leading underscore ( _ )
__stdcall Leading underscore ( _ ) and a trailing at sign ( @ ) followed by the number of bytes in the parameter list in decimal
__fastcall Leading and trailing at signs ( @ ) followed by a decimal number representing the number of bytes in the parameter list
__vectorcall Two trailing at signs ( @@ ) followed by a decimal number of bytes in the parameter list

The reason behind could be that 32/64-bit Windows calling conventions aren't really compatible. For example, function arguments in 64-bit mode are passed differently and different registers have to be preserved between calls.

Anyway, to answer the question, I can think of a couple of workaround solutions:

Solution 1

The code that generates the .asm should add the leading _ only in 32-bit mode (the generated assembly will probably have to differ in other ways too, especially if pointers are involved).

Solution 2

Use a preprocessor to add the leading _ in 64-bit mode:

#ifdef _WIN64
#  define foo _foo
#endif

extern "C" int foo(void);

int main()
{
    int x = foo();
    return 0;
}

Answer 2

Correct solution is to have different asm files for each architecture: x86, amd64, etc, because you will use different assembler instruction sets at all. And select which file will be compiled by make file or environment configuration depends on build architecture. So you can use '_foo' function name for x86 and 'foo' for x86_64