how to debug large executables with mingw-w64

Question

I'm trying to debug my executable build with the mingw-w64 compiler on windows and noticed the following: When you compile large source files that use templates intensively, you may receive the File too big/too many sections error. The solution is to add the option -Wa,-mbig-obj . However, if I do so I get the following error:

{standard input}: Assembler messages:
{standard input}: Fatal error: can't close myexe.exe.p/src_session.cpp.obj: file too big

As far as I'm aware there are certain limitations when building such large executables with the mingw-w64 compiler on windows (for debugging purposes of course, no problems in release mode).

I came up with some ideas on how to resolve this problem, but before I describe them please note my project structure

.
├── src
|   └── session.cpp
├── include
|   ├── scope.hpp
│   └── session.hpp
├── subprojects
│   ├── somesubproject
│   └── anothersubproject
└── meson.build

To put it simply, my project involves some heavily templated subproject which I think cause the issue here (for example this MQTT repository https://github.com/redboltz/mqtt_cpp ). I do not need any debug information for my subprojects at all, I only need debugging information for my own files session.hpp, session.hpp, session.cpp . Is it possible to turn on debugging information only for certain files, ie for everything except the subprojects?

Would a valid workaround be to compile and debug my project on my equivalent archlinux docker container? Will I encounter the same problem on Linux?

The project effortlessly compiles with the -Og flag, however that's not what I want as the compile optimized out a lot of interesting stuff.

Here you find the compiler instruction which I used

[1/4] "C:\msys64\mingw64\bin\g++" "-Ialosaclient.exe.p" "-I." "-I.." "-I..\include"   
"-Isubprojects\alosa1_common" "-I..\subprojects\alosa1_common" "-I..\subprojects\alosa1_common\include"  
"-I..\subprojects\hana\include" "-I..\subprojects\mqtt_cpp\include" 
"-I..\subprojects\tomlplusplus\include" "-I..\subprojects\pybind11\include" 
"-I..\subprojects\spdlog\include" "-IC:/msys64/mingw64/include" 
"-IC:/msys64/mingw64/include/python3.8" "-fdiagnostics-color=always" "-pipe" "-D_FILE_OFFSET_BITS=64" 
"-std=c++17" "-g" "-Wa,-mbig-obj" "-DBOOST_ALL_NO_LIB" "-pthread" "-DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT"
"-MD" "-MQ" alosaclient.exe.p/mqtt_pch.hpp.gch "-MF" "alosaclient.exe.p\mqtt_pch.hpp.gch.d" 
-o alosaclient.exe.p/mqtt_pch.hpp.gch "-c" ../pch/mqtt_pch.hpp
[2/4] "C:\msys64\mingw64\bin\g++" "-Ialosaclient.exe.p" "-I." "-I.." "-I..\include"  
"-Isubprojects\alosa1_common" "-I..\subprojects\alosa1_common" 
"-I..\subprojects\alosa1_common\include" "-I..\subprojects\hana\include" 
"-I..\subprojects\mqtt_cpp\include" 
"-I..\subprojects\tomlplusplus\include" "-I..\subprojects\pybind11\include" 
"-I..\subprojects\spdlog\include" "-IC:/msys64/mingw64/include" "-IC:/msys64/mingw64/include/python3.8" "-fdiagnostics-color=always" "-pipe" "-D_FILE_OFFSET_BITS=64" "-std=c++17" "-g" "-Wa,-mbig-obj" 
"-DBOOST_ALL_NO_LIB" "-pthread" "-DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT" "-fpch-preprocess" "-include" "mqtt_pch.hpp" -MD -MQ alosaclient.exe.p/subprojects_alosa1_common_src_common_crc32.cpp.obj 
-MF "alosaclient.exe.p\subprojects_alosa1_common_src_common_crc32.cpp.obj.d" 
-o alosaclient.exe.p/subprojects_alosa1_common_src_common_crc32.cpp.obj "-c" ../subprojects/alosa1_common/src/common/crc32.cpp
[3/4] "C:\msys64\mingw64\bin\g++" "-Ialosaclient.exe.p" "-I." "-I.." "-I..\include" 
"-Isubprojects\alosa1_common" "-I..\subprojects\alosa1_common" 
"-I..\subprojects\alosa1_common\include" "-I..\subprojects\hana\include" 
"-I..\subprojects\mqtt_cpp\include" "-I..\subprojects\tomlplusplus\include" 
"-I..\subprojects\pybind11\include" "-I..\subprojects\spdlog\include" "-IC:/msys64/mingw64/include" 
"-IC:/msys64/mingw64/include/python3.8" "-fdiagnostics-color=always" "-pipe" "-D_FILE_OFFSET_BITS=64" 
"-std=c++17" "-g" "-Wa,-mbig-obj" "-DBOOST_ALL_NO_LIB" "-pthread" 
"-DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT" "-fpch-preprocess" 
"-include" "mqtt_pch.hpp" -MD -MQ alosaclient.exe.p/src_session.cpp.obj 
-MF "alosaclient.exe.p\src_session.cpp.obj.d" -o alosaclient.exe.p/src_session.cpp.obj "-c" ../src/session.cpp
FAILED: alosaclient.exe.p/src_session.cpp.obj
"C:\msys64\mingw64\bin\g++" "-Ialosaclient.exe.p" "-I." "-I.." "-I..\include" "-Isubprojects\alosa1_common" "-I..\subprojects\alosa1_common" "-I..\subprojects\alosa1_common\include" "-I..\subprojects\hana\include" "-I..\subprojects\mqtt_cpp\include" "-I..\subprojects\tomlplusplus\include" "-I..\subprojects\pybind11\include" "-I..\subprojects\spdlog\include" "-IC:/msys64/mingw64/include" "-IC:/msys64/mingw64/include/python3.8" 
"-fdiagnostics-color=always" "-pipe" 
"-D_FILE_OFFSET_BITS=64" "-std=c++17" "-g" "-Wa,-mbig-obj" 
"-DBOOST_ALL_NO_LIB" "-pthread" "-DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT" 
"-fpch-preprocess" "-include" "mqtt_pch.hpp" 
-MD -MQ alosaclient.exe.p/src_session.cpp.obj -MF "alosaclient.exe.p\src_session.cpp.obj.d" -o alosaclient.exe.p/src_session.cpp.obj "-c" ../src/session.cpp
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/as.exe: alosaclient.exe.p/src_session.cpp.obj: section .pdata$_ZNSt8_Rb_treeINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_St8functionIFbhhEEESt10_Select1stISB_ESt4lessIS5_ESaISB_EE5beginEv: string table overflow at offset 10000080
{standard input}: Assembler messages:
{standard input}: Fatal error: can't close alosaclient.exe.p/src_session.cpp.obj: file too big
ninja: build stopped: subcommand failed.

Answer 1

If the error is raised after Fatal error: can't close xxx.obj: file too big , then the assembler directive -Wa,-mbig-obj will not work. There seems to be a limitation imposed on the object file size, which the mingw-w64 compiler can produce.

This problem does not occur when building with the native GCC, so the solution here is to debug large executables on the docker container. Using Visual Studio Code, the debug configuration launch.json needs to be set up as follows:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "C++ debug",
      "type": "cppdbg",
      "request": "launch",
      "stopAtEntry": false,
      "environment": [],
      "externalConsole": false,
      "MIMode": "gdb",
      "windows":{
        "program": "/usr/project/build/session",
        "cwd": "/usr/project",
        "args": ["/usr/project/test/config/linkcommand.toml", "ABCDE"],
        "pipeTransport": {
          "pipeCwd": "${workspaceRoot}",
          "pipeProgram": "docker",
          "pipeArgs": [
              "run", 
              "-i", 
              "-v",
              "c:/projtms/alosa1-client:/usr/project",
              "${config:MECOS_windows_mingw_toolchain}", 
              "sh",
              "-c"
          ],
          "debuggerPath": "/usr/bin/gdb"
        },
        "sourceFileMap": {
                "/usr/project":"C:\\ProjTMS\\alosa1-client"
        },
      },
      "setupCommands": [
        {
          "description": "Enable pretty-printing for gdb",
          "text": "-enable-pretty-printing",
          "ignoreFailures": true
        }
      ]
    }
  ]
}

Please recall that my project has the following structure:

c:/projtms/alosa1-client
├── src
|   └── session.cpp
├── include
|   ├── scope.hpp
│   └── session.hpp
├── subprojects
│   ├── somesubproject
│   └── anothersubproject
└── meson.build

Note that the workspace c:/projtms/alosa1-client is mounted in the docker containers /usr/project folder, therefore the sourceFileMap option needs to be specified accordingly.

This setup allows to debug executables without having any limitation imposed on object file size.