Autotools: GCC's Makefile.in has a bug (7.3.0 and earlier). How to re-program automake to find 'ar' and 'objdump'

Question

GCC's cross compiling autotools is supposed to be flexible, but I've isolated a bug that's been breaking cross compiler builds that ought to work.

Note: Some systems will "poison" default compiler tool names to prevent using wrong tools by default. On my system, x86_64-pc-gnu-linux-ar will execute but "ar" is not found.

I need to build cross compiler toolchains with custom names. gcc's configure script supports this with --program-prefix or --program-transform-name. However, when using a custom name, all compile time tools have to be explicitly named on the configure line. gcc configure is not intelligent enough to find tools it has just built with a name change. (too stupid).

The GCC manual states how to explicitly name tools:

configure AR=x86_foo_b_ar AR_FOR_TARGET=ARMv6m_foo_b_ar ... 

However, it doesn't work right. Autoools sometimes ignores the supplied names and the build fails. In particular, it ignores 'AR' and 'OBJDUMP' variables.

Apparently the toplevel gcc configure was created at a later date than lower level configures.

Makefile.in without Makefile.am in GCC?

Makefile.am does not exist in some subdirectories, but it does exist in newer subdirectories.

This causes inconsistencies in variable passing from the top-level makefile.

Internally, the top level "configure" script has variables AR_FOR_HOST (alias for AR), AR_FOR_BUILD, and AR_FOR_TARGET. These variables are used to re-define "AR" when entering sub-directories to force a generic make script to compile for a particular target.

I've even gone so far as to define the internal variables correctly as well as "AR" and "OBJDUMP" on the configure command line. ( Shouldn't be needed ).

gcc-7.3.0/configure --host=x86_64-pc-linux-gnu --program-prefix=armv6m-softfloat-eabi-newlib- AR_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ar AR=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ar AS_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-as AS=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/as DLLTOOL_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/dlltool LD_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ld LD=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ld LIPO_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/lipo NM_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-nm NM=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/nm OBJCOPY_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objcopy OBJDUMP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objdump RANLIB_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ranlib READELF_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-readelf READELF=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/readelf STRIP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-strip STRIP=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/strip CC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cc CXX_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cxx WINDRES_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windres WINDMC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windmc --target=armv6m-softfloat-eabi --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/armv6m-softfloat-eabi-newlib/gcc-bin/7.3.0 --includedir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include --datadir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0 --mandir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/man --infodir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/info --with-gxx-include-dir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include/g++-v7 --with-python-dir=/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/python --enable-languages=c --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 7.3.0-r3 p1.4 --disable-esp --enable-poison-system-directories --disable-bootstrap --with-newlib --enable-multilib --disable-altivec --disable-fixed-point --with-float=soft --disable-libgcj --disable-libgomp --disable-libmudflap --disable-libssp --disable-libcilkrts --disable-libmpx --disable-vtable-verify --disable-libvtv --disable-libquadmath --enable-lto --without-isl --disable-libsanitizer --enable-default-pie --enable-default-ssp

I'm wanting gcc to both make and use tools that start with the prefix: armv6m-softfloat-eabi-newlib- (Arm cortex m0 chipset is what I am using)

But "make" still fails when attempting to execute "ar" in the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file. On line 28 of the old .../libcpp/Makefile.in it says "AR = ar"

So, the AR variable is hardcoded to "ar" But, "ar" doesn't exist on my system. I've tried editing .../libcpp/Makefile.in with "AR = dummyname" , and the build crashes with "can't fine dummyname" instead of can't find "ar". So, the bug is on line 28.

All other variables in the .../libcpp/Makefile.in are of the form:

CC = @CC@
INSTALL = @INSTALL@

etc.. On a positive note: The compiler used by .../libcpp IS the fully qulaified name I gave to gcc-7.3.0/configure. That success made me think I could fix the bug by editing the makefile to read:

AR = @AR@

But the build fails with "Can't find AR@"

I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug. What's the @variable@ name format do?

Does the configure.ac in the subdirectory have to define "AR" in some way for @AR@ to be linked to the value in the toplevel directory?

I've tried a few other tests while building different gcc versions. Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ() macro.

For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails and complains that I must use autotools 2.64, ONLY. eg: AC_PREREQ(2.64)

So, fixing the bug via autotools doesn't seem practical. I'm hoping to simply patch the .../libcpp/Makefile.in, since that file is exactly the same in so many versions of gcc.

Questions: Why is "ar" hard-coded ? Is this a serious legacy issue? and what is a minimal patch that won't interfere with other configurations of GCC?

Is it better to modify the shell or the Makefile; eg: like the top level configure shell script could define a bash function that would be inherited by make as "if" it were a program.

if [ -z ${AR##*-*} ] ; then
    ar() { $AR } 
fi

---

Edit: A quick-fix patch for gcc-7.3.0 This is not a "correct" fix, but just a work-around.

I've found three places where the sub-directories ignore variables passed in from the toplevel configure. .../libcpp/Makefile.in on line 29 .../gcc/configure just before line 29531 .../libcc1/configure just before 14574

The second and third errors are from a defective macro in configure.ac. I haven't traced it back because I can't run autoconfig anyway.

I added a line to the configure(s), to see if passing the default OBJDUMP override variable would allow gcc to compile. It does. I'm not sure I've chosen the right override variable for all cases of gcc compile switches, but at least it proves where the bug is.

Patch file follows:

--- gcc-old/libcpp/Makefile.in                                                  
+++ gcc-new/libcpp/Makefile.in                                                  
@@ -28,3 +28,3 @@                                                               
 INSTALL = @INSTALL@                                                            
-AR = ar                                                                        
+AR ?= ar                                                                       
 ARFLAGS = cru                                                                  
--- gcc-old/gcc/configure                                                       
+++ gcc-new/gcc/configure                                                       
@@ -29531,4 +29531,6 @@                                                         
      ;;                                                                        
    esac                                                                        

+   if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi                 
+                                                                               
    if test x"$enable_plugin" = x"yes"; then                                    
--- gcc-old/libcc1/configure                                                    
+++ gcc-new/libcc1/configure                                                    
@@ -14574,4 +14574,6 @@                                                         
      ;;                                                                        
    esac                                                                        

+   if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi                 
+                                                                               
    if test x"$enable_plugin" = x"yes"; then

Answer 1

TL;DR : there are a lot of things you could try, but the very first would be to specify AR on the command line when you run make :

make AR=x86_foo_b_ar

That shouldn't be necessary when you've already specified the same to configure , but if it doesn't work then that suggests a problem one or more levels up from the Makefile.in you're looking at. Variable definitions specified on the make command line override definitions in makefiles.

---

"make" still fails when attempting to execute "ar" from the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file.

To be clear, since understanding the system you are trying to use is immensely helpful in troubleshooting it, automake does not run at configuration or build time. It is used by the package maintainer to build one or more Makefile.in files to be included in source distributions, such as the one you obtained. Of course, this is not the only way to create Makefile.in files, and the configure script does not care how you create them (or other input files).

I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug. What's the @variable@ name format do?

Does the configure.ac in the subdirectory have to define "AR" in some way for @AR@ to be linked to the value in the toplevel directory?

The @variable@ construction is used for values that are expected to be substituted by the configure script when it builds a corresponding output file. For that to take place, there needs to be at least a corresponding AC_SUBST([variable]) or its equivalent in the configure.ac (sometimes named configure.in , instead). Normally, that's preceded somewhere in configure.ac by code assigning an appropriate value to shell variable variable .

If you modify configure.ac then you need to rebuild the configure script, and in that case it's probably safest to rebuild the whole build system, as a package maintainer would do. There may be a script provided for that purpose in the package ( autogen.sh is a common name for such scripts), but the default mechanism is to run the Autotools program autoreconf in the top-level directory of the project source tree.

I've tried a few other tests while building different gcc versions. Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ() macro.

For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails and complains that I must use autotools 2.64, ONLY. eg: AC_PREREQ(2.64)

That description is not consistent with the documentation of AC_PREREQ , nor with my experience with that macro. AC_PREREQ tests for the specified Autoconf version or newer . It does not demand an exact Autoconf version. There may be something else in the build system that does so, but it's not AC_PREREQ .

In any case, one alternative would be to obtain and install Autoconf 2.64. You may even be able to install it alongside your existing version. Some systems even provide pre-built packages for exactly that purpose.

So, fixing the bug via autotools doesn't seem practical. I'm hoping to simply patch the .../libcpp/Makefile.in, since that file is exactly the same in so many versions of gcc.

Patching a Makefile.in does not require afterward re-running the autotools, so it's at least conceivable that that would work. Even for Makefile.in files that were generated by Automake. You could consider having a look at how AR is defined in some of the Automake-generated Makefile.in files in the project (supposing there are any) for an idea of how it should look.

Why is "ar" hard-coded ? Is this a serious legacy issue?

I can only speculate. As a threshold matter, I'm inclined to suppose that in that Makefile , the archiver of the build system is the one wanted (not that of the intended host system, nor a cross-ar for host-target). It is reasonable in that case for AR = ar to be provided as a default, because that can be overridden via a declaration of that variable on the command-line.

That you are in fact not getting the AR you specify to configure looks like a bug to me -- probably a regression introduced at some point when some of the higher-level bits of the build system were updated. I have no trouble imagining such an issue slipping by, as a system configuration such as yours, in which the system's own archiver goes only by a non-standard name, is very uncommon.

and what is a minimal patch that won't interfere with other configurations of GCC?

The first thing to try is to pass the AR definition on the top-level make command line:

make AR=x86_foo_b_ar

Such definitions will be passed on to recursively-invoked sub- make s, and definitions on the command line (but not, by default, from the environment) override definitions in Makefiles.

Is it better to modify the shell or the Makefile; eg: like the top level configure shell script could define a bash function that would be inherited by make as "if" it were a program.

The top-level configure script could be modified to define a shell function and export it to child processes, but not to its parent or siblings. This is nothing specific to configure ; the shell just doesn't work that way. Whatever changes you make, if any, would be best made in Makefile.in files before running configure , or in the generated Makefile s afterward.