autotools: compiling with code from another directory

Question

I have the following structure:

+Makefile.am
+-common
| +-common.c
| +-common.h
|
+-module1
| +module1.c
| +Makefile.am
|
+-module2
| +module2.c
| +Makefile.am

where each moduleX is actually made of many C and header files, and therefore deserve their own subdirectory.

I want to compile each moduleX with the common.c code. I mean compile, not just link with a library because each module actually defines some macro that influence the compilation of common.c. In other words the Makefile of each module looks like:

check_PROGRAMS = moduleX
moduleX_CFLAGS = -I$(top_srcdir)/common -DCOMMON_OPTION_X
moduleX_SOURCES = moduleX.c ../common/common.c

The reason why I am writting ../common/common.c and not $(top_srcdir)/common/common.c is this bug , (also shown here ).

The top Makefile.am declares, of course, each module as subdir:

SUBDIRS = foo bar
TESTS = foo/foo bar/bar

On the real project ./configure && ./make distcheck fails with a "XXX.Po file not found" when building for distcheck.

I have tried to reproduce the problem in a much simpler scale ( download this tar file ), and there, it fails on "common.h" not being found.

I guess that in both cases, the problem is that I haven't succeeded to tell automake that the common part should be a part of each module (and therefore copied as well when building out of tree (VPATH))

What is the proper way to achieve this?

You are welcome to point out the modification needed to the tared example which you can untar with tar -xvf (see the README inside for build instructions)

Thanks!

Answer 1

WHATS HAPPENS:

To resolve "common.h" not being found issue for foobar.tar I've added EXTRA_DIST = common/common.h foo/foo.h bar/bar.h to the top Makefile.am. After that, the command ./bootstrap && ./configure && make dist will include *.h files into foobar-0.1.tar.gz.

Next, ./bootstrap && ./configure && make distcheck fails with Makefile:204: ../common/.deps/foo-common.Po: No such file or directory .

Here is log:

Making distclean in bar
make[2]: Entering directory `/home/user/foobar-0.1/_build/bar'
...
rm -rf ../common/.deps ./.deps
...
make[2]: Leaving directory `/home/user/foobar-0.1/_build/bar'
Making distclean in foo
make[2]: Entering directory `/home/user/foobar-0.1/_build/foo'
Makefile:204: ../common/.deps/foo-common.Po: No such file or directory
make[2]: *** No rule to make target `../common/.deps/foo-common.Po'.  Stop.
make[2]: Leaving directory `/home/user/foobar-0.1/_build/foo'
make[1]: *** [distclean-recursive] Error 1
make[1]: Leaving directory `/home/user/foobar-0.1/_build'
make: *** [distcheck] Error 1

You can see, ../common/.deps is removed as part of make distclean for bar. That results in make distclean for foo failed.

In order to avoid such automake behavior we have to place ../common/.deps/foo-common.Po and ../common/.deps/bar-common.Po inside foo/.deps and bar/.deps directories. To do that we have to place sources from common inside foo and bar directories. That's could be done during build time from Makefile .

SOLUTION: Here is solution based on umläute's idea. It's based on automatic source file generation during build.

topdir Makefile.am:

SUBDIRS = foo bar
TESTS = foo/foo bar/bar
EXTRA_DIST = common/common.h common/common.c foo/foo.h bar/bar.h

foo/Makefile.am:

check_PROGRAMS = foo
foo_CFLAGS = -I$(top_srcdir)/common
foo_SOURCES = foo.c
nodist_foo_SOURCES = $(foo_common_SOURCES)

foo_common_SOURCES = common.c
CLEANFILES = $(foo_common_SOURCES)

$(foo_common_SOURCES):
    echo "#include \"$(top_builddir)/common/$@\"" >$@

bar/Makefile.am

check_PROGRAMS = bar
bar_CFLAGS = -I$(top_srcdir)/common -DOPTION
bar_SOURCES = bar.c
nodist_bar_SOURCES = $(bar_common_SOURCES)

bar_common_SOURCES = common.c
CLEANFILES = $(bar_common_SOURCES)

$(bar_common_SOURCES):
    echo "#include \"$(top_builddir)/common/$@\"" >$@

Answer 2

in each moduleX add a wrapper file:

$ cat <<EOF >inc_common.c
#include "common.c"
EOF

and add that file to the moduleX_SOURCES instead of ../common/common.c .

btw, the -I and -D directives should go into moduleX_CPPFLAGS (rather than moduleX_CFLAGS )

Answer 3

I have had the same problem on several large, multiple executables projects.

I used the following on Ubuntu/Linux

The dependancy files can be created using the appropriate parameters with GCC, however, I used sed for that part of the job.

This has a makefile.top and makefile.bot are both in the top level directory (and no where else)

There are common files, in the top level directory, that are used by all the sub directories.

For your project, you would need to edit the 'alldirectories' macro in the makefile.top to list the sub directories that will contain the source and header files for each executable.

this is executed from the top level directory, using something like: make -f makefile.top

file: makefile.top

    SHELL = /bin/sh





    SRC := $(wildcard *.c)
    OBJ := $(SRC:.c=.o)
    DEP := $(SRC:.c=.d)
    INC := $(SRC:.c=.h)


    MAKE    :=  /usr/bin/make

    CC      :=  /usr/bin/gcc

    CP      :=  cp

    MV      :=  mv

    LDFLAGS :=  -L/usr/local/lib -L/usr/lib -L/lib

    DEBUG   :=  -ggdb3

    CCFLAGS :=  $(DEBUG) -Wall -W

    #CPPFLAGS += =MD

    LIBS    :=  -lssl -ldl -lrt -lz -lc -lm



    .PHONY: AllDirectories
    # the following statement needs to be edited as 
    # subdirectories are added/deleted/re-named
    #AllDirectories := \
    #    Command_Configuration \
    #    Communication \
    #    Main_Scheduler \
    #    Retrieve_CDS_Log \
    #    Retrieve_EventRecorder_Log \
    #    Retrieve_GPS \
    #    Retrieve_QES_Alarm_Log \
    #    Retrieve_QES_RealTime \
    #    Write_CDS_Log

    AllDirectories :=  \
        Main_Scheduler \
        Communication  \
        Retrieve_GPS   \
        Test_Communication_Dev 



    .PHONY: all
    #all: $(OBJ) $(AllDirectories)
    #   $(foreach d,$(AllDirectories), \
    #    ( cd $d && $(MAKE) -f makefile.top name=Tsk_$d all ); )

    all: $(OBJ) $(AllDirectories)
        $(foreach d,$(AllDirectories), \
        ( cd $d && $(MAKE) -f ../makefile.bot name=Tsk_$d all ); )



    #
    # create dependancy files
    #
    %.d: %.c
        # 
        # ========= START $< TO $@ =========
        $(CC) -M $(CPPFLAGS) $< > $@.$$$$;                      \
        sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@;     \
        rm -f $@.$$$$
        # ========= END $< TO $@ =========



    #
    # compile the .c file into .o files using the compiler flags
    #
    %.o: %.c %.d 
        # 
        # ========= START $< TO $@ =========
        $(CC) $(CCFLAGS) -c $< -o $@ -I. 
        # ========= END $< TO $@ =========
        # 



    .PHONY: clean
    #clean: $(AllDirectories)
    #   # ========== start clean activities ==========
    #   rm -f *.o
    #   rm -f $(name).map
    #   rm -f $(name)
    #   rm -f *.d
    #   $(foreach d,$(AllDirectories), \
    #    ( cd $d && $(MAKE) -f makefile.top clean ); )
    #   # ========== end clean activities ==========

    clean: $(AllDirectories)
        # ========== start clean activities ==========
        rm -f *.o
        rm -f $(name).map
        rm -f $(name)
        rm -f *.d
        rm -f ../bin/Tsk_*
        $(foreach d,$(AllDirectories), \
        ( cd $d && $(MAKE) -f ../makefile.bot name=Tsk_$d clean ); )
        # ========== end clean activities ==========



    .PHONY: install
    #install: $(AllDirectories)
    #   # ========== start install activities ==========
    #   $(foreach d,$(AllDirectories), \
    #    ( cd $d && $(MAKE) -f makefile.mak clean ); )
    #   # ========== end install activities ==========

    install: $(AllDirectories)
        # ========== start install activities ==========
        $(foreach d,$(AllDirectories), \
        ( cd $d && $(MAKE) -f ../makefile.bot name=Tsk_$d install ); )
        # ========== end install activities ==========



    # include the contents of all the .d files
    # note: the .d files contain:
    # <filename>.o:<filename>.c plus all the dependancies for that file 
    # I.E. the #include'd header files
    # wrap with ifneg... so will not rebuild *.d files when goal is 'clean'
    #
    ifneq "$(MAKECMDGOALS)" "clean"
    -include $(DEP)
    endif

file: makefile.bot

     SHELL = /bin/sh


    BINDIR  :=  /home/user/bin


    .PHONY: all
    all : $(BINDIR)/$(name) ../makefile.mak ../makefile.bot


    #
    # macro of all *.c files 
    # (NOTE:
    # (the following 'wildcard' will pick up ALL .c files
    # (like FileHeader.c and FunctionHeader.c 
    # (which should not be part of the build
    # (so be sure no unwanted .c files in directory
    # (or change the extension
    #
    SRC := $(wildcard *.c)
    OBJ := $(SRC:.c=.o)
    DEP := $(SRC:.c=.d)
    INC := $(SRC:.c=.h)


    COMMON_OBJ := $(wildcard ../*.o)
    #COMMON_SRC := $(wildcard ../*.c)
    #COMMON_OBJ := $(COMMON_SRC:.c=.o)
    #COMMON_DEP := $(COMMON_SRC:.c=.d)
    #COMMON_INC := $(COMMON_SRC:.c=.h)

    MAKE    :=  /usr/bin/make

    CC      :=  /usr/bin/gcc

    CP      :=  cp

    MV      := mv

    LDFLAGS :=  -L/usr/local/lib

    DEBUG   :=  -ggdb3

    CCFLAGS :=  $(DEBUG) -Wall -W

    #CPPFLAGS += =MD

    #LIBS    :=  -lidn -lssl -ldl -lrt -lz -lc -lm
    LIBS    :=   -lssl -ldl -lrt -lz -lc -lm



    #
    # link the .o files into the executable 
    # using the linker flags
    # -- explicit rule
    #
    $(name): $(OBJ) $(COMMON_OBJ) ../makefile.mak ../makefile.bot
        #
        # ======= $(name) Link Start =========
        $(CC) $(LDFLAGS) -o $@ $(OBJ) $(COMMON_OBJ) $(LIBS)
        # ======= $(name) Link Done ==========
        #



    # note:
    # using MV rather than CP results in all executables being re-made everytime
    $(BINDIR)/$(name): $(name)
        #
        # ======= $(name) Copy Start =========
        sudo $(CP) $(name) $(BINDIR)/.
        # ======= $(name) Copy Done ==========
        #



    #
    #create dependancy files -- inference rule
    # list makefile.mak as dependancy so changing makfile forces rebuild
    #
    %.d: %.c 
        # 
        # ========= START $< TO $@ =========
        $(CC) -M $(CPPFLAGS) $< > $@.$$$$;                      \
        sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@;     \
        rm -f $@.$$$$
        # ========= END $< TO $@ =========



    # 
    # compile the .c file into .o files using the compiler flags
    # -- inference rule
    #
    %.o: %.c %.d 
        # 
        # ========= START $< TO $@ =========
        $(CC) $(CCFLAGS) -c $< -o $@ -I. 
        # ========= END $< TO $@ =========
        # 



    .PHONY: clean
    clean: 
        # ========== CLEANING UP ==========
        rm -f *.o
        rm -f $(name).map
        rm -f $(name)
        rm -f *.d
        # ========== DONE ==========



    .PHONY: install
    install: all

    # include the contents of all the .d files
    # note: the .d files contain:
    # <filename>.o:<filename>.c plus all the dependancies for that .c file 
    # I.E. the #include'd header files
    # wrap with ifneg... so will not rebuild *.d files when goal is 'clean'
    #
    ifneq "$(MAKECMDGOALS)" "clean"
    -include $(DEP)
    endif