Is it possible to apply different compilation jobs to a different targets into a Makefile in order to use distcc

Question

I have a Makefile which has several targets. One of then it's just to generate cpp code automatically. The second one is designated to compile those generated cpp.

I have configured a cluster of distributed compilation and I just want to specify a amount of jobs to use for the compilation of the cpp files themselves. And another different amount of jobs for the generation part.

Is it possible?

The idea is use “make -j30” to distribute by distcc the builds to several available hosts, but the target dedicated to generate code defined into the Makefile cannot be splitted in 30 jobs because this target just will be processed locally. And the localhost has not 30 cores to process them, so the machine freeze. That is the reason i need to define amount of jobs for the generation phase and another amount of jobs for the compilation phase.

Answer 1

The idea is use “make -j30” to distribute by distcc the builds to several available hosts, but the target dedicated to generate code... will be processed locally. And the localhost has not 30 cores to process them, so the machine freeze

An alternative to manually setting a different static limit on the number of concurrent jobs is to let GNU Make throttle jobs dynamically by system load. See the -l or --max-load option documented here .

This should prevent local codegen jobs from overloading the system, while allowing the full number of distcc jobs to run in parallel (assuming they don't have much impact on the local system load - they shouldn't affect the run queue when they're blocked waiting on input from the remote build).

$ make -j30 -l5...

You'll have to choose a suitable load value for your system, and I can't help you with that. Try running top during a build and watch the load average section.