lRE: GNU Make 4.2 Query

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

lRE: GNU Make 4.2 Query

Cook, Malcolm-2
Thanks, Paul, for the open-ended request for suggestions....

I am most interested in improved control over parallelization.    I expect others have suggested things that have your consideration already, am I right, and are any of the proposals "worthy" in your estimation?  

Perhaps the ability to declare on a {static,pattern} rule level, the load number of CPUs the rule body is intended to us, allowing Make to use that number as a multiplier in determining whether running the job will (theoretically) saturate --jobs.  Perhaps a convention that the variable

--jobs and .NOTPARALLEL are the only control I see, and it is very gross level of control.  I have many cases where different "steps" of a workflow can be parallelized internally, and have few options to communicate that to Make.  

FWIW, when I have a pattern rule which is highly parallelizable, I have figured out how to force all matching targets to run in serial by making them depend upon one another in serial with a custom macro, .  Kind of a hack, but it allows me to have only one instance of the recipe running at a time by writing:

$(call NotParallel,${matchingTargets})

What might be useful would be to have a variable MAKEJOBSLOTS which defaults to 1 and declares the number of slots/cpus each job is expected/designed to consume, and then allow that variable to appear as pattern rules specific variable.

Thoughts?

On the topic, do you want to warn me against doing things like this?:

.j=$(subst ${space},,$(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS))))
### Exposes the value of -j to the makefile (e.g. to reason about
### process allocation).  NB: will be empty if not passed - does not
### default.

Finally, if you or anyone is interested, or want to give me suggestions for improvement, NotParallel is defined as:

################################################################################
## ALAS: .NOTPARALLEL: is global.  Here is a hack to allow only
## selected rules to be declared as NOTPARALLEL.
##
## First, we need a macro to ensure that ${1} is created AFTER ${2} is
## completed:
NotParallelDepends=${1}: | ${2}
## Then we need a macro to call NotParallelDepends on pairs of successive
## arguments:
NotParallel=$(foreach i,$(shell seq 2 $(words ${1})),$(eval $(call NotParallelDepends,$(word ${i},${1}),$(word $(shell echo $$(( ${i} - 1 ))) ,${1}))))

Cheers, and good luck with the next rev!

Malcolm Cook