Log in

No account? Create an account
Fighting with make(1) - Beware of the Train [entries|archive|friends|userinfo]

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

[Links:| My moblog Hypothetical, the place to be My (fairly feeble) website ]

Fighting with make(1) [Jul. 2nd, 2011|06:37 pm]
[Tags|, , , , , , , ]

I'm currently running a lot of benchmarks in my day job, in the hope of perhaps collecting some useful data in time for an upcoming paper submission deadline - this is the "science" part of "computer science". Since getting a given benchmark suite built and running is often needlessly complex and tedious, one of my colleagues has written an abstraction layer in the form of a load of Makefiles. By issuing commands like "make build-eembc2", "make run-utdsp" or "make distclean-dspstone" you can issue the correct command (build/run/distclean) to whichever benchmark suite you care about. The lists of individual benchmarks are contained in .mk files, so you can strip out any particular benchmark you're not interested in.

I want to use benchmark runs as part of the fitness function for a genetic algorithm, so it's important that it run fast, and simulating another processor (as we're doing) is inherently a slow business. Fortunately, benchmark suites consist of lots of small programs, which can be run in parallel if you don't care about measuring wallclock seconds. And make already has support for parallel builds, using the -j option.

But it's always worth measuring these things, so I copied the benchmark code up onto our multi-core number crunching machine, and did two runs-from-clean with and without the -j flag. No speedup. Checking top, I found that only one copy of the simulator or compiler was ever running at a time. What the hell? Time to look at the code:
TARGETS=build run collect clean distclean

%-eembc2: eembc-2.0
        @for dir in $(BMARKS_EEMBC2) ; do \
          if test -d eembc-2.0/$$dir ; then \
            ${MAKE} -C eembc-2.0/$$dir $* ; \
          fi; \
Oh God. Dear colleague, you appear to have taken a DSL explicitly designed to provide parallel tracking of dependencies, and then deliberately thrown that parallelism away. What were you thinking?¹ But it turns out that Dominus' Razor applies here, because getting the desired effect without sacrificing parallelism is actually remarkably hard.

What do we need? Well, we want to pass a command (which we're given in the name of the target) to every one of a list of make invocations. Explicitly listing every (command, benchmark) pair would work but wouldn't be very maintainable, and I'm developing a phobia of code-generation. It's easy enough to calculate lists of dependencies:
%-eembc2: eembc-2.0 $(foreach bmark,$(BMARKS_EEMBC2), $(bmark)-bmeembc2)
and it ought to be possible to pass in the command as a target-specific variable:
%-eembc2: command=$*
%-eembc2: eembc-2.0 $(foreach bmark,$(BMARKS_EEMBC2), $(bmark)-bmeembc2)

        if test -d eembc-2.0/$* ; then \
           ${MAKE} -C eembc-2.0/$* $(command) ; \
but that doesn't work: make does variable evaluation lazily, so $(command) isn't evaluated until the %-bmeembc2 rule is run, when $* is equal to the benchmark name! We can use secondary expansion to force the evaluation of $* in %-eembc2 rather than in %-bmeembc2, but this only works for dependency lists, not for target-specific variable assignments. So, we've got to bundle all the arguments we care about into the target name and then destructure it on the other side, using make's text-processing features. While we're at it, let's factor out some duplicated code:
bm_args=$(subst -, ,$*)
buildop=$(word 1, $(bm_args))
dir=$(word 2, $(bm_args))
version=$(word 3, $(bm_args))

        @if test -d eembc-$(version)/$(dir) ; then \
          ${MAKE} -C eembc-$(version)/$(dir) $(buildop) ; \
        else \
          echo eembc-$(version)/$(dir) does not exist. ; \

# In order to pass the build op to the prerequisites, we must use the second
# expansion phase.
%-eembc1: eembc-1.1 $(foreach bmark,$(BMARKS_EEMBC1), $$*-$(bmark)-1.1-bmeembc)

%-eembc2: eembc-2.0 $(foreach bmark,$(BMARKS_EEMBC2), $$*-$(bmark)-2.0-bmeembc)
and this one finally works. Testing it showed that all my cores were being utilised, and I got a 50% speedup overall - less than I'd been hoping for, but better than a slap in the face with a wet fish. There are still some subtleties here:
  • Since variable evaluation is done lazily, we can have buildop, dir etc as global variables and they'll pick up the correct value of $* when they're evaluated in %-bmeembc. Perhaps this is intuitive behaviour to you, but it wasn't to me.
  • Note the backslashes at the end of each line of shell: unless you do this, make will open a new subshell for each line in your recipe.
  • Why have a recipe just reading @true instead of a phony target? It turns out that pattern rules can't be phony.

That, I think, was much harder than it ought to have been. If there's a better approach to doing this in make, or a better overall design I could have used, please tell me in the comments!

But I wonder how redo handles this problem? I set up a minimal analogous problem, that of echoing the same string into a number of different files, and quickly came up with the following solution.

In default.all.do:
TARGETS="bar baz foo quux spoffle wilmslow"

export contents=$1
for i in $TARGETS; do
	echo $i.target
done | xargs redo-ifchange
In default.target.do:
redo-ifchange $1
echo $contents > $1
This is still too complicated and does too much work, though: we only want to regenerate bar, baz etc if the contents are going to change. A bit more messing about gave me a better solution.

In default.do:
TARGETS="bar baz foo quux spoffle wilmslow"

export contents=$1
redo-ifchange $TARGETS
In default.do:
redo-ifchange stamp
echo $contents > $3
and in stamp.do:
echo $contents | redo-stamp

We actually want the always-evaluate semantics in the original case, though, so we'd want something like the following (untested):

In default.eembc2.do:
source ../mk/boilerplate.od
export version=2.0
export buildop=$1
for bmark in $BMARKS_EEMBC2 do;
    echo $bmark.bmeembc
done | xargs redo-ifchange
and in default.bmeembc.do:
export dir=$1
if test -d eembc-$version/$dir ; then 
    make -C eembc-$version/$dir $buildop ; 
    echo eembc-$version/$dir does not exist. ; 
Note, incidentally, that redo also has a -j option, which is designed to coexist with make's.

Time to start teaching my colleagues about redo? I think it might be...

¹ He's also using recursive make, which means we're doing too much work if there's much code shared between different benchmarks. But since the time taken to run a benchmark is utterly dominated by simulator time, I'm not too worried about that.