You are viewing pozorvlak

Beware of the Train - Falsehoods programmers believe about build systems [entries|archive|friends|userinfo]
pozorvlak

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Links
[Links:| My moblog Hypothetical, the place to be My (fairly feeble) website ]

Falsehoods programmers believe about build systems [Dec. 6th, 2012|09:45 pm]
Previous Entry Add to Memories Share Next Entry
[Tags|, , , , ]

Inspired by Falsehoods Programmers Believe About Names, Falsehoods Programmers Believe About Time, and far, far too much time spent fighting autotools. Thanks to Aaron Crane, totherme and zeecat for their comments on earlier versions.

It is accepted by all decent people that Make sucks and needs to die, and that autotools needs to be shot, decapitated, staked through the heart and finally buried at a crossroads at midnight in a coffin full of millet. Hence, there are approximately a million and seven tools that aim to replace Make and/or autotools. Unfortunately, all of the Make-replacements I am aware of copy one or more of Make's mistakes, and many of them make new and exciting mistakes of their own.

I want to see an end to Make in my lifetime. As a service to the Make-replacement community, therefore, I present the following list of tempting but incorrect assumptions various build tools make about building software.

All of the following are wrong:
  • Build graphs are trees.
  • Build graphs are acyclic.
  • Every build step updates at most one file.
  • Every build step updates at least one file.
  • Compilers will always modify the timestamps on every file they are expected to output.
  • It's possible to tell the compiler which file to write its output to.
  • It's possible to tell the compiler which directory to write its output to.
  • It's possible to predict in advance which files the compiler will update.
  • It's possible to narrow down the set of possibly-updated files to a small hand-enumerated set.
  • It's possible to determine the dependencies of a target without building it.
  • Targets do not depend on the rules used to build them.
  • Targets depend on every rule in the whole build system.
  • Detecting changes via file hashes is always the right thing.
  • Detecting changes via file hashes is never the right thing.
  • Nobody will ever want to rebuild a subset of the available dirty targets.
  • People will only want to build software on Linux.
  • People will only want to build software on a Unix derivative.
  • Nobody will want to build software on Windows.
  • People will only want to build software on Windows.
    (Thanks to David MacIver for spotting this omission.)
  • Nobody will want to build on a system without strace or some equivalent.
  • stat is slow on modern filesystems.
  • Non-experts can reliably write portable shell script.
  • Your build tool is a great opportunity to invent a whole new language.
  • Said language does not need to be a full-featured programming language.
  • In particular, said language does not need a module system more sophisticated than #include.
  • Said language should be based on textual expansion.
  • Adding an Nth layer of textual expansion will fix the problems of the preceding N-1 layers.
  • Single-character magic variables are a good idea in a language that most programmers will rarely use.
  • System libraries and globally-installed tools never change.
  • Version numbers of system libraries and globally-installed tools only ever increase.
  • It's totally OK to spend over four hours calculating how much of a 25-minute build you should do.
  • All the code you will ever need to compile is written in precisely one language.
  • Everything lives in a single repository.
  • Files only ever get updated with timestamps by a single machine.
  • Version control systems will always update the timestamp on a file.
  • Version control systems will never update the timestamp on a file.
  • Version control systems will never change the time to one earlier than the previous timestamp.
  • Programmers don't want a system for writing build scripts; they want a system for writing systems that write build scripts.

[Exercise for the reader: which build tools make which assumptions, and which compilers violate them?]

linkReply

Comments:
From: ungratefulninja
2012-12-06 10:11 pm (UTC)

(Link)

- All build systems look like make.
From: ungratefulninja
2012-12-06 10:12 pm (UTC)

(Link)

And on a related, $DAYJOB-aggravating note:

- Tools always exit with non-zero status on failure.
- Tools never exit with non-zero status on success.
[User Picture]From: pozorvlak
2012-12-06 10:13 pm (UTC)

(Link)

Aaaaaargh. I feel your pain.
[User Picture]From: DRMacIver
2012-12-06 10:24 pm (UTC)

(Link)

Though I'm mostly irritated by those tools rather than the build systems which get that "wrong"...
[User Picture]From: gareth_rees
2012-12-06 11:39 pm (UTC)

(Link)


  • A build always runs on a single computer.
  • There's always a human available to interact with the build system.
[User Picture]From: pozorvlak
2012-12-07 12:13 am (UTC)

(Link)

Good ones!
[User Picture]From: dr_strych9
2012-12-07 12:08 am (UTC)

(Link)

• Build tools are used by writing scripts in one or more languages.
[User Picture]From: dr_strych9
2012-12-07 12:13 am (UTC)

(Link)

• Tools should not require users to write build scripts in a language of any kind.
[User Picture]From: pozorvlak
2012-12-07 12:15 am (UTC)

(Link)

I'm inclined to say that the less scripting your build manager has to do the better - though a tool that doesn't require scripting in any situation sounds like an impossibility (and a tool that doesn't allow scripting sounds like it would eventually become painful).
From: senji
2012-12-07 03:14 am (UTC)

(Link)

  • Build steps are idempotent
  • It can be known in advance how many times a particular build step should be executed
  • Build steps do not modify the repository
  • It is possible to determine whether the build will succeed
  • It is possible to determine whether the build will even halt


Edited at 2012-12-07 03:21 am (UTC)
[User Picture]From: pozorvlak
2012-12-08 11:46 am (UTC)

(Link)

*smacks forehead* - all good ones.
From: (Anonymous)
2012-12-07 04:12 am (UTC)

(Link)

What is the point of this?
From: (Anonymous)
2012-12-07 01:31 pm (UTC)

(Link)

disabusement
[User Picture]From: pozorvlak
2012-12-08 11:46 am (UTC)

(Link)

Sorry that wasn't clear - I've added an explanation to the top of the post.
From: (Anonymous)
2012-12-07 09:15 am (UTC)

(Link)

maven sux.

svn sux.

python sux.

lets agree on this first

[User Picture]From: Jens Timmerman
2012-12-07 10:35 am (UTC)

(Link)

* Users will only will only use one specific compiler, library and flags, so you can hardcode them in your build scripts.
* Users will always agree with the location you want to install stuff to.

Shameless plug:
At my current job (HPC system administator at Ghent University) we have been building a lot of software which had these assumptions.
So to automate all of them we created a framework EasyBuild (http://hpcugent.github.com/easybuild/) which is a layer on top off all these build systems that tries to correct their mistakes (be the human to answer questions during the installation, patching makefiles/code to work with different compilers, install under a prefix and generate module files...) and automate the process of building the software.
This is not usefull for programmers, but for end users who want to install the software.

Edited at 2012-12-07 10:36 am (UTC)
From: (Anonymous)
2012-12-10 03:06 am (UTC)

(Link)

build systems are a symptom of software languages that are not designed to build software systems
[User Picture]From: dr_strych9
2013-02-25 06:28 pm (UTC)

(Link)

I hate make(1). I only slightly dislike OMake, which I'm sure would piss you off for a variety of reasons, but I bet it would piss you off less than make(1) too.
From: (Anonymous)
2013-09-11 04:40 am (UTC)

(Link)

Greate post. Keep writing such kind of info on your blog.
Im really impressed by your blog.
Hello there, You have performed an incredible job. I'll certainly digg it and in my opinion recommend to
my friends. I am confident they will be benefited from this web site.
From: (Anonymous)
2013-10-15 11:38 am (UTC)

(Link)

Thanks for sharing this great content, I really enjoyed the insign you bring to the topic, awesome stuff!
water systems (http://articlestwo.appspot.com/article/water-filtration-systems)
[User Picture]From: edhorch
2013-10-22 09:05 am (UTC)

(Link)

Javac is responsible for a lot of these. Why this may or may not be a Good Thing is a debate that could rage for years.

BTW, I'm working now with a horribly botched and broken build system that illustrates many of the listed bad assumptions, even though it's all C/C++ code. Surprisingly few of the problems are the fault of make itself.
[User Picture]From: edhorch
2013-10-22 09:08 am (UTC)

(Link)

One more thing: there is a special place in hell for designers of products that can only be built or configured through a GUI.
From: (Anonymous)
2013-10-27 12:38 pm (UTC)

(Link)

Hmm it looks like your website ate my first comment (it was extremely long) so I guess I'll just sum it
up what I had written and say, I'm thoroughly enjoying your blog.

I too am an aspiring blog blogger but I'm still new to everything.

Do you have any recommendations for inexperienced blog
writers? I'd really appreciate it.
From: (Anonymous)
2013-11-11 02:39 pm (UTC)

(Link)

Amazing blog! Is your theme custom made or did you download it from somewhere?
A design like yours with a few simple adjustements would really make my blog jump out.
Please let me know where you got your design. Thank you
From: (Anonymous)
2014-02-07 05:37 pm (UTC)

(Link)

Good post. I learn something new aand challenging on blogs
I stumbleupon everyday. It will always be interesting to read through articles frokm other writers and practice a little something from other websites.
From: (Anonymous)
2014-02-14 06:26 pm (UTC)

(Link)

The downside to the contributions is that it can take several years or more for you to accumulate sufficient
funds to purchase assets within the plan. In the business
world and investments, many people embrace this delicate piece of
metal due to the fact that gold done great performances.
The other option, going for a portion, can be guaranteed as well
but often it is perhaps not.
From: (Anonymous)
2014-03-16 03:53 am (UTC)

(Link)

xbox 360 emulator roms download