The Second Virtue

As I discussed in my previous post Unit Testing is a Good Thing.  But once you’ve got enough testware, it starts to become a burden.  At that point, most companies worth their salt will invest in some high-spec hardware on which to run the continuous integration system.  This is seen as an opportunity to increase productivity by knowing earlier when some code is wrong.  This is sound logic.  But this doesn’t solve the problem for the developers trying to run the tests before they check in said breakage.  Larry Wall’s second virtue, Impatience, starts to come into play quite significantly here.

I have personally spent a lot of time and extended effort into making a large, complicated C code base build and test as accurately and quickly as possible.  And even then, with the build spread across eight 12-core blade servers, each with 32 Gb of RAM, we manage to build and test in around 20 minutes.  Now, even though the developers have high-spec’d four core boxes, they still can’t even come close to the raw power of eight server-class machines, even if they can borrow some of their CPU time with distcc.  We’ve therefore created a partial build which attempts to build just enough to catch around 90% of the errors with only a small portion of the required effort and even that can take upwards of half an hour.

Where, then, does the balance sit?  How much do we require gets built for every change before every developer is slowed down enough such that it actually becomes faster to just check in and hope (for a given value of hope).  And anyway, how many people are updating so often, without taking note of the readily available state of the build, such that they’re feeling the pain of the build being broken.

Read more of this post