Big Up Front Design Considered Scary Beyond All Belief

I had never before been able to put into words why I really hated it when a senior developer (Mr X) I worked with demanded I had a Big Up Front Design before I wrote any code, but I knew it irritated me to the core.  Because Documentation is a Good Thing™ right? Except that I am now trying to work alongside someone on a particular project (whom I shall call Mr Y), who is still under the influence of said senior developer.  I’m trying to help Mr Y  to produce a design which will satisfy Mr X so someone can eventually get around to the task of implementing the thing.

I’m not saying all design is bad, far from it!  I just think that most capable developers are really creative people who like to battle with the machine.  They are, in my humble opinion, considered capable because they’re constantly analysing their design and making tiny adjustments where necessary.  Even tiny things like changing that function parameter to be “const” because we’re not modifying it is almost entirely what I would call design.  The function would do the same job without that modifier, but now the signature more eloquently describes its intentions.  So what I’m really saying is that I don’t think design and implementation are separate activities.  This shouldn’t be revolutionary at all, but evidently from what I’m now observing, clearly there are some people who still believe it to be true.  Based on this thinking, I thought I was going to coin a new term: “Continuous Design”, but a quick google showed me there’s already a wikipedia page.  I guess I can’t add myself as a citation now!

So, let’s explore continuous design briefly.  To me, it’s the slight adjustments made as we discover new information – this necessarily and even especially includes the ‘implementation phase’.  If we remove ourselves from reality, as Big Up Front Design tends to do, the creation of the design document often becomes more important than the design itself.  That diagram needs to be neater.  This paragraph is too ambiguous to code.  If I’m being honest, I don’t care if you have a document explaining how you’re going to code something up.  I absolutely care if you have a document describing a protocol, and it would be nice to have a document explaining why you’ve coded something that way.  However, there are these handy inventions called comments which I am more likely to read, and indeed, trust. So designing continuously to me is that little bit of refactoring that you do to tidy things up.   It’s the slight adjustment that stops you from veering too far to one side.  It’s the comment or well chosen name explaining why this function exists! Sometimes it’s not even a conscious thought, it’s more likely labelled as “cleanup” but it all helps the readability, maintainability and extensibility – which are the only reasons we care about design, right?  Some of it is more deliberate – I’ll change the signature of that function to take in a buffer, because populating a global and returning a pointer to that is probably dangerous in a threaded environment.  But neither of those two cases deserves to be described in a document, so I hate it when people try.  Especially when they ask me to read this watered down version of the truth, which in the back of my mind, I know I can’t trust, thus asking me to waste my time reading it.  I think that this is the reason that I hate “interface definitions” prior to the implementation being started.  They’re often a watered down, happy path version of the true reality.

But getting back to the problem at hand; what I’m largely seeing is the context of decisions melting away.  So assumptions on how things work are starting to creep in, to the point where I had to start quoting areas of the code base in emails to drive my point home.  That scares me, because it means I’m having to read the code to them, which makes me worry what other context is being missed as part of this design.  I understand that we as a profession are really bad at reading code.  Most of us think we’re pretty good at it, but if we’re honest, true understanding of the behaviour only really seeps into our thick skulls after we attempt to modify it – that’s certainly true for me.  The less code you understand, the more assumptions will affect your design choices.  And unless you have some wonderful person on the other end of the email wire willing to quote the relevant pieces of code to you, your design will be broken.

beetlejuice

This context loss is all pointing to one very disparaging idea.  Mr Y is trying to design the extensions to the subsystem without getting his hands dirty.  He doesn’t understand many of the fundamental interactions because he hasn’t had to piece anything together.  He also appears to be using fear as a synonym for risk.  He seems to be mitigating the ‘risk’ by investigating himself right down the rabbit hole, producing many detailed documents, state machine diagrams and an almost endless supply of stories on the backlog to investigate areas he uncovers as he investigates.  As someone who understands the code, I was asked to review these documents to evaluate their usefulness as a fountain of truth, and their level of detail and complexity made me feel like the Maitlands from Beetlejuice in the picture to the right.  What I don’t feel he’s doing, is furthering his understanding (or indeed anyone’s who is tasked with reading these monsters)  to the point where he can make good decisions on his own.  So his fear is driving him to investigate rather than get stuck in and try something – which I don’t really blame him for.

Hang on a second.  How can it not be the fault of the individual when it’s fear driving decision making?  Simple.  The abuse and misunderstanding of several different viewpoints from the several different consultants hired recently have all collided into the impression that if we just do these couple of things like use Agile Methods, Set-Based Concurrent Engineering, Test Driven Development, Record Our Technical Debt, Continuous Integration, etc, etc then everything will be Just Fine.  These are all Good Things™, until they’re used as excuses for not doing anything.  These are all supposed to be ways of getting things done, not ways of avoiding making decisions.  Fear isn’t exactly driving the decision making, it’s driving him to make no decisions at all.  Making the wrong decision and causing instability, however minor, is such a hated thing in my company, such that nearly all of the “design output” I’ve seen to date has been trying to capture in excruciating detail the current state of play, in an effort to not put a foot wrong.  This is despite the fact that the “protocol” and the “why” documents were produced by the people who last worked/are working on it.  It saddens me that all these well-tested practices are being used as a way to slow down development, not speed it up, without actually producing anything real to date.

I think the point I’m heading towards is that Big Up Front Design is Scary Beyond All Belief because you can do it so easily in isolation.  In isolation from reality, isolation from the truth, in isolation from the other people who are working on the project and in the worst instances, in isolation of actually producing anything of value.

Advertisements

About Michael Malone
30 Year old web dev, self-confessed Linux lover, Ruby enthusiast, and obsessed with programming. Former embedded C and desktop .NET developer.

4 Responses to Big Up Front Design Considered Scary Beyond All Belief

  1. Wei Li Jiang says:

    Agreed!

    I think it’s invaluable to document intent, constraints, trade-offs, abstract models, and high level behavior. Anything more detailed is implementation detail, and very unlikely to survive the test of time, or even first commit to code. (An external API document is a possible exception.)

    When I want to review, change, or extend a chunk of code, I want to reference a document that describes what the code is trying to achieve, not an ambiguous, out of date, inaccurate, rehashed summary of the code. I want to know that even if somehow the code ends up as a dogs breakfast, the document will tell me how it should work and how to fix it.

    I think there is something to be said about prototyping here – on a complex and constrained system with existing requirements to meet, I wouldn’t trust a complex design that doesn’t have some concrete proof that it’s going to work.

    Let me review your facts, assumptions, choices, and approach. If you are competent, I should be able to trust you and your colleagues to do the right thing based on that. Anything detailed can wait until the code review.

  2. Ross Younger says:

    Too much documentation is counter-productive, as it’s practically guaranteed to be out-of-date as soon as it’s written.

    OTOH no documentation tends to be less than helpful as it means a newbie looking at that code has no idea where to start, and don’t get me started on comments that are more boiler plate than wording, or useless ones that came from the fake plastic comment factory.

    There is a middle ground, which I like to call _smart documentation_, which gives you just enough of a leg up to let you hit the ground running.

    It’s akin to a roadmap. It’ll have some sort of list of the major functional parts of the system and some pointers to how they inter-relate – perhaps a high-level architectural description (layer A is linked with layer B, layer Q might live in a separate device so uses this other message-passing system, see this research paper for more). Maybe also some notes on the pitfalls that have been encountered, something like “we tried the foobar algorithm but we hit a suboptimal case and it often deadlocked on us”.

    It may live with the codebase, in the same revision control system, so that if you do have to get out your virtual trowel and start digging through history, you can immediately see what was current then. It’s probably quite easy to maintain (plain text, or some simple text-based mark-up language) so is more likely to be kept up to date when things change or as more is learnt.

    Sadly, it’s all too rare…

    • @Ross: Yes, the smart documentation is a nice idea. The boiler plate docs I’ve seen have been trying to encourage that, but sadly they try to be so generic and “part of the standard” that they contain everything and few people feel comfortable in deleting great swathes of it. Doxygen, I think tries to solve the problem you’re talking about with the “keep the docs with the code”, but nearly misses the point entirely. Doxygen only really does a good job of publishing APIs, not of capturing “why” or “protocol” documents. (I keep mentioning protocol documents, because they’re easily the documents I most frequently reference). I agree that some sort of text-based document would be a nice way to go, as they play more nicely with version control and as you say, can be seen over time. Do you know of any light-weight text markup which could be transformed into something pretty?

      I like the idea of documenting that which didn’t work, which is something I must admit I’ve not done enough of in my time, so thank you for pointing that one out! I feel like that should have been obvious to me, but it’s a fine line, I suppose.

      @Wei Li, yes the design should be inherent in the code itself, but if it’s not, I want it somewhere else too. And yes, the design should *always* be a simplified description of the implementation. If it has the same complexity as the code, then the inexactness of natural language will get in the way and actually make it worse.

      • Ross Younger says:

        At a previous job we used DocBook, which is great for pretty output, but I’d call it as heavier weight than HTML. Maybe MarkDown fits the bill? It only outputs XHTML, but (of course) fully styleable so might even be turnable into something plausible on paper or PDF without too much effort.

Comment on this

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: