I recently had a discussion with a colleague about the design of a feature our team was working on. One component was a particular sticking point for this fellow.

"Dude, why so complicated? You should read a book on Extreme Programming!"

Implicit in this colleague's comments was that this thirty-year veteran of the craft was getting long in the tooth and wasn’t keeping up with things...

Extreme Programming, for those that may not already know, is a software development methodology, a specific breed of "Agile" software development that some have characterized as the greatest thing since (fill in your favorite creation) and others have labeled as a flash-in-the-pan passing trend. Stripping away both the glitter and the mud, XP (as it’s sometimes referred to) is a tool, not a panacea. It has its strengths and weaknesses, and over-application of this methodology is as likely to kill a project as under-application.

In short, to know how to use the tool, it’s also necessary to know when to use the tool.

Now, my colleague’s argument was that I was "over-engineering" one aspect of the feature, seemingly in direct contradiction to one of the XP tenets that typically gets expressed as build only what you need, and no more. The idea is that XP, desiring to foster a light, efficient, highly responsive development environment, is built on short, incremental improvements of the project. Typically, you never know what’s coming down the road, so you should code to today’s requirements. Just make sure that the code is as ready as possible to adapt to whatever does come down the road at you.

On the whole, this is a good thing. But there are exceptions; circumstances when this practice breaks down. Like I said, XP is a tool. You have to know when to use it, and when not to.

To illustrate a circumstance when Build Only What You Need breaks down, let’s consider a maze problem. Imagine that you’re plunked into the middle of a labyrinth, and your goal is the Golden Tower, which you can see glowing in the darkness in one direction. The first instinct is to pick the path that gets you closer to the tower. If it’s to your east, you pick a path as close to eastward as you can, then keep doing that until you reach the tower ... or a dead end.

But imagine the following (topologically simplified) maze problem, the Corral in the Labyrinth:

As said, the darkness allows you only limited foresight, features getting less clear the farther away you look—a condition not unlike looking into the future—so it's good to develop a strategy. A rule or a dozen.

A simple but effective rule might be "head in the direction of the tower, and in no other direction." In this case, you will have the illusion of making progress right up until the end. But the tower will remain just beyond reach. You have to backtrack, basically undoing all the effort you put into going down that path, if you want to get where you're going.

When the entire maze is visible, it’s clear that the most direct route is also the incorrect route. This rule that works so well for general cases clearly fails in this specific case (and any more complex variant, really.)

In fact, there is no static strategy (rules set in advance) that will solve all mazes efficiently, because there can always be another maze that confounds even the most complex set of rules. (And besides, avoiding complexity applies to methodology, too!) A fairly simple methodology that solves a large class of problems is goodness, even if it has its flaws. Basically, that’s XP.

What do you do about the flaws? Most of the time, they’re just part of the process. Whether going through mazes or working on software projects, a bit of backing out, of throwing away code and starting over, is inevitable.

But imagine if a whole team is in the maze, and one of the members says "Hey! I’ve been here before. We need to turn left, then head toward the tower." Leveraging experience can be a huge benefit, as opposed to applying dogmatic methodologies, even if it’s not always obvious to everybody at the time.

So too with this component my colleague and I were discussing: it wasn’t immediately obvious why the apparent complexity was "needed"—at least not to someone who hadn’t been down this path before—and that’s understandable. The concerns are valid, but that doesn’t mean that rote application of a methodology is always the best course.

In other words, you have to know how much is too much.