I'm trying to figure out the intellectual lineage of the "cone of uncertainty".

There are several versions of this graphic. The one with the widest recognition is probably the one used to model the future paths of hurricanes. Another that makes some intuitive sense is the "binomial lattice" studied in the theory of real options. (See http://fr.wikipedia.org/wiki/Fichier:Arbre_Binomial_Options_Reelles.png )

In both cases this is diagrammed from a present-time perspective, with the cone widening toward the future. We have some certainty about what is going to happen a few seconds from now - in all likelihood it will be whatever is happening now, more or less. And the further into the future we try to peek, the murkier it becomes.

But what makes a lot less sense for me is the "inverted" cone, popularized by Steve McConnell after a diagram originally from Barry Boehm. (See http://www.construx.com/Page.aspx?cid=1648 for one presentation.) This is diagrammed from a future-time perspective, and shows uncertainty as a symmetrically widening range as we move further toward the present, which (given the usual convention for diagrams with a time axis) is on the left of the figure.

The diagram became well-known when it was published in "Rapid Software Development", in 1996. McConnell cites a 1995 Boehm as the source, and the diagram can in fact be found as early as 1981 in Boehm's writings, but it is a little unclear just what the empirical underpinnings are.

The Wikipedia article actually makes it harder to sort out the influences; for instance it references two ancient articles from the chemical industry from around 1958, but these refs are a very recent addition to WP and might be a case of citogenesis ( http://xkcd.com/978/ ), especially insofar as they imply that Boehm was influenced by these two earlier papers. I don't think that was the case at all.

Conceptually, the problem is that the diagram is symmetrical, meaning that it is possible for an early estimate to be an over-estimate or an under-estimate. This does not really square with widespread experience of software projects, which are much more often late than they are early.

What puzzles me is what the diagram is supposed to mean. A narrative interpretation goes like this: "very early in the project, if you try to estimate how long it will take, you're likely to end up anywhere within a factor of 4 of what the project will eventually end up costing". So a 1-year project can be estimated as a 3-month project early on, or as a 4-year project. Even after writing a first cut of requirements, a 1-year project can be estimated as a 6-month project or as a 2-year project.

I dispute that this latter case it at all common: a project that has reached this phase will in general take at least as long as has been planned for it, an instance of Parkinson's Law. The Cone suggests that the distribution of project completion times follows a well-behaved Gaussian. What the Cone also suggests is that the "traditional" project management activities help: "By the time you get a detailed requirements document the range of uncertainty narrows considerably."

My friend and former colleague on the Agile Alliance board, Todd Little, published empirical data in IEEE Software that contradicted the Cone of Uncertainty. Such data can be hard to come by, if only because it's hard to know what "officially" counts as an estimate for the purposes of measuring accuracy (Todd used the project manager's estimates, included in project status reports).

Todd's article seems to have generated a mini-controversy, summarized here by Brad Appleton:

This strongly suggests, to me, that the Cone of Uncertainty belongs to the "folklore" category - it is a belief that is hard to let go of precisely because it has little empirical or conceptual backing. It has perceptual appeal, insofar as it supports a message that "estimation is hard", but it also has a very, very misleading aspect: it suggests that uncertainty inevitably narrows as a project nears its projected release date.

Weinberg's analysis of "slip charts", and widespread experience, contradicts this. Many projects and tasks remain in the "90% done" state for a very long time. So if you wanted a diagram that truly represented how awful overall project estimation can be, you would need something that represented the idea of a project that was supposed to be delivered next year, for 15 years in a row. (Yes, I'm talking about Duke Nukem Forever.)

Apparently as a result of the controversy, and in a further departure from the original concept from Boehm, McConnell insisted strongly that the Cone "represented a best case" and that in fact, in addition to the Cone one should envision a Cloud of Uncertainty, shrouding estimates until the very end of the project. Metaphorically one "pushes" on the Cloud to get at something closer to the Cone.

By then though, that model has lost all connection with empirical data: it has become purely suggestive. It has no predictive power and is basically useless, except for the one very narrow purpose: providing an air of authority to "win" arguments against naive software managers. (The kind who insist on their teams committing to an estimate up front and being held accountable for the estimate even though too little is known.) But we should not be interested, at all, in winning arguments. We should be interested in what's true and in what works.

The "Cone" isn't really a good pictorial representation of the underlying concept that we want to get at, which is really a probability distribution. It has drifted loose from whatever empirical moorings it had thirty years ago.

How can we do better, not for the purposes of argumentation but for the purposes of effective management of our projects?
Shared publiclyView activity