% fortune -ae paul murphy

Traditional development and open source

"Everyone knows" what the only right way to do serious development is: start with a budget, then develop a formal set of requirements, write a specification, and ultimately progress to coding and testing as a kind of late stage after-thought. Duh. Except that essentially none of the world's most technically successful software was ever developed this way.

Consider Linux: it started as a runnable demonstration in an argument about whether x86 interrupts should be embedded in the Minix kernel - and you still don't see the Linux luminaries tossing around little UML diagrams when discussing change: they go from concept discussions to sample code and then discuss both what the thing ought to do, and how it should work, in terms of that sample code.

Consider the most successful open source application of them all: Apache. See any waterfalls? I don't - those guys responded to, and then drove change in, a fairly arbitrary third party server specification by prototyping their way to the top and doing documentation afterward.

Consider stuff that started as open source, went commercial, and then retained an open source element. Stuff like Sendmail - no waterfalls: genius, yes: an initial understanding of the requirement? sure; but separation of requirements, specifications, and coding? no.

Netscape offers an abject lesson in this. The original product, the Mosaic browser, grew from a typical open source development process at NCSA when Marc Andreessen and Eric Bina used CERN's PostScript dependent html interpreter as the prototype for their X11 implementation. During that process diagrams were sketches on napkins, real discussions centered on code and their product took the market by storm - but then came 1994, Jim Clarke, and the prospect of what became NASDAQ's third biggest IPO to that date.

As part of that process, Netscape, the company, gained "adult supervision" - and those supervisors insisted on formalising development processes. So what happened? progress stopped, Microsoft gained three years in which to respond, and the combination of easy cash on the way out with increasing internal hassles created such chaos that development, and therefore eventually the entire company, collapsed.

Eight years later FireFox is a contender again - and change discussions are once again couched in terms of code: not formal specifications, diagrammatic or otherwise.

There's a second common element here too: products that are intended to be used by well defined machines (the way a browser uses a web server or Sendmail uses a mail delivery agent) tend to achieve the most lasting successes. What's going on there, of course, is that such "users" define a core set of well understood requirements around which the developers build what they understand to be the right answers - a virtual prototying process which inevitably drives corresponding change in that initial specification.

Bottom line: the replacement of the traditional development process by active prototyping and specifications discussions couched in working code are hallmarks of open source success. The winners are the guys who are smart enough to deeply understand the core issues and then "just do it" - building highly functional prototypes, then getting others to build on those prototypes, and making no distinction between specification and code until after the product's initial success creates a third party demand for documentation.


Paul Murphy wrote and published The Unix Guide to Defenestration. Murphy is a 25-year veteran of the I.T. consulting industry, specializing in Unix and Unix-related management issues.