% fortune -ae paul murphy

Innovation in IT

I've heard a lot of comment, recently, about the lack of innovation in IT in general and in database and OS development in particular. I disagree and think we're seeing more innovation now than ever before.

This isn't true at the greeding edge where salesmen meet managers. "SOA" (service oriented architecture), for example, is about as real and innovative as reality television, but it is true where the lab meets production.

Looking at what's going on across information technology I see three great forces driving enormous innovation:

1. both Sun and IBM have embarked on strategies of massive hardware change likely to drive software and business adaptation for the post Intel generation.

   Sun's first "Niagara" hardware offers a 15x throughput improvement over their US3i, IBM's first cell based blades offer similar gains in floating point performance over their Power5. Both technologies call for revolutions in software, both rely on Unix, and both will ultimately change everything we think we know about deploying and managing organisational computing.

2. there is a revolution in access, freedom, and motivation affecting open source projects going on. As little as ten years ago most of the people working on new ideas in software did their work within the confines of their business, their faculty, or their hobby. The sense of ownership was strong, but progress relatively slow. Now sourceforge and freshmeat have essentially removed the barriers to entry facing people seeking "multi-spectrum" feedback and some help turning good ideas into production code. As a result there are now nearly 100,000 projects listed with somewhere around a third of those being actively worked on and one or two percent of them destined to eventually own their markets in the way that Windows/XP and Apache do now.

3. the geo-political forces driving markets for innovation are stronger now than at any time since the nineteen fifties. At the myopic end of this is the worldwide need that something be done to combat problems like SPAM and viruses, at the most globally significant end is the emerging political and economic battle for dominance between Communist China and the western economies. In between is the war on terror - and the consequences for social and economic change when a couple of hundred thousand American military return to civilian life to do what they've done after every major war: take a few of years to settle out, and then turn the economy upside down in a frenzy of business innovation and new wealth creation.

Imagine this big picture as a spinning globe and stab at something for a closer look. Pretty much anything you hit will show the effects of these forces. Look at database technology, for example, and what you'll see is enormous change. At one end hundreds of groups are nibbling away at applications of existing technologies; bridging the gap, for example between storage and retrieval technologies like those embodied in mySQL and presentation technologies like openGL and PostScript. At the same time, research ideas like UCB's back-up free temporal PostGres implementation of the early nineties are coming into commercial focus as big companies like IBM, Oracle, and Sun explore the cost and management implications of multi-terabyte storage needs -something SQL, as we know it today, won't survive.

Very little else will survive unscathed either. Personally I see the good guys winning, in technology and across the globe, but whatever your valuation of change it's probably safe to say that the next ten years will make the last twenty five seem like a period of great stability.