SmoothSpan Blog

Sun’s CTO Greg Papadapoulos has been espousing what he calls the Red Shift theory of computing . Aside from having its own Wikipedia entry, the Red Shift theory has something to offer to a variety of audiences.  It gives permission to believe the computing industry is entering another period of hypergrowth.  It provides commentary on Moore’s Law, and which types of problems may or may not encounter the Multicore Crisis.  It gives a reason to believe Sun can once again regain its lost glories by leading the Red Shifted contingents.  And lastly, it provides yet another way to talk about whether your organization is a hip Web 2.0 “Red Shifted” organization, or whether you’re one of those oh-so-yesterday “Blue Shifted” deals.

What then is the Red Shift theory?  For starters, red and blue shift have to do with Doppler effects on light that tell us whether stars or galaxies are moving towards us or away from us and hence whether the universe is expanding or contracting.  Ignore all of that, it has little to do with the theory at hand, which has another meaning and simply uses the terminology as packaging.  Simply put, Papadapoulos postulates that demand for computing resources is segmented into a hyper growth “Red Shifted” segment and a much slower growing “Blue Shifted” segment.  In fact, the definitions for “fast” and “slow” have “fast” being growth that is much faster than Moore’s Law and “slow” being growth that is much slower. 

Slow growth is fueled by demand that grows, well slowly.  This demand is basic spurred by the use of computers to manage conventional financial transaction.  In other words, this segment is what most of the Enterprise Software Industry does today.  If much of your computer usage is built around this kind of activity, you live in a Blue Shifted world.  We shouldn’t expect much to happen in this world, and if we are bored with Enterprise Software today, it’s because too much of it is doing the basic plumbing for the Blue Shifted world.  That world isn’t suddenly going to wake up and start going gangbusters again, it’s done.  In fact, consolidation, virtualization and more power efficient components will be the dominant activities as enterprises try to reduce costs for core applications and services.  Virtualization et al will further reduce growth as the Blue Sector figures out how to use what it has ever more efficiently.  Growth here has regressed to the mean of GDP growth, which is slow indeed by computer industry standards.  End of an era.

The Red Shifted world is the more exciting world.  The thought leader for Red Shift is Web 2.0.  Not far out of the limelight are such applications as financial market simulations, drug industry research, and computer animation.  The shift to SaaS (growing 43 percent annually, according to a recent report by RBC Capital Markets), while it involves moving Blue Zone applications, will also deliver Red Shifted growth  because of the rate of conversion.  Their demand for computing is slated to increase at a rate faster than Moore’s Law, which is voracious indeed.  To make matters even more interesting, Papadopoulos goes on to argue that companies who embrace Red Shifted applications will grow much faster than those that stick to their Blue Shift knitting.  To paraphrase Will Smith, “I gotta get me summa dat Red Shift!”

Of course the theory goes on to describe a bright future for Sun which is well positioned to deliver scale efficient infrastructure, which they call “brutally” efficient infrastructure.  Microsoft and IDC agree with the vision, so they see something bright in their future around it too.  Sun goes on to project that at some point in the not too distant future, there will be just 5 massive data centers worldwide doing all of this business.  Wow!  Of course they haven’t heard about my web hosting plan for world domination yet, so maybe there will be 5+1 where the “+1” is a consortium of much smaller vendors delivering a shared utility computing fabric.

Personally, I like and agree with many aspects of the Red Shift Theory.  I’ve said many times during my Enterprise Career that Moore’s Law has passed up the growth in financial transactions and that this will lead to a cheapening of infrastructure cost for conventional Enterprise applications.  It’s about time too.  The centralization and skillset focus that SaaS brings to the table will bring further economies of scale to the table and make traditional computing still cheaper.  I also agree that bringing in the Web 2.0 collaborative-connectedness paradigm is another world changer, and one we are much closer to first steps on than SaaS.

There are some aspects of the theory I wonder about, however.  For example, Sun is obsessed that their “brutal efficiency” mantra means Big Iron/Big Servers.  When you have a hammer, everything is a nail. The trouble is that market experience seems to imply commodity computing is a lot cheaper than Big Iron.  Google, Yahoo, Amazon, et al are built on Lintel (Linux + Intel compatible) machines that are cheap.  Their infrastructure lashes together thousands of these boxes.  My experience at Callidus Software with our grid-computing based Enterprise software was that it was only the database that benefited from expensive Big Iron boxes.  Moreover, having written our app to run on its own mini-grid, we tended to minimize the DB as much as possible to the point where only 25% of the cpus had to be Big Iron DB-class machines.  Interestingly, the 25% often cost as much as the remaining 75%!  Because of this, I’m much more sold on utility computing infrastructures delivered on commodity Lintel stacks.  Just the sort of thing Amazon offers with EC2 and S3.

The other interesting viewpoint I came across was a number of folks who had concluded that the practical upshot of the Red Shift theory is that the future belongs to the database, not the processor.  The argument is that if you’re data intensive, you are by definition in the Red Zone, so companies like credit card processors are there.  I don’t agree.  Database scaling will be an important axis for Red Shifted companies to master, but the database is an effect, not a cause, and not all data-intensive activities like credit card processing will necessarily grow at rates faster than Moore’s Law.  I do think we’ll see some fundamental changes in how the world works with databases, and perhaps those changes will break Oracle’s hedgemony at the high end, but the DB is the tail wagging the dog.  I also think the data has huge value and we’re just beginning to think about the idea that the data may in fact be more valuable even than the software, particularly in a Web 2.0 context.  However, first you have to be doing something that generates all those data volumes, delivers truly valuable data, and that gets us back to the original Red Shift argument about which kinds of apps qualify.This is all just another side of the whole Multicore Crisis too.  After all, what do you do if your business is growing faster than Moore’s Law and you just bought the biggest machine they make?  Lest you laugh, this is actually what happened to eBay during the bad old days of their service outages.  We were located on their campus in Campbell and I used to watch the satellite crews set up to interview the eBayers about what had gone wrong.  Fundamentally, they had a monolithic architecture doing their auction search and once they had it running on the biggest mainframe Sun could sell them, they were stuck.  This precipitated a total rewrite to get things to horizontally scale.  I’m sure it was a harrowing experience, but we will see it play out over and over again as the Red Shift collides with the Multicore Crisis.

It’s an exciting world we live in!

p>

Submit to Digg | Submit to Del.icio.us | Submit to StumbleUpon