SmoothSpan Blog

For Executives, Entrepreneurs, and other Digerati who need to know about SaaS and Web 2.0.

Archive for October 21st, 2007

Pile O’ LAMPs: What Would Fielding Say?

Posted by Bob Warfield on October 21, 2007

I’ve been pondering the Pile O’ Lamps concept that I first read about in Aloof Architecture and Process Perfection.  Read the posts yourself for the horse’s mouth, but to me, the Pile O’ Lamps concept is basically asking whether a computing grid of LAMP stacks is a worthwhile architectural construct that could be highly reusable for a variety of applications.  I say grid, because in my mind, it achieves maximal potential if deployed flexibly on a utility computing fabric such as Amazon EC2 where it can automatically flex to a larger cluster based on load requirements.  If it is fixed in size by configuration (which still means changeable, just not as quickly and automatically), I guess it would be more proper to call it a LAMP cluster.

LAMP refers to Linux as the OS, Apache as the web server, mySQL as the database, and a “P” language (usually PHP or Python) as the langauge used to implement the application.  It has become almost ubiquitious as a superfast way to bring up a new web application.  There are some shortcomings, but by and large, it remains one of the simplest ways to get the job done and still have the thing continue to work if you move into the big time.  A Pile of Lamps architecture would presumably simplify scaling by building it in at the outset rather than trying to tack it on later.

In general, I love the idea.  People are effectively doing what it calls for all the time anyway, they just do so in an ad hoc manner.  I got ambitious this Sunday morning and thought I’d drag out Fielding’s Dissertation and see how the idea stacks up.  If you’ve never had a look at Roy Fielding’s Architectural Styles and the Design of Network-Based Software Architectures, you missed out on a beautiful piece of work from the man that co-designed the Internet protocols.  This particular document sets forth the REST (Representational State Transfer) architecture.  What’s cool about it is that Fielding has a framework that he uses to evaluate the various components of REST that is applicable to a lot of other network architecture problems.  See Chapter 3 of the Dissertation for details, but that is my favorite part of the document. 

His concept is to create a scorecard for various network architectural components, and then use that scorecard together with the domain requirements of the design problem to arrive at an optimal architecture.  He says that’s how he got to REST, and it certainly seems to make sense as you read the Dissertation.  Here is a rendition of his ranking criteria for the models he considers:

Fielding Framework

A “0” means the architectural style is beneficial to some domains and not others.  Positive means the style has benefit and negative means it is a poorer choice.

The components that make up REST look like this:

RESTful according to Fielding

There are 3 components that go into it:

  • Layered Cached Stateless Client Server:  The row marked LCS+C$SS
  • Uniform Interface, which isn’t in the original Fielding taxonomy, but which he says adds the qualities listed.
  • Code on Demand:  This is the ability of the web to send code to the client for execution based on what it requests.  So, for example, Flash or AJAX.

The “RESTful Result” is simply the total of the other attributes.  You can see it hits pretty darned well on most of the categories with the exception of Network efficiency.  As noted, this primarily means it isn’t suited to extremely fine grained communication, but is fine for a web page.  Pretty cool framework, eh?

Incidentally, Fielding’s framework really dumps on CORBA for all the right reasons.  Give it a read to see why.

Now let’s look at the Pile of Lamps.  Note that we aren’t trying to compare it to REST–they solve different problems.  Fielding tells us to do the analysis based on our domain, so put aside the RESTful scores, they aren’t meaningful to compare to anything but REST competitors.  Here is the result for Pile of Lamps:

Pile of Lamps

I view the LAMP stack as Layered Client Server, which is already a decent protocol.  A Pile of Lamps seems to me is basically adding a cached and replicated capability to the LAMP stack, so I add the cached/replicated repository to the equation.  You can see that it amplifies the LAMP stack while taking nothing away.  Basically, it makes it more efficient, more scalable, and it delivers those benefits in a simple way.  This makes total sense to me, given the concept. 

One can use the framework to fiddle with other potential additions to the Pile of Lamps idea.  For example, what if statelessness were pervasive in this paradigm?  I leave further refinement of the idea to readers, commenters, and the original authors, but it looks promising to me.  I’d also encourage others to delve into Fielding’s work.  It has application well beyond just describing REST.

Related Articles

A Pile of Lamps Needs a Brain

Posted in grid, multicore, Open Source, platforms, software development, strategy, Web 2.0 | 3 Comments »

Scoble Deconstructs TechMeme

Posted by Bob Warfield on October 21, 2007

I loved Robert Scoble’s post on reverse engineering TechMeme.  His description of how it works sounds right.  Scoble describes something very similar to a genetic algorithm, which I’ve talked about in this context before.  There is a set of sites that are the “fabric” of TechMeme.  What TechMeme tries to do is measure how many of these sites are talking about the same thing.  The more fabric sites are taking about a topic, the more it rises to the top.  This fabric concept is TechMeme’s equivalent of Google’s Page Rank.

This is like letting each meme written about in the fabric be a test organism.  The sites that wrote about the meme create a fitness function (along with other factors we’ll see in a moment).   The more sites that write, the higher the fitness score, and hence that meme is a more “successful” organism.

There are other factors that TechMeme uses that are interesting.  Scoble’s announcement of his new son got there because the fabric liked it and the words Scoble used mimicked the launch of a new technology, which is precisely what TechMeme is aimed at tracking, although Gabe Rivera says the system was mistaken to have flagged a baby as such.  There are other such tweeks, such as taking a view that a longer story is more authoritative than a short story.

The discussion of what TechMeme doesn’t consider or what look like weaknesses was particularly interesting to me:

  • It doesn’t consider video.
  • It doesn’t consider competing non-blog aggregators like Digg, Reddit, or YCombinator.
  • It isn’t very diverse.  If you aren’t in the “fabric” of A-List bloggers (Scoble guesses 2000-5000 bloggers), you aren’t considered.  An A-List can come along much later and you’ll be scooped if you aren’t in the fabric or unless a lot of people in the fabric link to you.
  • Story length matters a lot.  Too short or too long and you don’t get on TechMeme.
  • TechMeme favors stories about itself.
  • It doesn’t do foreign language sites.
  • Stories about small technologies don’t get considered. 

The overall picture reflects some interesting biases.  The fabric represents a “hidden leaderboard” that is pretty well hard-wired into the system.  Until that group picks up a story, it can’t move onto TechMeme.  That’s pretty directly the “echo chamber” effect that some like Todd Cochrane have complained about.  It’s basically a system that tells you what the top bloggers (as defined by TechMeme’s fabric) are talking about.  The question to ask yourself is how you value that? 

The top bloggers are not exactly hard to find.  They talk about each other constantly for one thing.  They have large authority on Technorati.  Are you reading them already?  Do you want them filtered for you by TechMeme so you only see things more than one of them talks about?  Are you sure the top bloggers are talking about everything you want to know about?

I like TechMeme, but I like it more as an analytical tool than a news finding tool.  Scoble reads 900 blogs (?!??), while I only read about 160.  Nevertheless, I make it a point to read TechMeme last, and I am rarely ever surprised.  Instead, I get to see which of the stories I just read from my other sources popped up there.  In other words, it isn’t a meme discovery service for me, it’s a meme weighting service. 

Posted in Marketing, strategy, Web 2.0 | 3 Comments »