SmoothSpan Blog

For Executives, Entrepreneurs, and other Digerati who need to know about SaaS and Web 2.0.

Archive for October 28th, 2007

A Pile of Lamps Needs a Brain

Posted by Bob Warfield on October 28, 2007

Continuing the discussion of a Pile of Lamps (a clustered Lamp stack in more prosaic terms), Aloof Schipperke writes about how such a thing might manage its consumption of machines on a utility computing fabric:

Techniques for managing large sets of machines tend to either highly centralized or highly decentralized. Centralized solutions tend to come from system administration circles as ways to cope with large quantities of machines. Decentralized solutions tend to come from the parallel computing space where algorithms are designed to take advantage of large quantities of machines.

Neither approach tends to provide much coupling between management actions and application conditions. Neither approach seems well adapted for any form of semi-intelligent dynamic configuration of multi-layer web application. Neither of them seem well suited for non-trivial quantities of loosely coupled LAMP stacks.

Aloof has been contemplating whether a better approach might be to have the machines converse amongst themselves in some way.  He envisions machines getting together when loads become too challenging and deciding to spawn another machine to take some of the load on.

Let’s drop back and consider this more generally.  First, we have a unique capability emerging in hosted utility grids.  These range from systems like Amazon’s Web Services to 3Tera’s ability to create grids at their hosting partners.  It started with the grid computing movement which sought to use “spare” computers on demand, and has now become a full blown commercially available service.  Applications can order and provision a new server literally on 10 minutes notice, use it for a period of time, and then release the machine back to the pool only paying for the time they’ve used.  This differs markedly from stories such as iLike’s, who had to drive around in a truck borrowing servers everywhere they could, and then physically connect them up.  Imagine how much easier it could have been to push a button and bring on the extra servers on 10 minutes notice as they were needed.

Second, we have the problem of how to manage such a system.  This is Aloof’s problem.  Just because we can provision a new machine on 10 minutes notice doesn’t mean a lot of other things:

  • It doesn’t mean our application is architected to take advantage of another machine. 
  • It doesn’t mean we can reconfigure our application to take advantage in 10 minutes.
  • It doesn’t mean we have a system in place that knows when it’s time to add a machine, or take one back off.

This requires another generation of thinking beyond what’s typically been implemented.  New variable cost infrastructure has to trickle down into fixed cost architectures.  For me, this sort of problem always boils down to finding the right granularity of “object” to think about.  Is the machine the object?  Whether or not it is, our software layers must take account of machines as objects because that’s how we pay for them.

So to attack this problem, we need to understand a collection of questions:

  1. What is to be our unit of scalability?  A machine?  A process?  A thread?  A component of some kind?  At some level, the unit has to map to a machine so we can properly allocate on a utility grid.
  2. How do we allocate activity to our scalability units?  Examples include load balancing and database partitioning.  Abstractly, we need some hashing function that selects the scalability unit to allocate work (data, compute crunching, web page serving, etc.) to.
  3. What is the mechanism to rebalance?  When a scalability unit reaches saturation by some measure, we must rebalance the system.  We change the hashing function in #2 and we have a mechanism to redistribute without losing anything while the process is happening.  We also must understand how we measure saturation or load for our particular domain.

Let’s cast this back to the world of a Pile of Lamps.  A traditional Lamp stack scaling effort is going to view each component of the stack separately.  The web piece is separate from the data piece, so we have different answers for the 3 issues on each of the 2 tiers.  Pile of Lamps changes how we factor the problem.  If I understand the concept correctly, instead of independently scaling the two tiers, we will simply add more Lamp clusters, each of which is a quasi-independent system.

This means we have to add a #4 to the first 3.  It was implicit anyway:

    4.  How do the scaling units communicate when the resources needed to finish some work are not all present within the scaling unit?

Let’s say we’re using a Pile of Lamps to create a service like Twitter.  As long as the folks I’m following are on the same scaling unit as me, life is good.  But eventually, I will follow someone on another scaling unit.  If the Pile of Lamps is clever, it makes this transparent in some way.  If it can do that, the other three issues are at least things we can go about doing behind the scenes without bothering developers to handle it in their code.  If not, we’ll have to build a layer into our application code that makes it transparent for most of the rest of the code.

I think Aloof’s musings about whether #3 can be done as conversations between the machines will be clearer if the Pile of Lamps idea is mapped out more fulling in terms of all 4 questions.

Posted in grid, multicore, platforms, strategy, Web 2.0 | 1 Comment »

Multitenancy Can Have a 16:1 Cost Advantage Over Single-Tenant

Posted by Bob Warfield on October 28, 2007

Multitenancy is one of those things that has been more qualitative than quantitative.  True blue SaaS believers put it down as a must-have to even call an offering SaaS.  Those less devout are understandably somewhat skeptical.  The heretics will say that liberal use of modern virtualization technologies is good enough.  Everyone seems to agree that the purpose behind multitenancy is to lower costs for the SaaS vendor.  These savings are translated into lower costs to their customers and a better shot at profitability.

In this article, I wanted to try to quantify those savings as well as put forward a definition for multitenancy that I think is clearer than a lot of what I’ve seen out there.  You’ve got the short answer in the title of this post, but I’ll walk you through how I got to the 16:1 value. 

First, let’s try to get to a useful definition of multitenancy.  Wikipedia defines multitenancy as follows:

Multitenancy refers to the architectural principle, where a single instance of the software runs on a software-as-a-service (SaaS) vendor’s servers, serving multiple client organizations (tenants). Multitenancy is contrasted with a multi-instance architecture where separate software instances (or hardware systems) are set up for different client organizations. With a multitenant architecture, a software application is designed to virtually partition its data and configuration so that each client organization works with a customized virtual application instance.

This definition is fine up to the point where the word “partition” comes up with a link back to a discussion of LPARs which are defined as follows:

In computing, a logical partition, commonly called an LPAR, is a subset of computer’s hardware resources, virtualized as a separate computer. In effect, a physical machine can be partitioned into multiple LPARs, each housing a separate operating system.

There is a lot of confusion here that comes from attaching a particular notion of how Multitenancy might be implemented (LPARs) with the concept itself.  I found the same thing reading SaaS-Man’s “Myths of Multitenancy” where the “myths” are based on particular implementation assumptions.  Let’s get one thing absolutely crystal clear before going further: there is no single implementation or architectural design pattern that can be called multitenancy.  Rather, multitenancy is an abstract concept largely rooted in benefits more than features.

To get away from implementation specific definitions (which are more examples than definitions), I am defining multitenancy as software that has the following properties from the perspective of three audiences seeking benefits:

For the SaaS Vendor:  Multitenancy is the ability to run multiple customers on a single software instance installed on multiple servers.  For the vendor,  operations can be performed at the level of instance, tenants, and users within a tenant.  This is done to increase resource utilization by allowing load balancing among tenants, and to reduce operational complexity and cost in managing the software to deliver the service.  For example, a patch can be easily rolled out to all tenants by patching a single instance instead of many.  Everyone’s data can be backed up in one operation by backing up a single instance.  The operations costs are then lower due to economies of scale and increased opportunities for automation.

For the SaaS Customer (a Tenant):  Multitenancy is transparent.  The customer seems to have an instance of the software entirely to themselves.  Most importantly, the customer’s data is secure relative to other customer’s data and customization can be employed to the degree the application supports it without regard to what other tenants are doing.  The only tangible manifestations of multitenancy are lower costs for the service, and better service levels because its easier for the service provider to deliver those levels.  The tenant will also want the ability to manage the system from their perspective (for example to manage security access among their users) and will want that to be as seamless as possible and certainly to never be impacted by other tenants.

For the SaaS User (one seat on the Tenant’s account):  Multitenancy is transparent.  They just see the application as a user of any application would. 

Further definiton around multitenancy gets into the realm of being too specific about implementations.  For example, there are folks who say multitenancy means an application won’t be highly customizable because it’s too hard to build.  It makes no sense to generalize about the properties of multitenancy on such a basis, so it should be left aside as something that relates to specific examples of multitenancy.  There are implementations of customizability that are incompatible with multitenancy, but I know it is entirely possible to build multitenant systems that are highly customizable if the proper metadata approaches and other customization tools are provided.  Eventually, I will write a post about how this can be done.

The details of a particular implementation of multitenancy can vary greatly from one vendor to the next.  It makes sense the details should vary just as Enterprise apps vary a lot from one domain to the next.  For example, most web software vendors like Twitter probably don’t think of themselves as Multitenant architectures, but they are with the degenerate case that a User and a Tenant are one and the same.

I bring up the odd example of Twitter because it is the start of one dimension that matters a lot for implementation choices:  number of users per tenant.  A single user per tenant is a nice fine grained unit.  In many ways it may be the easiest to create because load balancing and partitioning are so much simpler, although I don’t want to say they are simple by any means.  This is a meaningful class of SaaS, BTW, because most desktop productivity applications can be modeled in this way, so it’s relevant to Google’s view of what SaaS is, for example.

Next up would be to have a fairly small number of users per tenant. had an average of 21 seats per tenant when I recently did the calculation.  That’s very small.  If you are sure you’ll never have many more than that based on the nature of your application, you can largely treat this the same way as one per tenant with a few tools to make it easy to manage small groups of users as a unit.  Of course, Salesforce can’t do that, because they also have much larger customers than just 21 seats!

Let’s jump over to the opposite end of the scale, because it is an interesting perspective too.  Suppose the application will average more like 1,000 or even more users per tenant.  Suddenly the server utilization and automation arguments are a little bit less valuable.  We have 50x larger tenants than in the Salesforce example.  We have 50x fewer customers to reach the same level of seats.  Whatever repetitive tasks we may perform for every tenant have to be done 50x less often even if we do them manually on a tenant-by-tenant basis.  Is this an argument not to go multitenant for such applications?  Possibly, because multitenancy offers a lot less benefit to such a domain.  One could envision that virtualization and a lot of automated scripting could get us nearly all the way there, at least until we had so many of these large customers that it made sense to do something more sophisticated.  Interestingly, several SaaS vendors I talked to have mentioned that the management overhead for a customer is nearly the same almost regardless of size.  There are some operations that are size dependent, but a properly implemented SaaS app pushes as much of that sort of thing as possible back to the customer as self-service (for example, to manage accounts and passwords).

Okay, we have a working definition that’s very simple:

Multitenancy is the ability to run multiple customers on a single software instance installed on multiple servers to increase resource utilization by allowing load balancing among tenants, and to reduce operational complexity and cost in managing the software to deliver the service.  Tenants on a multitenant system can operate as though they have an instance of the software entirely to themselves which is completely secure and insulated from any impact by other tenants.

Let’s turn now to the question of estimating the cost benefits of a well constructed multitenant architecture.  I’ve collected statistics on the relative cost to provide service for a number of public SaaS offerings for which such numbers are available:

Cost of SaaS 

We can see a mix of business SaaS like as well as web software companies like Google.  The first takeaway is that costs can vary quite a lot from one company to the next.  This is a function of pricing (selling too cheaply relative to the costs of delivery raises the %) and of internal operating efficiencies.  It’s hard to draw much conclusion, so let’s just work from the average of 26%.  An average web software company can deliver $1 of revenue for about 25 cents spent on hosting and managing their software for customers.

Lest you be thinking that all of the Cost of Service is hardware, let’s consider the case of Google.  Gartner reports that runs on the order of 1 million commodity class machines at this time.  Even if we assume they’re replaced annually, which they aren’t, that places cost per machine at $4,225 and the same report says they spend about $1,800 on a machine, storage, and all hardware.  The rest of the cost is for network connectivity, datacenter physical space, and personnel to run all those little boxes.

I want to emphasize that we absolutely can’t be too cavalier about even the 1,000 seat per tenant case when it comes to operational efficiency.  At $50/seat, that customer is paying $600K of revenue per year.  If the SaaS vendor wants to spend 25% to deliver the service, they can afford to pay no more than $150K to keep the lights on for that large customer.  Much of that will have to go for the hosting (hardware, network charges, and datacenter).  What’s left is a fraction of an IT person for operations on this tenant.  Even a vendor with such gross numbers of seats per tenant must therefore be able to deliver the service very efficiently.  Considerable automation of manual tasks will be necessary.

Now let’s consider the costs of running traditional Enterprise on-premises software.  Talking to SaaS vendors about how they came by their pricing, there is a rule of thumb that says the annual contract cost of a SaaS offering is roughly equal to the perpetual license most vendors sell.  Oracle’s Timothy Chou says the industry averages are that it costs 4x the license per year to run traditional enterprise software.  Can you see how I got to a 16x operational efficiency advantage for SaaS over conventional enterprise on-premises?  Take Chou’s 4x and divide by 26% and you’re there.

I walk away from this analysis with two big takeaways.

First, SaaS vendors need to manage by the numbers pretty tightly when it comes to the cost of service delivery.  They’re very unlikely to hit the 26% average on their first release, and I’ve been told this by a number of young SaaS companies and their venture capitalists.  That means being on a continuous program of improvement.  If it was me, I’d want a dashboard of live metrics around these costs that really kept it top of mind together with trending so I could be sure costs were moving in the right direction.  Achieving a 16x advantage over conventional thinking is not a seat of the pants project.  It’s going to take some real effort and some real smarts to get there.   

Second, this advantage is an interesting concept for SaaS customers to contemplate.  Delivering a service efficiently means not screwing it up.  Escalations and problems will drive such costs through the roof.  The SaaS vendor really only has the choice of making the customer happy, and doing so in a highly automated way that ensures the happiness is no accident.  Isn’t this really the lesson Japanese car manufacturers used to become so successful?  Focusing on quality actually lowers costs.  SaaS vendors live and breath exactly this because they’re financially incented to do so.  On-premises vendors, by contrast, get paid a lump sum up front and don’t have this operational efficiency monkey on their backs.  The customer bears that cost, and apparently, it is a cost that’s 16x greater.

Is it any wonder SaaS vendors see multitenancy as their Holy Grail?

Posted in business, enterprise software, platforms, saas, strategy | 22 Comments »

%d bloggers like this: