SmoothSpan Blog

For Executives, Entrepreneurs, and other Digerati who need to know about SaaS and Web 2.0.

Archive for October 29th, 2007

User-Contributed Data Auditing?

Posted by Bob Warfield on October 29, 2007

We’ve all read the headlines about user-generated content, which has become a big deal in the modern Web world.  The rap is that only about 1% of that content is any good.  While many writers are down on the idea, 1% of the web is huge and growing like crazy, so it matters.  There’s another rash of articles dealing with user-generated metadata or user-generated structure.  These are all worthwhile concepts that show how people using the web can add back tremendous value. 

It’s the Data, Stupid is the title of a thought-provoking WSJ piece that made be think of another wonderful user-generated contribution.  Ben Worthen, the article’s author, has this to say:

One of the hottest techs right now is software that helps business people view and interact with their companies’ data. But none of this software will help a lick if the data you’re working with isn’t any good.

This is a huge problem for businesses right now.  There’s an old joke making the rounds of BI circles to the effect that the answers gleaned from the first BI project was great right up until the second BI project started producing answers that didn’t agree with the first, at which point we were worse off than before the first when at least ignorance was bliss.  According to Worthen, Accenture has surveyed businesses and learned that only 29% make data quality a part of their initiative.  Having worked in Enterprise software for some time, I can tell you that most of the data one encounters out there has problems of one kind or another.  Nearly any project has to deal with cleaning up the data well enough for whatever purpose it is to be used for. 

But during my travels, I came upon a unique situation where users were actually incented to massively improve the quality of the data.  I’m speaking of the Incentive Compensation experience at Callidus Software.  Most people don’t think about it, but there is a huge amount of data that must be collected to calculate sales compensation that is directly relevant to many questions on the revenue side of the business.  The data mart associated with such systems can easily answer questions such as:

  • Who were the most important customers for each period and how is the relationship evolving in terms of repeat business?
  • What were the most important products sold?
  • Who were the best sales people?
  • What were the best territories?

There are many more examples, but this data was solid gold for our customers.  I got this news at one user conference some years ago when I was essentially mobbed by a group of customers.  One of them indicated that while their title was VP of Compensation, they were producing reports out of the incentive data mart for virtually every functional organization in the company.  The reason people were looking in the comp datamart was that the information there was unusually accurate.  Since salespeople were being compensated on it, and since they had visibility to see the data through a web page our software provided, an entire group of people were suddenly motivated to clean up the data.  It helped that our software could actually reward them for doing this by going back and restating results.  The  VP’s of Compensation for these customers were demanding tools to help them with delivering this information to others, so eventually we built the TrueAnalytics product.

The moral of the story is to find ways to incent your users to clean up the data and they will do so, spectacularly.  But they need visibility and a reason to care before it can happen.

Posted in business, saas, strategy | 6 Comments »

Practical Experiences With Never Rewriting Software

Posted by Bob Warfield on October 29, 2007

Dharmesh Shah says we should (almost) never rewrite software, and he is right.  Rewrites come about, he says, from the following causes (the comments are mine):

1.  The Code Sucks:  Given who wrote the old code versus the new code, why won’t the new code suck too?  If your code is brittle, refactor it.  Fixing brittleness is one of things refactoring does, and it does so without breaking the world for a long time as a side effect.  If things are so bad refactoring won’t help, you probably need to rebuild the R&D team first and then they’ll insist the code be rewritten.  The prognosis is not good for this sort of thing, but sometimes you can recover.  Unfortunately, the fish rots from the head.  Who hired the original team?  Are they hiring the new team?  Is the company really prepared to hold position for a major rewrite and team rebuild.  This is dire experimental surgery that I hope you’ll avoid at all costs. 

2.  “We’re So Much Smarter Now”:  There is often a grain of truth here, but what will it do to your installed based to do a major rewrite that expunges what you see as flawed assumptions?  Note that you can refactor more than just code.  Refactor the user model.  Get it right without breaking the old ways.  You can be smarter without boiling the ocean.

3.  We Picked the Wrong Platform/Language:  I haven’t seen this one come up so often in the way Shah describes, where new guys hired in want a change.  We’ve tended to focus so much on good chemistry for communication purposes that we hire birds of a feather.  See my article on using fewer people to build better software for more.  However, sometimes platforms are forced on us, and Shah gives similar examples in his write up.  I can think of three major examples throughout my nearly 25 year career.  First was client server.  There was definitely a time to shift from using a desktop DB engine to client server on various projects I was aware of.   Next was Windows versus character-based DOS UI’s.  Lastly was the Internet.  These are all such radical paradigm shifts that unless you were amazingly prescient or just an amazing architect, you probably had to contemplate some major rewrite-level work to get there.  I have also presided over a langauge shift.  We ported Quattro Pro (the DOS version, Windows was in C++) from an unsupported (but reasonable mature) Modula-2 compiler to Turbo Pascal while at Borland.  The short story is it wasn’t worth it.  It took a lot of work despite how similar the languages were, and despite promises of how much better code Turbo Pascal would generate, the product actually ran a bit slower.  You’d have a hard time convincing me a rewrite to change languages was ever worth it.

I want to go back to the refactoring.  This is something I’ve always intuitively done.  We used to call it “interative middle-out programming”.  It’s the idea that you keep things running at all times and your changes are measured transformations.  Even testing can be approached in this way, although that’s the subject for another post.  I firmly believe that schedules and product roadmaps need to allocate refactoring time.  This is time spent that delivers no feature a product manager, customer, or sales person could recognize as such, but that improves the code.  Usually, not too much can be allocated there.  I strive to hold out 20% on top of whatever schedule pad is there, and yes, I will sacrifice that 20% if we fall behind.  About every 3rd or 4th major release, one must invest more, say 40% in housekeeping.

If you do all that, it is amazing how long a code base can remain viable.  The Modula-2 version of Quattro Pro on DOS lasted for well over 10 years with no signs of distress.  Like a house, the code must have good “bones”.  That is, the top-down architecture has to be right and can’t fight you too much.  But if you have those “good bones”, you can keep running with the same code base, add tons of functionality, and the team is amazingly much happier too.  Part of it is that the refactoring time let’s them exercise their computer science tendencies and not just work on yet another mundane feature.  All great developers need to scratch that itch.  They want to put a clever algorithm or structure in place.  It makes the code better, but it makes the team better to.  Part of it is rotating developers onto new projects if they get too bored.  Let the new young Turks have a shot.  Make sure the new guys know they’re the ones paying the bills.

So remember:  budget time for refactoring, and skip the major rewrite.  Everyone will be happier!

Posted in saas | 3 Comments »