SmoothSpan Blog

Seth Godin is perhaps my all time favorite blogger, and I have recommended him frequently.  We mostly agree, but not this time.  His recent post is trying to convey the need to “just do it” as Nike says, without a lot of testing and analysis.  Godin is big on getting on with it and not giving yourself excuses to hesitate, and I don’t disagree with that sentiment, but that doesn’t mean you can’t test.

His recent post is about Netflix, a company whose founders I know well.  I remember sitting at the Starbucks in downtown Los Altos when Marc Randolph, who originated the idea for Netflix and was its first CEO, first told me about it.  I worked for Reed Hastings, who acquired my startup Integrity QA.  Reed met Marc because Marc was our VP of Marketing at Integrity.

One of my all-time favorite business anecdotes involves Marc Randolph telling an audience that failed to test a marketing program that went on to be a disaster that the end result could have been tragically knowable.

So let’s look at Seth Godin’s assertions about diving in and think about how to dive in prudently.  You should read his post, where he gives Netflix credit for being a culture of testing, but goes on to suggest that testing isn’t what made their success. 

Except they didn’t test the model of renting DVDs by mail for a monthly fee.

Seth, you’d be surprised at what can be tested and how.  My recollection is that there was some survey work done before the initial round of financing was raised.  You also have to consider that a startup itself, in it’s very early stages, is a test.  You’re financed for a fraction of the money that it takes to really know in order to verify that traction is available.  Yes, there will be serious ramifications in that your company may not get another chance to test something else, but if you go into it thinking of it as a test that has to succeed, you will understand how your Board views what you are about.

And they didn’t test the model of having an innovative corporate culture.

Well yeah, they did.  The culture at Netflix is an evolution of culture from Reed and Patty McCord’s (she runs HR) earlier days at Pure Atria.  Reed has always been a person who believed in people and culture more than almost anything and they got a chance to try even more interesting things at Netflix.

And they didn’t test the idea of betting the company on a switch to online delivery.

Sorry, Seth, but here to, it got tested.  They didn’t go cold turkey or whole hog (that’s some farm animals!) by dumping the mail model.  They ramped it up and if you don’t think they’d have backed off again if it didn’t work, you should get to know them better.

Testing is not a tactic, it’s a mindset and a culture all its own–one that’s key for startups and any companies trying to get into new markets.

What do you do absent enough information to make a sound decision?  There are a three choices I’ve seen employed:

–  You go with your gut.  By the time there is enough information, it’s too late.  Put all the wood behind the arrow your gut says is the one.

–  You go ask somebody.  Find a consultant or luminary.  Be careful how you ask or you’re just trading your gut for theirs.  Even the highest paid consultants don’t necessarily have the right answers.

–  You conduct rapid experiments to collect the data you need to confirm your early hunches.

I rank those in order from least to most desirable from my perspective.  Not much use for the gut.  It’s better than no decision at all, anything is.  But it is hugely risky and largely a matter of luck, or at least an inability to have enough introspection on your intuition to know why and explain it to others.

Where consultants are concerned, if they can’t explain the “why” by giving you new and well-corroborated data on which to make a decision, I wouldn’t take their gut either.  Most of them don’t do what you do every day.  At best they talk to people who do and may or may not have understood what they heard or know how to apply it to your circumstances.

The fast experiment approach is the one that avoids those Tragically Knowable mistakes.  I often see people moan and roll their eyes.  They just want to get on with making the big bet and rolling the dice.  Do the hard work.  Know the knowable.  Then make your decision and sleep well at night.  For those who say it isn’t knowable, you’d be surprised.

Related Articles

From someone who was there, more confirmation testing works!