SmoothSpan Blog

Jason Lemkin says forget building wireframe UI’s and start out interviewing 20 customers, because you just won’t understand your customers until you do.  Here’s the gist of why you need 20 interviews before you do anything else:

And you have to do 20.  I know it’s hard to get to 20.  But it’s the right number:

• You need the First 5 Interviews just to truly understand the white space and the current opportunity.  Yes, you probably think you already understand it.  But you are the vendor, not the purchaser.  You need to understand your prospective app from the purchaser’s perspective, for real.
• You need the Next 5 Interviews to confirm your pattern recognition.  You learn from the first 5, you confirm in the next 5.
• You need Interviews 11-20 to Nail Your Pitch and Hone Your Thesis.  Once you truly understand the white space from a buyer’s perspective, and you’ve figured out the nuances and challenges … it’s time to nail your pitch for real.  And by doing this, you’ll also hone your thesis and strategy.   That’s what interviews 11-20 are.  To get real critical feedback on what you’ve learned.  To learn about corner cases that may in fact be critical insertion points for you to win.  To dig in on what is really 10x better, not just 2x or 5x better.

And let me tell you, at least from my experience, don’t expect all 20 to be positive.  Many of My 20 Interviews in both my start-ups were very critical.  Or worse, lukewarm.  Lukewarm is even worse, because it says yeah it’s sort of interesting … but no way I’d buy … and implicitly … your idea is a huge waste of time.  I’d rather get the negative feedback 😉

I get the Steve Jobs thing.  You just have to build it.  You do.  But this is SaaS.  You’re solving a business’ problem.  They don’t know how to solve it, or what you should build.  But they do now how to express their problem.  Acutely, and thoughtfully.

Here’s the funny thing–in some ways I agree with Jason and in others I totally disagree.  It depends really on what it is you’re trying to learn from your 20 interviews.  Jason says you’re trying to understand the white space and the current opportunity and that you’re trying to nail your pitch and hone your thesis.  He’s thinking about it like a Sales Guy, more power to him, someone in your company ought to be.  But there’s more to life and startups than Sales Guys.

Here is what I worry about validating in the early days of any startup:

1.  What is the problem we’re trying to solve for customers?

2.  Is it a real problem?  Do a large portion of customers believe they have this problem?

3.  Will the solution we’ve imagined actually solve that problem?  Do the customers agree that it solves their problem?  Can we charge enough to make a real business for this solution?

4.  Do we have a pitch that communicates we have a real and effective solution quickly?

I see Jason’s 20 interviews as helping to solve #2 and #4, but not really making much impact on #1 and #3.  Moreover, I see #4 as being pretty tactical, unless, of course, you need that finely honed pitch to convince investors.  It’s tactical because we won’t need it until it’s actually time to get customers to deploy beta product.  In other words, we have quite a lot of time to solve it.  #3 is not so tactical.  In fact, if we don’t solve #3 right up front, we could easily spend the bulk of our time building a product that we think solves the customer’s problem but that customers aren’t confident in.

Let’s drop back and work on each one of these 4 critical questions and see how and when in the startup cycle they should be tackled.

1.  What is the problem we’re trying to solve for customers?

This is a tough one for many entrepreneurs.  I’ve seen a few decide to try to find a hard problem to solve by interviewing potential customers.  What keeps you up at night?  What do you hate about your job?

Ugh.  That seems so hit or miss.  None of the ones I met who’ve tried this got very far with it.  They got problems that software was just simply not the cure for or they got problems that are more conditions of the human race than anything.  Jason says customers, “Know how to express their problem.  Acutely, and thoughtfully.”  Actually, they don’t, not so much.  They know how to resonate with a problem that you’ve stated acutely and thoughtfully.  They know how to resonate when you state a problem as a near miss to how they really think about it.  But customers are not product designers nor company founders for the most part.  They’re severely myopic.  As Henry Ford famously put it,

“If I had asked people what they wanted, they would have said faster horses.”

So don’t rely on customers to tell you what the problem is.  Really on them to confirm you’ve found a problem they feel.  Later, they’ll remember it as you having discovered the problem from them, but that’s not really how it works most of the time.

There are a lot of reasons for this.  The myopia is one but another is they just don’t know much about the medium you work in–software.  They have little idea what software can do for them.  They think of most software in terms of software they already have, which is another form of myopia.

You are not going to figure out that problem, in all likelihood, through a few simply interviews.  You’re going to have to live the life of your customers for a while, walk in their shoes, and feel their pain.  You need to know their domain intimately, which is not something that’s going to happen in 20 interviews, no matter how good they are.  That’s how two great Sales Guys wound up creating two great CRM companies–Tom Siebel with Siebel Systems and Mark Benioff with Salesforce.com.  I interviewed with Tom Siebel when he had less than 20 employees and the one thing that was absolutely clear about the man was that he knew the domain and its problems cold.  That’s the kind of solid domain knowledge you really should have in your startup.  Who can you point to who has lived and breathed the problems and knows them cold?

Is that the only way?

No, there are plenty of exceptions.  There are proxies available too.  Finding a large online community of your desired customer can give you huge insights into what their world is all about.  What do they ask questions about most frequently?  What do they complain about frequently?  These communities are so helpful to startups both for gathering information as well as for getting out the word that I’m not sure I’d want to do a startup that couldn’t identify an online community specializing in its customers.  In this age of Content Marketing, it seems to me that such a community would be a very valuable indicator that the market was going to be reachable and at reasonable costs.  I don’t want to have to advertise or cold call my way into existence, though many have certainly done so.

To put this into the perspective of Jason’s 20 interviews, you need domain knowledge for your startup before anything else.  You won’t get it from the 20 interviews, and it is just table stakes that you have to find.  Maybe you’re counting on a founder for it.  Maybe you have the world’s ultimate advisory board.  Maybe you’ve sold to these poeple in a former life and know all about them.  Maybe you’ve spent a year studying and interacting with their online communities.  Whatever it is, you’d better have it.

2.  Is it a real problem?  Do a large portion of customers believe they have this problem?

Having gotten your domain knowledge together, you believe you’ve discovered a real problem.  Now that you can articulate that problem, it’s time to confirm it with potential customers.  Jason’s 20 interviews are perfect for this stage.  I don’t know that there is anything magical about 20 or that this even has to be done via interviews.  If you’re going to be using feet on the street to move your product (e.g. a scratch golfin’ highly paid salesforce), you should probably get started with interviews.  If your Sales Guy can’t line up 20 interviews in his sleep, there is something wrong, so may as well set him to the test.  If you weren’t planning to get a Sales Guy until later and you can’t line up 20 interviews on your own, you better get the Sales Guy sooner.  There’s a lot of good that comes from achieving the 20 interviews and here Jason and I agree wholeheartedly.

But in addition to 20 interviews, I highly recommend a few other things.

One of my mentors has used the method multiple times of creating a web site that’s all about the proposed problem and solution.  He sets it up like there’s a product ready to go, except there’s no way to order it.  You can simply request more information.  He gauges the quality of the idea by whether he can get many information requests.  And yes, he understands the need to do some tuning up before giving up.  This is a case where having online communities of your potential customers is awesome.  If you have a simulacrum site as described (looks like a real company and product), you can ask the folks in the community what they think of the company and idea.

If you don’t like that approach because it just feels a little too funky, try my approach.  I started a blog before I started my current company and I waited to see if I could drive significant traffic to that blog discussing the kinds of problems I wanted to solve.  I would talk about how people were solving their problems today, rather than how I proposed to solve them.  I measured traffic using all of the standard web analytics to see what resonated and what did not.  I built a lot of credibility and I had a following already in place that I leveraged to a significant Beta test and significant cash sales when I was finally ready to launch the product.  But, before I started building the product, I knew from how people were reacting to my writings that I understood the customers and their problems very well.  I achieved what I call “Content-Audience Fit” (a precursor to Product-Market Fit).  More about that below under #3.

3.  Will the solution we’ve imagined actually solve that problem?  Do the customers agree that it solves their problem?  Can we charge enough to make a real business with this solution?

Here’s where I probably disagree with Jason the most.  He says, “So if you haven’t started yet, as fun as it is to just build the wireframes and get a codin’ … do the 20 Interviews.”

Here’s my problem–as I’ve mentioned, the Customers really can’t articulate their problem unaided.  They can only resonate with your articulation.  We may have to agree to disagree on that, but figuring out how to resonate well is a huge function for Sales precisely because the customer can’t do it for themselves.  They often need help even in how to present it to each other to get buy-in within their organizations.  When you go interview them, you’re going to get a variation on the three blind men encountering an elephant for the first time.  One touches the trunk, and thinks it’s a snake.  Another touched the tale, and thinks it’s like a straw fan swishing back and forth.  While the third touched the legs and pronounced that elephants are like trees.

When it comes to envisioning a solution, especially in software, things get far worse.  At least they have all experienced the problem, even though it may appear to be a snake, a fan, or a tree.  But nobody outside your company even has a glimmering about your proposed solution.

I’ve done lots of focus groups over the years and lots of the kinds of interviews Jason talks about, and I am here to tell you it is pointless to do either if you expect to get feedback about a software solution unless you have at least some wireframes and storyboards to show.  With modern tools, it’s just not that hard to produce these things.  I’m working on our third product at CNCCookbook.  I was able to put together a UI prototype that is substantially what our finished product will be with about 6 weeks worth of effort.  It’s been hugely valuable in securing feedback for the product and I have learned an awful lot from it.  Perhaps the biggest problem it has is people start to mistake it for a finished solution or they assume it’ll be ready much sooner than it will and they get too excited.  They’re ready to buy immediately.

Yes, it may be fun to build wireframes, but it is also fun to have real meaningful conversations with prospective customers about your proposed solution.  You’re going to learn so much more about everything if you can do that around a real demo.  You’ll learn about positioning, sub-problems and edge cases that as Jason says are critical insertion points to win.  You’ll learn whether your proposed solution really fixes the customer’s problem in their eyes.  You’ll feed on the enthusiasm they have for what they see, and that enthusiasm is valuable fuel for your startup, for convincing critical new hires, and for any potential investors you may have.

If it’s really going to be hard for you to get interviews with 20 real prospective customers, people who are solid citizens in their markets and who are not your buddies.  People who will give you the straight scoop, help guide you, and who you’re hoping will be early adopters.  If it’s that important and that tough, I can’t imagine wanting to do it without bringing along a UI mock up.  It isn’t just my current venture, I have done this at every single one of the 7 startups I’ve been with.  I have done it for every new product release.  It’s actually integral to how I approach the Agile Software experience.

If your Sales Guy can’t get 20 meetings, get another one.  If your Product Guy can’t get you a UI prototype quickly, get another one there too.  Both are equally as important.

One more thing–that last part, “Can we charge enough to make a real business with this solution?”  That’s critically important to answer ASAP.  It’s also nearly impossible to get a very good answer to it without a UI prototype.  Yes, they will give you some answers, but I am talking about real answers that will hold water when it’s time to cash the checks. Don’t you want to know whether customers will pay up for what you’re going to build as early as possible?

4.  Do we have a pitch that communicates we have a real and effective solution quickly?

This is what I call “Content-Audience Fit“.  I believe achieving that fit needs to come ahead of finishing the product if for no other reason than that you won’t know if the product is finished nor will you be ready to efficiently leverage content for product traction unless you do.

Jason’s 20 meetings go towards this end, but it’ll take more than 20 to really nail your pitch.  Personally, I don’t like to burn real perspective customers, investors, or other scarce as hen’s teeth resources for a company if I can find another way to test this stuff out and perfect it.  The online world and Content Marketing are your gateway to doing so. Once you can resonate with those audiences, break out the Rolodex (kids you’ll have to look up what that is) and start asking for meetings.  You’ll be bringing to that meeting a laundry list of good-as-gold asssets:

–  By this point you have a UI prototype to demo.

–  You have verified that you can talk about the problem and with the audience with enough credibility that they’re at least starting to come to you for answers.

–  You’ve had the opportunity to test a number of things with your growing audience.  You’ve probably even been able to do some surveys.

–  You have a corpus of content that you can point potential meeting invitees to that helps establish your bona fides and gets the conversation off on the right track.  If done right, this can be a warm call and not a total cold call.

Most importantly, none of this is all that expensive or time consuming.  You can do it on your own nickel without waiting for a Series A VC round.  I know I have more than once.  I’d set a goal of 3-6 months to get to this point.  If everyone is firing on all cylinders, you’re producing good content, you’ve gotten through your UI prototype, and you’ve made contact with a decent sized audience, you’ve accomplished a lot at your startup.  You’re right where you need to be.  Now line up those 20 meetings, get in there and make those 20 people your first Beta tests, and hit the ball out of the park for them.  They’ll love you for it and you’ll have set the stage for your next 6 months as you drive to launching the Beta and eventually real Sales.