Last year, Steve Blank challenged Lean Startup Circle members to update his customer development checklist (Appendix B in his book) for their specific business. His checklist is built for Enterprise Software which doesn’t readily translate to other types of startups, including web startups (especially when you get to Customer Validation).
After the talk, David Binetti pulled together a great group of people from Lean Startup Circle to create a Customer Development Checklist for web startups. I was honored to be asked. However, after a few exchanges trying to nail down the type of web startup to tackle first, we uncovered several tactical differences that made me realize that defining such a model in a group setting is too hard to do. We would either end up with a highly generalized model or no model. I, for one, am not comfortable extrapolating a model from third-party accounts of what might have worked for other companies (like 37signals), especially not without first-hand experience building a similar type of startup.
So I decided to take a stab at defining a Customer Development Checklist for my web startup.
First Some Background
This model is based on my experiences building and running two products: BoxCloud and CloudFire. Both use a subscription pricing model. BoxCloud was built using a release-early release-often development model and was initially launched with a Freemium pricing model — later changed to a free-trial-only model. CloudFire is being built using a lean-startup/customer development model and was launched with a free-trial-only model.
For a SaaS product like mine, I strongly believe you need to
a) charge for your service, and
b) validate pricing sooner rather than later.
Free trials, Freemium, free introductory periods, etc., are all tactics to lower sign-up friction and should be applied (split-tested) judiciously on a case-by-case basis. However, my key takeaway is that even if you’re considering Freemium, you should validate the premium part of Freemium first before giving anything away.
I’ll cover Customer Discovery in Part 1 and Customer Validation in Part 2. Hopefully, I’ll get to write Parts 3 and 4 one day.
Customer Discovery: What should I build and for whom?
Here’s my Customer Discovery Flow (you’ll probably want to click to enlarge and skim it before reading on).
The 3,000 Foot View
For a web startup, the purpose of Customer Discovery is to identify a problem worth solving, define the “right” minimum viable product to build, and test the business model using three separate Build/Measure/Learn loops. Most web startups rely on a product website for distribution and blogs, SEO, and SEM, for initial customer acquisition channels- leaving price as the biggest unknown in the business model.
Sidebar: There is a somewhat loose definition of how the term MVP gets used. Many have used it (myself included) to refer to anything (a landing page, a problem presentation, screenshots, etc.) that allows you to learn about customers with the least effort. Here, I am using the stricter definition of MVP to mean the minimum set of features needed to learn from earlyvangelists. In other words, Release 1 of your Product.
It All Starts With Stating Your Assumptions
Before you can test what you think you know, you have to write it down. It’s normal to try and shorten this step, but I found it to be a very worthwhile exercise. Apart from minor terminology changes, most of Steve’s questions in his hypothesis worksheets hold up even for a web startup. I will not reproduce them here, but if there is interest, I’ll make my versions available as a separate download.
Test the Problem
This will look very similar to Steve Blank’s flow. That’s because when it comes to Customer Discovery, I haven’t found a more effective way to maximize validated learning than “Getting out of the Building.” This comes from someone who used to prefer to stay in the building and talk to customers over email. Now, I have an 800 number tied to my mobile phone and schedule face-time opportunities with customers.
With my last product, I used a teaser/landing page with some pre-launch buzz to collect email addresses and measure interest. While it was encouraging to have interested users, it told me nothing about their problem, who they were, what I should ship first, or what I should charge. What do ten email addresses a day tell me? What do 20, 40, and 100? Why did the other 70% abandon my landing page? Was it the product, was it the copy, the graphics, or something else? What?
Building a good landing page is hard. Unless you have exceptional customer insight (or are your customer), iterating without talking to customers is slow and painful as you split-test one page against another with very little initial test traffic. Yes, it feels like more legwork to find people who will have a conversation with you but a 15-minute unscripted conversation has more validated learning pound for pound than all the data you can crunch from web analytics.
I have never tried using surveys with my landing page registrations because I hate filling them out myself. Plus, to make them easy to fill, you have to be very specific, which assumes you know exactly what you want to know, which is hardly the case.
The beauty of Steve’s process is that it tests the problem separately from your solution.
To paraphrase Dave McClure:
Customers care about their problems NOT your solution.
During the “Problem Presentation,” you state the top 3 problems, then shut up and listen, which is key to getting it to work. It works because you aren’t asking customers to validate or design a solution that addresses the “Customers don’t know what they want” argument. It works because it isn’t a pitch. I take that back. It is a pitch. But it’s the customer that’s pitching their problems to you. I know I’ve hit the right “problem nerve” based on how passionate a customer gets during an interview.
I like to structure my “Problem Presentation” like this:
1. State the top 3 problems
2. Ask the customer to prioritize problems and identify any higher-priority problems
3. Have the customer describe how they solve the problem today
4. Very briefly describe how you might solve the problem
5. Ask the Customer whether your approach would solve their problem
6. Would they use your solution if it were free?
7. Would they pay $X/yr?
8. Ask for referrals to other customers
Build Your MVP
Unlike Enterprise Software, which can be chock full of features, web startups need to focus on the smallest feature set needed to learn from earlyvangelists or the MVP. After the first reality check, you should end up with a prioritized top 3 problem list that drives the features for your MVP. I stress the importance of then building out the MVP to the point where it’s demo-able. It will be hard for customers to visualize your solution without one. Screenshots and mockups may be used as stand-ins only if a demo is absolutely out of the question.
Test Your MVP
With the MVP built out, you then go test it against the original set of interviewees plus some.
I like to structure my “Product Presentation” like this:
1. State the problem
2. Use the demo to tell a story of how your solution solves the problem
3. Test pricing again
4. Ask for referrals to other customers
5. End with a call to action: sign-up, or commitment to sign-up
Tip: I practice delivering the demo using screencasting software, which lets me iterate until it’s short and crisp, but I also end up with a video I can use later on the product website.
Iterate or Exit
The last step in Customer Discovery is to summarize what was learned and decide whether to iterate or exit.
What’s Next?
Next time, I’ll cover my flow for Customer Validation which I promise will look very different (from Steve’s) for a web startup.
Update: The workflow described in this post has been refined even further and turned into a book: Running Lean — with step-by-step guides, techniques for finding prospects, and field-tested interview scripts.
You can learn more here: Get Running Lean.
Translations
This article has been translated to Serbo-Croatian language by Jovana Milutinovich.