I gave a talk at the Capital Factory Demo Day event to a group of investors and entrepreneurs last week. The key message from the talk was:
Lean Startup is a rigorous process for iterating from Plan A to a plan that works.
I started by outlining the key behaviors of Lean Startups, but my main objective was to raise the bar on how Lean Startup is practiced today. Through my workshops, I have encountered several well-intentioned startups that want to practice Lean. They understand that Lean Startups are NOT about being cheap but about speed. They get metrics, MVP, and Continuous Deployment but struggle with one of the most fundamental concepts that undermine their efforts — the ability to correctly define and run experiments.
I argue that without proper hypotheses formulation and measurement, all you’re left with are a bunch of tactical techniques that don’t deliver on learning and might actually do more harm than good. Some of this difficulty is understandable as Lean Startups attempt to apply the Scientific Method to a startup that isn’t an exact science. Others believe that “Lean Startup” is simply a framework or collection of techniques that help guide their journey to product/market fit. I disagree. Even though startups are not an exact science, Lean Startup is a rigorous process or, at a minimum, a rigorous meta-process that requires the same level of objectivity you would demand from a scientific inquiry.
Here are some rules I presented for running experiments:
Build a Lean Canvas
Before doing anything else, you need to capture your business model hypotheses. By capture, I mean write down (or type up). I covered my adaptation on Alex Osterwalder’s business model canvas last time, which is my preferred method - Lean Canvas.
Here it is again:
The canvas is your startup’s roadmap which is systematically tested through tightly defined experiments. I like the 1-page business model format because it not only forces you to be concise but is a lot more portable and accessible than a business plan — which means it will be read by more people and probably updated more frequently.
Start with what matters
Where you start on the canvas is based on the stage of your startup. The very first thing that matters is making sure you have a problem worth solving which translates to working on the Customer Segment, Problem, and Solution boxes. So, for instance, before achieving Problem/Solution fit, expending a lot of effort optimizing channels is a form of waste and not the best use of time.
But also tackle the riskiest parts early
That said, it’s just as critical to identify the riskiest parts of the business model and start tackling those early. For instance, one area that does trip many startups is the channel or path to customers. Too many startups use fillers like Adwords or SEO here, only to find out later that these channels aren’t viable for their business. So while a lot of effort shouldn’t be spent optimizing channels, building a significant path to customers is something every startup should be testing from day 1. The good news is that if you follow a Customer Development process, you have to do this anyway to find prospects. Test your channel assumptions at a smaller scale.
Formulate falsifiable hypotheses
The next step then is formulating a set of falsifiable hypotheses, which is the area I see startups struggle with. Randy Komisar/John Mullins describe a great technique in their book: “Getting to Plan B” for going from what they describe as a “Leap of Faith” to a testable hypothesis. What most people write down as business model hypotheses are leaps of faith, and they miss the step of converting them into testable hypotheses. The goal here is clearly defining the conditions under which a hypothesis can be absolutely proved or disproved — QUICKLY. Speed is key. Otherwise, you accumulate just enough evidence to convince yourself that the hypothesis is correct.
Here is an example of what I mean:
The first statement here is an example of a hypothesis that cannot be disproved because the expected outcome of driving early adopters is too vague. How many early adopters do I need to sign-up to prove this hypothesis — 10, 100, 1000? The second statement not only has a specific expected outcome but is also based on a repeatable action which makes it testable.
Build an accessible dashboard
The final step is then attaching a measurable metric to the hypothesis and tracking it on a weekly dashboard that looks something like this:
An important point here is to anchor the hypothesis around a key macro metric versus a vanity metric or, if the key macro event is “too far in the future” (e.g., actual purchase), come up with an acceptable proxy for a macro event. Again, speed is key. So in this example, what’s being tested is that a blog post could drive enough interest in a new product that leads to customer interviews for the customer discovery process.
Make the results auditable
Testing hypotheses can be scary for founders, especially when they are proven wrong. I’ve found the best way to keep the results objective and minimize some of the drama is to make the results auditable by anyone on the team. I not only make the dashboards available but all the data behind them which includes the people (customers) behind the numbers. That way, if someone has a question on a particular result, they can dig in deeper or even arrange to pick up the phone and call the customer directly.
Run board meetings in a lessons-learned format
But the best way to measure progress (learning) in a Lean Startup is by frequently reporting on the results of experiments using a Lessons Learned format like this one.
The left-hand side summarizes the key hypotheses and insights from the last iteration and highlights the next set of experiments that must be run. Progress is tracked using the graph on the right, which shows the key macro metric that drives this startup. In this case, this happens to be revenue.
Lean Startup is a rigorous process for iterating from Plan A to a plan that works.
Communicating progress this way lets the startup stay grounded in learning while iterating towards a plan that works. A Lean Startup is fact-based, not faith-based. But without this level of rigor and objectivity, there is a danger of creating a data-based pattern-seeking organization that might be just as bad or even worse at furthering “rationalized dogma.” What do you think?
Side note: The screenshots in this post were all taken from a home-grown tool I use for tracking my experiments which is now available here: http://www.leancanvas.com.
How to Identify a lean startup from Ash Maurya
View more presentations from ashmaurya.