Whether it’s getting the green light on a new venture or securing an additional round of funding, entrepreneurs and innovators are constantly faced with the challenge of demonstrating progress to their stakeholders.
Up until now, we have typically relied on two measures of progress: how much stuff we’re building and how much money we are making. Both of these are unreliable proxies of progress that can mislead us down the wrong path of building something nobody wants.
Traditional accounting metrics, like revenue, profit, and ROI, aren’t helpful at the early stages because they all track negative or near-zero. Even at later stages, relying solely on aggregate revenue and profitability can mask uncovering the right growth strategies. The common tendency is to want to collect and analyze as much data as possible. But in a world where we can measure almost anything, it’s easy to drown in a sea of non-actionable data.
Read on to learn how to effectively define, measure, and communicate progress with your internal and external stakeholders without drowning in a sea of numbers.
The Starting Gate
The need to demonstrate at least the “potential for progress” often starts right at the business planning phase, where we have to justify our new venture to a VC, CFO, or spouse as a pre-requisite to securing the runway.
Meet Bob and Josh. Both have great ideas for a business. This is the most exciting time of their venture when anything seems possible. Bob spends the next several weeks writing a 60-page business plan. He knows that the trick is starting with the right “exit number” and then working backward. The right “exit number” needs to be big enough to maximize the odds of getting the funding, but just within the realm of believability. There is a running joke in business schools that the best spreadsheets get funded. So he labors endlessly on his forecasts, often made up of hundreds of numbers. Then he hits the pitching circuit to raise funding for his idea.
Josh takes a different route. He believes it would be more effective first to build out his solution and make it easier for others to visualize his vision. Halfway through, he realizes that he underestimated the scope of his solution and decides he needs to secure additional resources to continue. So he, too, drafts up a business plan and hits the pitching circuit. After several additional months of pitching and lots of rejection, both manage to raise just enough capital to move forward.
Off to the Races
Bob and Josh hire a team and spend the next several months tracking progress against the execution of their plan. Since revenue is typically non-existent during this venture period, they settle for measuring progress by ensuring they build their products on schedule and within budget.
Fast forward a year
Both have been very busy and managed to launch their products to market. But while they have some revenue to show, they have missed their projected targets by a lot. Under pressure to demonstrate more promising revenue numbers to their stakeholders, they resort to several short-term accounting tactics and product strategies, such as taking on custom development projects. These only provide a temporary band-aid to the revenue problem and further distract them from building a repeatable and scalable business model.
Since all the money is now spent, they go back to their stakeholders and under the guise of “learning,” attempt to pitch a brand new vision that promises an even bigger exit. All they need is a larger team and ten times more money.
They are both fired.
There is a better way
Mary too, has an idea for a business, but she takes a “lean” approach to starting up. She knows that the top reason products fail is not due to a failure in building out the product but rather due to a failure to build a repeatable and scalable business model. She intends to navigate her entrepreneurial journey by following the 3-step innovation meta-process outlined in Running Lean:
Rather than spending weeks writing a full-fledged business plan or building out her solution, she opts instead for quickly sketching a 1-page business model using a Lean Canvas. She then gets outside the building and begins stress-testing her riskiest assumptions around her customer and problem hypotheses through a series of small and fast experiments. She uses this learning to quickly define a solution. But rather than rush to build out this solution, she instead assembles an “offer” first.
Her offer is made up of three things:
Armed with her offer, she again gets outside the building and tests her offer on potential customers. Through these conversations, she further refines her offer to the point where her customers want to buy her product (Problem/ Solution fit). By now she also has a clear picture of what the first iteration of her product (or minimum viable product) needs to be able to do. She gets to work building out this minimum viable product (MVP) and starts her journey toward Product/ Market Fit.
Compared to Bob and Josh, Mary gets started much faster and on more solid footing, backed by early customer validation. This early customer validation also paves the way for her to easily secure additional resources from her stakeholders to move forward.
But that’s when her problems begin.
Drowning in numbers
While it was fairly easy for Mary to pinpoint her starting risks, things get much murkier after launch. Her company is signing up dozens of users a day, and there is no way her team can talk to everyone as she had done in the earlier days of the company. When they do manage to talk to some customers, the feedback is generally all over the place because their customer segments have broadened well beyond their initial early adopters.
To whom do they listen? Their most vocal customers or the silent majority?
Her answer is to invest heavily in metrics.
“In God we trust. All others bring data.”
- Edwards Deming
Her team starts with a few simple off-the-shelf tools and supplements them with their own homegrown dashboards. Pretty soon, they are tracking thousands of different data points. They get that drowning feeling.
The problem with metrics is that they can tell you what’s going wrong but not why. If you’ve ever used Google Analytics, you’ve experienced this same feeling. Google Analytics alone spits out over a thousand different numbers. While it's interesting data, unless you can quickly turn this data into insights that drive business results, you aren’t making progress.
In today’s world, where we can measure almost anything, it’s increasingly important to know what to measure. Mary needed a solution.
The Curse of Specialization
She organized her team into departments, assigned each one a set of core metrics, and tied these metrics to their performance and compensation structure. Her sales team was tracked on accounts closed, the marketing team on leads generated, and the development team on product quality metrics. While these department-level KPIs were designed to drive focus and optimize for overall organizational throughput, they started having the opposite effect.
For instance, sales quotas were typically met in the last week of the month. But customer cancelations (or churn) started going up while more deals were getting closed. Marketers were spending all their budget to generate hundreds of additional leads, but the overall conversion to paying customers wasn’t going up. And developers were busier than ever, building more features at an incredible pace. But customer retention and satisfaction were going down, not up. While these KPIs seemed well-intentioned in isolation, they weren’t working at a macro level.
When all else fails, one can always fall back on revenue as a measure of progress. Not really. The problem with relying on revenue as a measure of progress is that revenue is generally a longer customer lifecycle event which can mean having to fly blind for a long time. And even when that revenue is realized, unless you can accurately tie it back to specific actions or events from the past, it is easy to confuse correlation for causality. This was rampant across Mary’s teams.
A rising tide lifts all boats, but a falling tide lifts all fingers.
Whenever Mary’s company had a good quarter, everyone pointed to their department-level KPIs and took credit for it. But during a bad quarter, the same teams used the same KPIs to rationalize why it wasn’t their fault.
The company’s initial momentum began to wear down, and growth stagnated. It became increasingly difficult for Mary to justify the return on investment to her stakeholders, and she, too, found herself spinning the numbers in board meetings. Her two go-to measures of progress were either the amount of stuff her team was currently building (build velocity) or the amount of revenue they made that quarter, depending on which was better. Eventually, she, too, was fired.
Is There a Way Out?
The mistake Bob and Josh made is that they spent a disproportionate amount of time focusing on the wrong numbers and promised a fictional plan they couldn’t deliver.
Mary had a much better early start but, despite her best intentions, quickly found herself drowning in the wrong numbers as she scaled up her product and team. Not only were these the wrong numbers because they were unreliable indicators of progress but worse, they prioritized the wrong actions, which led her company off course.
To summarize, the traditional measures of progress fall apart for the following reasons:
Revenue is near zero at the earlier stages, so we settle for build velocity as a measure of progress. But measuring progress in terms of executing an untested plan is no better.
Measuring qualitative learning is fuzzy, so we invest heavily in quantitative metrics. But metrics can only tell you what’s going wrong, not why. And we end up drowning in a sea of non-actionable data.
Even when we are making revenue, unless we can tie cause and effect, we can easily be misled down the wrong path.
As a result, we build two different stories of our business. The story we tell our stakeholders and the story we tell ourselves. They both start the same but diverge significantly over time because each uses a different definition of progress.
Is there a way out of this dichotomy?
The answer lies in modeling the output of your business model as a system. Once you do this, we can use a single metric (aka traction) to serve as a reliable measure of progress throughout the innovation cycle.