I can summarize my last post with a strong call to action:
Focus on Product/Market Fit First.
Here I’ll be laying the groundwork for how I plan to measure and optimize the process of achieving product/market fit. Metrics are, of course, the answer. But I have spent enough cycles getting lost in Google Analytics’ numbers, that it’s not just about numbers, but the right numbers. The right numbers vary over time and have to be actionable.
Dave McClure has some great models for startup metrics which I’ve decided to adopt. He talks about creating a 1-page business model, and identifying 3–5 actionable metrics that you live by (conversion dashboard).
There are 5 basic steps to any conversion process that Dave defines as :
- Acquisition: How do users find you?
- Activation: Do users have a great first experience with your product?
- Retention: Do users come back and use your product?
- Referral: Do users tell others about your product?
- Revenue: How do you make money?
While you should start laying the foundation for measuring all 5 steps from day one, you don’t have to start optimizing all of them. That said, you should have definitely thought through all the steps and be able to state how (you think) you’re going to address each one (Customer Development Hypotheses). It is also critical to have had some customers travel through the entire conversion funnel first (Customer Validation) before starting any optimization. Lastly, for any optimization to be statistically meaningful, you will need to establish a few channels that drive just enough new users entering the top of the funnel (Acquisition). These could be either through free channels (blogs, seo) and/or paid channels (adwords, banners). Just don’t break the bank (not yet anyway).
1-Page Business Model
For CloudFire, I am starting with 3 core metrics: Activation, Retention, and Revenue. Here’s my 1-page business model that summarizes my user types, conversion events, and priorities:
Note: It’s important to frame the metrics as specific actions the user takes which makes them both measurable and actionable.
Activation is really critical to measure as CloudFire is a downloaded product which creates more friction to overcome. Retention is probably the most important metric for validating early traction and product/market fit. As I mentioned last time, Sean Ellis uses a 40% retention test to gauge product/market fit.
Some people advocate deferring Revenue to later but I believe getting people to keep paying for a product is the ultimate product/market validation. Since CloudFire is a premium service with no FREE plan, Retention and Revenue should follow each other quite closely. I believe that unless the end user isn’t the one ultimately paying, you have to charge from day one. You should already have asked the pricing question during Customer Discovery and should already have acquired a few paying customers from Customer Validation. Now it’s time to test if that pricing model will scale.
Referrals, while probably the most critical channel for CloudFire adoption, will take a back seat to Activation and Retention for now. People don’t refer products that don’t work or solve problems. You have to be remark-able first.
Parent Conversion Dashboard
Here’s my conversion dashboard for the Parent segment:
Note: Conversion dashboards let you visualize the entire conversion process but I am only going to *initially* focus on Activation, Retention, and Revenue.
The estimated values on the right are just that — estimated values that help quantify each event relative to the one real transactional revenue event. They will come in handy when I start optimizing a particular metric and want to measure the effectiveness of different approaches.
How I’m measuring Acquisition
This is the easiest metric to measure. What I’m looking for here are unique website visitors that didn’t bounce, spent enough time on the site, and made it to the Pricing page. Most analytics tools can provide this out-of-the-box. I was using Google Analytics but more recently have been using KISSmetrics.
While measuring overall Acquisition is easy, optimizing it is quite another story. Measuring the effectiveness across all campaigns in Google Analytics is a lot of work because the relevant data is spread out across multiple screens. I am not really at this stage yet, but I have come across 2 new services in private beta (KISSmetrics is one of them) that show a lot of promise here. KISSmetrics fundamentally uses people versus page views as the basic unit which makes funnel and conversions visualization a snap.
How I’m measuring Activation
I define Activation as the set of steps: Sign up -> Download -> Create First Gallery. Since the Activation steps start on the product website and end in the downloaded application, I had already starting building my own custom funnel analysis tool before I discovered Mixpanel.
I have only started using KISSmetrics and I think that it too might be able to handle custom events. If it does, it would be interesting to compare the two. I definitely like the 1-page KISSmetrics reports and would prefer to use a single analytics tool, if possible.
How I’m measuring Retention
For now, I’m using a custom database report to find users that have shared at least one thing per month for the last 3 months.
Mixpanel seems to be designed for measuring user retention around a particular action. I simply send an event each time an album or movie is shared and Mixpanel tracks it and automatically generates cohort reports. The data is also available through an api which I’ll probably need for my specific metric.
Another approach to measuring Retention is to tie it to Revenue. Since CloudFire has a 14-day trial, it’s fairly easy to measure retention that way. Beyond the 14-days, CloudFire is priced annually and I am considering A/B testing a monthly pricing plan on Nivi’s suggestion. Another approach might be to offer a money back guarantee. The idea is to give users an easy way out. If they stay, you know it’s because of the product. There is no better measurement of Retention than recurring payment.
I haven’t had much experience running surveys, but it might also be interesting to conduct a survey based report using Sean Ellis’ 40% test.
How I’m measuring Referral
I am not currently measuring this metric but am using Mixpanel to track an event each time a user uses the “Tell a Friend” feature and let Mixpanel track it for me.
How I’m measuring Revenue
I am using a homegrown report to measure users that convert after the 14-day free trial.
Visitor Conversion Dashboard
Here’s my conversion dashboard for the Visitor segment:
At this stage, I’ve defined but not implemented this dashboard. Since Parents come first in the sharing process, I’ve decided to implement that out first.
To reiterate, my sole focus is working towards Product/Market Fit. In Dave McClure’s model, that equates to optimizing for Activation and Retention. I’m going to measure but ignore the rest.
I am also a little weary of falling into the trap of just relying on numbers. Numbers tell you what is happening but not why. Yes, you can build yet more numbers to try to get to the bottom of an issue, but almost every time, it’s just easier to reach out and talk to a user.
Next time, I should have some actual numbers from my Parent Conversion dashboard to share with you and start diagnosing the health of Activation and Retention.
Update: If you liked this content, consider checking out my book: Running Lean which dedicates 50 pages alone on this topic.
You can learn more here: Get Running Lean.