Experiments in Pricing

What you charge for your product is simultaneously one of the most complicated and most important things to get right. Not only does your pricing model keep you in business, but it also signals your branding and positioning. Unlike with other elements, it is harder to iterate on pricing. Once set, coming down is usually easier than going up. So when is the right time to tackle pricing? Most people choose to defer the “pricing question” as late as possible because they don’t think they (or the product) are ready. I hear a lot that a minimum viable product is, by definition, (embarrassingly) minimal. How can you possibly charge for it?

A minimal product is not synonymous with a half-baked or buggy product. If you’ve followed a customer development process, your MVP should address the top 3 problems identified by customers as important and should do it well. 80% of your efforts should be going towards ensuring that by improving existing features versus cranking out new ones.

Steve Blank bakes price exploration right into the initial customer interviews. Price, like everything else, is built on a set of hypotheses that needs to be tested early. Steve makes you phrase the pricing question by first asking potential customers if they’d use the service for free. This is to gauge if the product’s value proposition is compelling at all. You then ask if they’d use the service for $X/yr. How do you come up with X? You can roll the dice and adjust along the way or use Neil Davidson’s excellent guide to software pricing to start with a more educated guess. Once your MVP is built, Steve asks you to sell it to your early customers. There is no clearer customer validation than a sale.

Sean Ellis, on the other hand, argues that achieving initial user gratification (product/market fit) is the first thing that matters and suggests keeping price out of the equation so as not to create unnecessary friction:

“I think that it is easier to evolve toward product/market fit without a business model in place (users are free to try everything without worrying about price). As soon as you have enough users saying they would be very disappointed without your product, then it is critical to quickly implement a business model. And it will be much easier to map the business model to user perceived value.”

Both Steve and Sean advocate removing price from the equation but at different points. Steve does it during the customer discovery process but makes you charge for your MVP. Sean does it with the MVP and only makes you charge after product/market fit. I can see the merits of both approaches and wondered which was right for my product: CloudFire — Photo and Video Sharing for Busy Parents.

Why Not Just Use Freemium?

On the surface, Freemium seems like the best of both worlds: Get users to try your service without worrying about the price, then up-sell them into the right premium plan later. However, the mistake many people make is giving away too much under the free plan, which leads to low or no conversions. It’s human nature — we all want to be liked. More importantly, we don’t yet have enough information to know how to price or segment the feature set. I made this mistake with my first product BoxCloud where an early visionary customer called me up and said, “I like your product and want to pay for it, but your pricing doesn’t require me.” After a few further failed feature segmenting iterations, I decided to forgo the free plan and offered premium plans with a trial period. Sales went up, and so did the quality of feedback which I attribute to the difference between customers versus users.

Hiten Shah shared a similar story with me about his experience with Crazy Egg. Even 37signals has greatly deemphasized their free plans to almost being fine print on their pricing pages.

Lincoln Murphy just published a timely white paper on “The Reality of Freemium in SaaS” which covers many important aspects to weigh in when considering Freemium, such as the concept of quid pro quo, where even free users have to be giving something back. In services with high network effects, participation is enough. But most businesses don’t have high enough network effects and wrongly chase users versus customers.

What I particularly liked in this paper is calling out
“Freemium as a marketing tactic, not a business model.”

I strongly feel that, especially for SaaS products, starting with free and figuring out premium later (all too common) is backward. If you know that you will be charging for your product, start by validating if anyone will pay first. There is no better success metric, and it leads to less waste in the long run. Focussing on the premium part of freemium first lets you learn about your unique value proposition — the stuff that will get you paid. You can then come back and more intelligently offer a free plan (if you still want to) with the right success metrics clearly defined. Even if you think you have a one-dimensional pricing plan (number of projects) like I did, you’d be better served testing it with paying users because pricing experiments take a much bigger toll than other types of experiments.

Practice Trumps Theory

So how do I put all this to the test:

The biggest mind shift in following a lean startup process is going from thinking you know something to testing everything you think you know.

Testing the Waters Early

I followed Steve Blank’s advice and built some pricing questions into my initial face-to-face customer interviews. Because CloudFire is a re-segmented product in an existing market, potential customers did rely on competitor pricing for reference points. This had to be balanced against the perceived value of our unique value proposition — saved time from faster and easier sharing of lots of photos and videos. Through these interviews, I determined that, like their sharing needs, my potential customers valued simple hassle-free pricing, and $49/year for unlimited photo and video sharing was a fair price they were willing to pay. That is what I charged them once my MVP was ready.

Testing Web Visitors

I wanted to run the same set of pricing tests with web visitors that I did during my interviews. Short of split-testing a free and paid version of the MVP, which is *technically illegal* and unfair to paying customers, I decided to split-test three different products with three different prices:

  1. $49/yr for unlimited photo and video sharing
  2. $24/yr for unlimited photo sharing
  3. FREE for 500 photos

All plans have a 14-day free trial except for the free plan, which is free forever.

Here are the variations I tested:

Original: Single unlimited plan

This is the simple single plan option I determined during the customer discovery process, which served as the control.

Variation 2: Multiple plans

I segmented the product into two offerings: unlimited photos+video and unlimited photos only. I wanted to test price sensitivity and gauge interest in video sharing. Not many people I interviewed were currently taking lots of videos, but they all wanted to be doing more.

Variation 3: Freemium

This has the two plans from above, along with a limited free plan. Yes, this is a freemium case. I wanted to measure if a limited free plan would disproportionately drive the right type of traffic (busy parents in my case).

Variation 4: No Price During Introductory Period

I added a fourth variation to test Sean Ellis’ advice on removing price till product/market fit, but I tested this differently. I was not comfortable offering the full product at price and for FREE at the same time. So rather than including this page with my A/B tests, I instead tested it with new parents I interviewed.

The Results

First Place

Original: Single plan — Second Place in Conversions and Best Overall Performer
Surprisingly the original page was the best overall performer.

Second Place

Variation 3: Freemium — Most Conversions but Second Place Overall
Not surprisingly, the Freemium variation drove the most conversions but only outperformed the original by 12% and had the lowest retention. Referral stats combined with random polling/emailing revealed a majority of the users that signed up were just curious (and not parents).

Third Place

Variation 2: Multiple plans — Least Conversions and Worst Overall Performer
People reacted least favorably between the two paid plans.


Variation 4: No Price During Introductory Period — Non-starter
The parents I interviewed did not understand the introductory period without explanation and were reluctant to try the service without knowing how much the service would cost. Probing further, they weren’t willing to invest the time in building up web galleries and inviting others, only to find that the service might be priced out of their expectation.

What I learned

It does pay to align pricing with your overall positioning. Our unique value proposition is built around being “hassle-free and simple,” and people seemed to expect that in the pricing model as well. A lot of our existing customers were already paying for their existing sharing service, so the leap from free to paid was not a big one. While Sean suggests removing price before fit for consumer-facing products, he suggests always charging for Enterprise customers to gain their commitment. This is another case where pricing needs to be explicit. Using Cindy Alvarez’s model, our customers appear to be Time-Poor and Cash-Rich. Offering no-hassle free trials were sufficient to remove the commitment risk. Money-back guarantees might be another way to further lower this risk.

The biggest lesson learned, though, is how accurate my initial customer interview findings were compared to all the hypotheses that followed. Pricing is more art than science. Your mileage will vary, but whenever possible, get out of the building, talk to a customer, and consider testing the price sooner rather than later.

What do you think? Why do you think these variations finished the way they did? What other variations would you like to see us try? How else do you think we could increase conversions?

You've successfully subscribed to LEANSTACK Blog
Great! Next, complete checkout to get full access to all premium content.
Error! Could not sign up. invalid link.
Welcome back! You've successfully signed in.
Error! Could not sign in. Please try again.
Success! Your account is fully activated, you now have access to all content.
Error! Stripe checkout failed.
Success! Your billing info is updated.
Error! Billing info update failed.