Marketing
Growth and experiments
Whether customers are worth what they cost to win, and whether a test result is real.
Whether growth pays: what a customer is worth against what it costs to win, and whether an experiment’s result is real.
Unit economics
Is each new customer worth what it costs to win? Four numbers that investors in subscription businesses check first: lifetime value against acquisition cost, months to earn that cost back, the magic number, and net revenue retention.
Both are made-up businesses. Change any number to use your own.
- Cost to win a customer
- —
- Lifetime value (gross profit)
- —
- Months to earn the cost back
- —
- Magic number
- —
- Net revenue retention
- —
- Gross revenue retention
- —
Allowing for customers who leave. Where the curve crosses the cost line, the customer has paid for itself.
- Lifetime value here is gross profit, not revenue, and assumes churn stays at today’s rate. A stricter measure than the revenue-based figure often quoted.
- The thresholds are investors’ rules of thumb: lifetime value above three times acquisition cost and the cost earned back within twelve months (Skok), a magic number above about 0.75. Businesses that sell to large companies often run longer paybacks on purpose.
A/B test
Did the new version really convert better, or is the gap the noise you’d expect from chance? And before running a test: how many visitors does it need to find the change you care about?
- Conversion, A
- —
- Conversion, B
- —
- z statistic
- —
- p-value (two-sided)
- —
Planning
- Decide the sample size before you start and read the result once at the end. Checking every day and stopping at the first p below 0.05 makes false wins far more likely.
- The test uses the normal approximation, which needs at least about ten conversions and ten non-conversions in each version.
Sources
- Skok, D. SaaS Metrics 2.0: detailed definitions, For Entrepreneurs (lifetime value above three times acquisition cost; months to recover acquisition cost under twelve).
- Magic number as annualised new recurring revenue over the previous quarter’s sales and marketing spend: Chang, D. (2021), From $0 to $1M: the magic number, Scale Venture Partners; thresholds of 0.5 and 0.75 as summarised by Klipfolio.
- Fleiss, J. L., Levin, B. & Paik, M. C. (2003). Statistical Methods for Rates and Proportions, 3rd ed. Wiley (two-proportion test and sample size). Kohavi, R., Tang, D. & Xu, Y. (2020). Trustworthy Online Controlled Experiments. Cambridge University Press.
- All example data on this page is synthetic, generated for the demo.