Before you start
You will be able to
- Distinguish a metric from a key performance indicator (KPI), and a count from a rate, and explain what each tells a manager.
- Calculate conversion rate, average order value, revenue, returning-customer share and rating response rate from raw counts, naming the numerator and denominator of each.
- Compare a KPI with its target and direction, stating the gap in units, in percentage points or as a percentage of the target as appropriate.
- Classify measures as leading or lagging indicators and explain how a chain of them connects activity to results.
- Evaluate a set of KPIs for balance and for the risk of gaming that Goodhart’s law describes.
Review first
- Business questions and measurable objectives — A KPI is a measurable objective reviewed every period, so it needs the same operational definitions, time frames and comparisons.
- Problem-solving frameworks — The driver tree revenue = visits × conversion × average order value organises the shop’s KPIs.
- Data quality: is the data fit for the question? — A KPI is only as trustworthy as the counts it is built from.
Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.
The idea
Fictional case
From July 2026 Ana Lim reviews Lantern Coffee’s online shop every month. The shop’s platform can report dozens of numbers: visits, page views, time on site, products viewed, orders, refunds, ratings. Ana has written down four targets: a conversion rate of 3.0%, an average order value of $45, revenue of $15,000 a month and 35% of orders from returning customers. Her question is practical. Which numbers should she look at every month, exactly how should each be calculated, and how will she know whether a month was good?
Metrics and KPIs
A metric is a quantity calculated from data by an exact rule: the number of orders in July, the average rating given in August, the share of visits made on the app. A business can calculate hundreds of metrics, and most of them are useful to someone at some time.
A key performance indicator (KPI) is a metric chosen because it tracks progress towards an objective the business cares about. It comes with three things a plain metric lacks: a target, a direction (whether higher or lower is better) and someone responsible for it. Every KPI is a metric; few metrics are KPIs. Ana’s four targets make four KPIs. Average time on site is a metric she may glance at, but it is not a KPI: it has no target, and there is no agreement on whether more time is good (an engaged customer) or bad (a customer who can’t find the checkout).
The distinction matters because attention is limited. A monthly review of forty numbers is a review of none; a review of four to eight KPIs, each tied to an objective, leads to decisions. The other metrics stay available for diagnosis when a KPI moves.
The anatomy of a KPI
The lesson on business questions showed that a measurable objective needs an operational definition: what exactly is counted. A KPI needs that and more. Write each one down with these parts:
- Name
- Short and unambiguous: “web conversion rate”, not “conversion”.
- Formula
- The exact numerator and denominator, and what each includes. Paid orders only? Test orders removed? Visits by staff excluded?
- Unit and period
- Percent, dollars or a count; per week, per month or per quarter.
- Data source
- Which system produces each count. Two systems rarely agree to the last order.
- Target and direction
- The value to reach and whether higher or lower is better, or a band to stay within.
- Owner and review
- Who is accountable for the number, who reviews it and how often.
Written down, these parts form a KPI dictionary, the measurement equivalent of the data dictionary that B02 builds for a dataset. Without one, two people asked for “July’s conversion” can honestly return different answers from the same shop. July’s 294 orders divided by its 9,800 visits give 3.00%; if those visits came from, say, 8,650 different people, dividing by people gives . Both are calculated correctly. They answer different questions.
Counts and rates
A count says how many: 294 orders in July. A rate divides one count by another so that groups or periods of different sizes can be compared: orders per 100 visits, or the share of orders that come from returning customers. Counts measure scale; rates measure efficiency or quality. A business needs both, because each can mislead on its own.
In July Lantern’s website took 205 orders from 7,400 visits, and its app took 89 orders from 2,400 visits. By count the website is the stronger channel, with more than twice the app’s orders. By rate the order reverses: the app converted of its visits into orders and the website . Both statements are true. The count tells Ana where most of her orders come from; the rate tells her which channel turns a visit into a sale more reliably.
Targets and direction
A target turns a number into a judgement. Lantern’s July average order value (AOV) was $41.20 against a target of $45, a gap of dollars, or 91.6% of the target. Revenue, $12,112.80 against $15,000, reached 80.8% of its target. Conversion, exactly 3.00%, met its target of 3.0%.
For a KPI that is itself a percentage, state the gap in percentage points: a conversion rate of 2.7% against a target of 3.0% is 0.3 of a point short. The next lesson shows why “0.3%” would be the wrong way to say it.
Direction is usually obvious, since more revenue is better, but not always. For a refund rate or a delivery time, lower is better. For stock on hand, both too little and too much are bad, so the target is a band. Say the direction explicitly in the KPI dictionary, because dashboards often colour a change “good” or “bad” automatically, and a rule written for revenue will praise a rising refund rate.
Targets should also be consistent with one another. Ana’s targets for the drivers of revenue don’t guarantee her revenue target. At a conversion rate of 3.0% and an AOV of $45, each visit is worth dollars on average, so $15,000 needs visits. In a month of 10,000 visits, meeting both the conversion and AOV targets still leaves revenue at $13,500. The first widget below lets you test other combinations.
In depth
Five measures for an online shop
Lantern’s case data hold the raw counts for four online-shop KPIs and one supporting metric. The table defines them, and the formula cards later in the lesson give them in symbols. Lantern’s shop platform reports average order value directly, so revenue is calculated as orders × AOV; in most systems AOV is itself calculated as revenue ÷ orders, and the two routes agree.
| Measure | Formula | Direction | Target | July |
|---|---|---|---|---|
| Conversion rate | Orders ÷ visits × 100 | Higher | 3.0% | 3.00% |
| Average order value | Revenue ÷ orders | Higher | $45 | $41.20 |
| Revenue | Orders × AOV | Higher | $15,000 | $12,112.80 |
| Returning-customer share | (Orders − first orders) ÷ orders × 100 | Higher | 35% | 38.1% |
| Rating response rate | Orders rated ÷ orders × 100 | Higher | None set (supporting metric) | 40.1% |
July 2026: 9,800 visits, 294 orders, AOV $41.20, 182 first orders (orders that were a customer’s first ever), 118 orders rated.
The arithmetic is simple; the judgement lies in the definitions. Conversion divides by visits, which Lantern’s analytics tool counts as sessions. Google Analytics, for example, defines a session as a period during which a user interacts with a website or app, ending by default after 30 minutes of inactivity, so one person who browses in the morning and again in the evening makes two sessions.
Dividing by sessions gives orders per visit; dividing by distinct visitors gives a higher figure, because each person can visit several times. Neither is wrong. Switching between them from one month to the next is.
Returning-customer share needs an operational definition too. Lantern’s shop records how many of a month’s orders were a customer’s first ever order: 182 in July. Every other order counts as returning, so July’s share is . This is a share of orders, not of customers: a loyal customer who ordered three times counts three times. A share of customers would need a different count, the number of distinct customers who ordered, and would usually be lower.
Names differ between tools, so check the definitions before comparing figures from two systems. Google Analytics now calls an action that matters to the business, such as a purchase, a “key event”, and keeps the word “conversion” for actions used to measure advertising campaigns. An advertising platform’s conversion rate may divide by ad clicks rather than visits; the M02 course uses that version.
Ratio KPIs and their denominators
Most KPIs are ratios, and most surprises in ratios come from the denominator. Three checks catch nearly all of them.
- The numerator must come from the denominator. Web conversion is web orders ÷ web visits. Dividing all orders by web visits, because app visits are recorded in another system, mixes two populations and overstates the rate.
- The periods must match. A rating can arrive weeks after the order. Lantern counts an order as rated in the month it was placed, so July’s response rate is July’s rated orders ÷ July’s orders, even if some ratings arrived in August.
- Watch both parts move. If a campaign doubles visits but brings people who rarely buy, orders rise and conversion falls. If Lantern stops the campaign, conversion rises while orders fall. The rate alone can’t tell good news from bad.
Small denominators make rates jumpy. One extra order from a new product page with 20 visits moves that page’s conversion rate by percentage points; one extra order in a month of 9,800 visits moves the shop’s rate by about 0.01 of a point. Report a rate with its denominator, and be wary of ranking items whose denominators are small. The interpreting-results lesson returns to this.
When should you report a count and when a rate? Use counts when scale is the point: revenue pays the bills, and planning packing staff needs the number of parcels, not a share. Use rates when you compare units of different sizes, such as two channels, two months with different traffic, or this year against last. The safest habit is to pair them: orders with conversion, revenue with AOV, rated orders with the rating response rate.
Leading and lagging indicators
A lagging indicator measures an outcome after it has happened: revenue for the month, the returning share over a quarter, profit for the year. It is what the business ultimately cares about, but by the time it moves, the causes are in the past. A leading indicator moves earlier and, if the link holds, signals where a lagging indicator is heading: visits, checkouts started, newsletter sign-ups, the ratings that first-time buyers give.
Leading and lagging are positions in a chain, not fixed labels. Conversion leads revenue but lags the checkout experience that produces it; a first order leads the returning share months later. A leading indicator earns its place only if it actually predicts the outcome. That is a claim to test with data, which B05 and B06 teach you to do, not something to assume because the story sounds right.
Goodhart’s law
Writing about monetary policy in 1975, the economist Charles Goodhart observed that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”. The idea, now called Goodhart’s law, is usually quoted in the anthropologist Marilyn Strathern’s 1997 wording: “When a measure becomes a target, it ceases to be a good measure.” People respond to what is rewarded, and a single number can usually be moved in ways that don’t serve the objective behind it.
Every one of Ana’s KPIs can be gamed. Conversion rises if Lantern stops paying for traffic that rarely buys, even as orders fall. AOV rises if free delivery starts only at $60, even if small orders disappear. The rating response rate rises if customers are offered a discount for rating, which also changes who rates and how. None of these is fraud; each is a rational response to a narrow target. The defences are structural:
- Pair measures that pull in opposite directions: conversion with orders, AOV with orders, speed with quality. Gaming one shows up in the other.
- Add guardrails. A guardrail metric is a measure that must not get worse while a KPI is pushed: the average rating while AOV is raised, the refund rate while conversion is raised.
- Fix the definitions and audit them. A KPI whose formula can be changed quietly eventually will be.
- Read the counts behind every rate, and review the set when a KPI stops tracking its objective.
Balanced sets
Robert Kaplan and David Norton argued in 1992 that financial measures alone give an incomplete, backward-looking picture of a business, and that what you measure is what you get. Their balanced scorecard adds measures from three other perspectives to the financial one: how customers see the business, the internal processes it must excel at, and its ability to innovate and learn.
Lantern’s set is a small business’s version. Revenue and AOV are financial; the returning share is a customer measure; conversion measures a process, the shop’s ability to turn a visit into a sale. What the set lacks is a direct measure of quality: nothing in it would fall straight away if the coffee arrived stale. The average rating, reported with the rating response rate so that readers know how many customers it represents, would fill that gap as a guardrail.
Dashboards, in outline
A dashboard is the everyday tool of descriptive analytics: it displays a small set of KPIs on one screen or page, so that the people who own them can see at a glance where things stand. A good one shows, for each KPI, the current value, the target, the comparison that matters (last month, or the same month last year) and a short trend, and it links to the counts and definitions behind each number. It doesn’t rely on colour alone: some readers can’t tell red from green, so mark a value below target with a word or a symbol as well. B03 builds dashboards in a spreadsheet; the point here is what belongs on one, not how to draw it.
Formulas
Conversion rate
- Conversion rate: orders per 100 visits%
- Orders placed in the period, as the KPI dictionary defines an orderorders
- Visits (sessions) in the same period and channelvisits
Assumes. O and V cover the same period and the same channel.
Dividing by visitors (people) instead of visits gives a different, higher rate. State which you use and keep to it.
Average order value
- Average order value$ per order
- Revenue from the orders in the period$
- Orders in the periodorders
A mean: a few large orders, such as gift boxes, pull it up.
Revenue from its drivers
- Revenue$
- Ordersorders
- Visitsvisits
- Conversion rate%
- Average order value$ per order
The driver tree in formula form. CR ÷ 100 × AOV is the average revenue per visit.
Returning-customer share of orders
- Share of the period’s orders placed by returning customers%
- Orders in the periodorders
- Orders that were a customer’s first ever orderorders
Assumes. Every order that is not a first order counts as returning.
A share of orders, not of customers.
Rating response rate
- Rating response rate%
- Orders placed in the period that were ratedorders
- Orders placed in the periodorders
Report it beside the average rating: it says how many customers the average speaks for.
Gap to target
- Actual value of the KPIunits of the KPI
- Target valueunits of the KPI
When the KPI is a percentage, the gap A − T is in percentage points.
Worked examples
Fictional figures
In July 2026 Lantern’s shop had 9,800 visits and 294 orders, an AOV of $41.20, 182 first orders and 118 rated orders. Ana’s targets are a conversion rate of 3.0%, an AOV of $45, revenue of $15,000 and a returning share of 35%. Calculate the four KPIs and the rating response rate, and judge each KPI against its target.
- Conversion rate: . The target is 3.0%, so the gap is 0 percentage points: on target.
- AOV: $41.20, as reported. Gap dollars; share of target .
- Revenue: dollars. Gap dollars, or of target.
- Returning share: , which is 3.1 percentage points above the 35% target.
- Rating response rate: . There is no target; the figure tells Ana that July’s average rating speaks for about two orders in five.
Answer. Conversion 3.00% (on target); AOV $41.20 (91.6% of target); revenue $12,112.80 (80.8% of target); returning share 38.1% (3.1 points above target); rating response rate 40.1% (no target).
What it means. July met its conversion and returning-customer targets but missed revenue by almost a fifth. The driver tree says why: each visit was worth dollars, so $15,000 would have needed about 12,136 visits at July’s conversion and AOV. Revenue was short because of order value and traffic, not because visitors failed to buy. The project asks you to build this scorecard for all six months.
Fictional figures
Westerly Paper Co., a fictional online stationery shop, recorded in March: 48,000 sessions from 30,000 distinct visitors; 3,600 sessions that started the checkout; 1,500 orders placed, of which 60 were cancelled the same day. Calculate conversion four ways: orders placed per session, paid orders per session, paid orders per visitor, and paid orders per checkout started. In April a paid campaign added 12,000 sessions and paid orders rose to 1,560 from 60,000 sessions in all. Which definition should the shop’s dashboard use, and what happened in April?
- Orders placed per session: . Paid orders are , so paid orders per session are .
- Paid orders per visitor: . Higher, because each visitor averaged sessions.
- Paid orders per checkout started (checkout conversion): . This measures the checkout alone, not the whole shop.
- Choose. All four are correct; they answer different questions. For the shop as a whole, paid orders per session is the usual choice: sessions are what the analytics tool counts reliably, and paid orders are what earns money. Checkout conversion belongs on the dashboard as a diagnostic, because it isolates the checkout.
- April on the chosen definition: , down 0.4 of a percentage point, while paid orders rose by . The campaign’s 12,000 sessions brought about 120 extra orders, a conversion of roughly , if the shop’s other traffic converted as it did in March.
Answer. March conversion: 3.125% (orders placed per session), 3.00% (paid orders per session), 4.80% (per visitor) and 40.0% (per checkout started). Use paid orders per session, with checkout conversion as a diagnostic. In April conversion fell to 2.60% while paid orders rose by 120: the campaign added low-converting traffic.
What it means. A falling rate is not automatically bad news. April’s lower conversion came with more orders, because the denominator grew faster than the numerator. Whether the campaign paid for itself depends on its cost and the margin on 120 orders, a question for M02. A dashboard showing conversion without orders would have reported April as a bad month.
Fictional figures
Ana is considering a bonus for her shop team if monthly conversion reaches 3.5%. In a typical month (hypothetical figures) the shop has 11,000 visits: 3,000 come from a paid social-media campaign and convert at 1.5%, and the other 8,000 convert at 3.6%. AOV is $45 for both. What would happen to conversion, orders and revenue if the team stopped the campaign, and should Ana adopt the bonus?
- Orders now: . Conversion ; revenue dollars.
- Without the campaign: 288 orders from 8,000 visits, a conversion of , above the bonus threshold. Revenue dollars.
- The trade: conversion rises by about 0.57 of a percentage point and the bonus is paid, while revenue falls by dollars a month. Nobody sold more coffee; the denominator shrank.
- Apply Goodhart’s law. A bonus on conversion alone rewards the team for removing low-converting traffic whether or not that traffic is profitable. Whether the campaign is worth its cost is a real question, but it should be answered with its cost and margin, not settled by a bonus.
- Redesign the measure. Make revenue (or orders) the headline KPI, keep conversion by channel as a diagnostic, and add the average rating as a guardrail so that the team can’t raise revenue by, say, aggressive discounting that annoys customers.
Answer. Stopping the campaign would lift conversion from about 3.03% to 3.60% and trigger the bonus, while orders fell from 333 to 288 and revenue from $14,985 to $12,960. Ana should not pay on conversion alone; a balanced set led by revenue, with conversion as a diagnostic and the rating as a guardrail, rewards what she wants.
What it means. The example is Goodhart’s law in miniature: the measure was a good indicator of shop performance until it became the target. Balanced sets work because the cheap ways to move one measure show up as damage in another.
Explore
Common mistakes
“Every useful number is a KPI.”
A KPI is a metric with a target, a direction and an owner, tied to an objective. Promote every metric and the review becomes a list nobody acts on. Keep four to eight KPIs and leave the rest for diagnosis.
“Conversion rose, so the shop did better.”
A rate can rise because its numerator rose or because its denominator fell. In the applied example conversion rose from 3.03% to 3.60% while revenue fell by $2,025. Read every rate with its counts.
“Close enough: it’s the same KPI.”
Orders per session and orders per visitor, or orders placed and orders paid, give different numbers. Changing the definition partway through a series creates a jump that is not real. Write the formula down and keep it, or restate the history when you change it.
“Conversion is up 0.6%.”
For a KPI that is a percentage, the difference between two values is in percentage points. From 3.0% to 3.6% is 0.6 percentage points, which is a 20% relative rise. The next lesson treats this in full.
“Pay people on the number and it will improve.”
It will, but perhaps not in the way you want. Goodhart’s law: a single target invites changes that move the number without serving the objective. Use balanced pairs and guardrails.
“It must be a leading indicator; it happens first.”
Coming first in a process is not the same as predicting the outcome. Check, with past data, that movements in the candidate measure really are followed by movements in the result before you steer by it.
Guided practice
Work through these with hints when you need them. Every hint and every worked solution is there to be used.
Loading four questions…
On your own
No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.
Loading six questions…
In practice
Designing KPIs is a recurring decision in every part of a business, and the same mistakes recur. Three fictional cases show the pattern.
A contact centre. An insurer measured its call-centre agents on average handling time, with a target of five minutes a call. Handling time fell and complaints rose: agents were ending difficult calls and asking customers to ring back. The insurer kept the measure but paired it with first-contact resolution, the share of problems solved in a single call, and reported the two side by side. Handling time crept up; repeat calls, and with them total cost, fell.
A regional airline. A carrier rewarded station managers for on-time departures. Some stations met the target by closing aircraft doors early and leaving late-connecting passengers behind. The fix was a balanced set: on-time departures alongside misconnected passengers per thousand and complaints, each with a named owner.
A battery-storage installer. The sales team tracked signed contracts per month, a lagging count. A new head of sales added two leading indicators, site surveys completed and quotes issued, after checking that over the previous two years they had moved a month or two ahead of signed contracts. When surveys fell in a quiet month, she moved staff to finding new leads before contracts fell.
Fictional case
Before her first monthly review, Ana turns her four targets into a KPI dictionary. Writing it forces three decisions she had not made: visits are sessions as the analytics tool counts them, excluding staff and test sessions; an order is a paid order, net of cancellations made the same day; and the returning share is a share of orders, not of customers. She adds the average rating as a guardrail, reported with the rating response rate, and gives each KPI an owner.
| KPI | Formula | Source | Target and direction | Owner |
|---|---|---|---|---|
| Conversion rate | Paid orders ÷ sessions × 100, by channel and in total | Shop platform; analytics tool | 3.0%; higher is better | Shop manager |
| Average order value | Revenue ÷ paid orders | Shop platform | $45; higher is better | Ana |
| Revenue | Paid orders × AOV | Shop platform | $15,000 a month; higher is better | Ana |
| Returning-customer share | (Paid orders − first orders) ÷ paid orders × 100 | Shop platform | 35%; higher is better | Marketing |
| Average rating (guardrail) | Mean stars on orders placed in the month, shown with the response rate | Shop platform | At least 4.3; investigate any month below | Operations |
The dictionary won’t stop the numbers moving, but when conversion falls, as it does in September, everyone will be arguing about the same number. The next two lessons show how to summarise such movements and how to interpret them without being misled.
Knowledge check
Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.
Loading six questions…
Summary
- A metric is any quantity calculated by an exact rule; a KPI is a metric tied to an objective, with a target, a direction and an owner.
- Write each KPI down in a KPI dictionary: name, formula (numerator and denominator), unit, period, source, target, direction and owner.
- Core online-shop measures: conversion = orders ÷ visits × 100; AOV = revenue ÷ orders; revenue = visits × conversion × AOV; returning share = (orders − first orders) ÷ orders × 100; rating response rate = rated orders ÷ orders × 100.
- Counts measure scale and rates measure efficiency. A rate moves when either part moves, so report the counts beside it and keep the numerator and denominator from the same population and period.
- Judge a KPI against its target as a gap and as a share of target; for percentage KPIs the gap is in percentage points.
- Leading indicators move before the lagging outcomes they predict, but the link must be checked with data.
- Goodhart’s law: a measure that becomes a target can be moved without serving its purpose. Defend with balanced pairs, guardrail metrics and fixed, audited definitions.
Key formulas
- Conversion rate
- Average order value
- Revenue from its drivers
- Returning-customer share of orders
- Rating response rate
- Gap to target
Terms introduced
- Metric
- A quantity calculated from data by an exact rule, such as the number of orders in a month or the average rating.
- Key performance indicator
- A metric chosen to track progress towards a business objective, with an exact formula, a target, a direction (higher or lower is better) and an owner. Abbreviated KPI.
- Conversion rate
- The percentage of visits, or of ad clicks, that end in a chosen action such as an order or a sign-up. Which denominator is used varies between tools and reports, so always state it.
- Average order value
- Revenue divided by the number of orders in a period: the mean value of an order. Abbreviated AOV.
- Lagging indicator
- A measure of an outcome after it has happened, such as revenue for a month or profit for a year.
- Leading indicator
- A measure that moves before an outcome and, if the link holds, signals where that outcome is heading, such as visits or checkouts started before orders and revenue.
- Goodhart’s law
- The observation that a measure tends to stop tracking what it was meant to track once people are rewarded for moving it; often put as “when a measure becomes a target, it ceases to be a good measure”.
- Guardrail metric
- A measure that must not get worse while a KPI is being improved, used to catch gains that come at a hidden cost.
- Balanced scorecard
- A set of measures proposed by Kaplan and Norton (1992) that adds customer, internal-process and innovation-and-learning perspectives to financial measures.
- Dashboard
- A single screen or page showing a small set of KPIs, each with its current value, target, comparison and trend, for the people who act on them.
Sources
- Google. “About Analytics sessions”. Analytics Help. support.google.com/analytics/answer/9191807
- Google. “Conversions vs. key events in Google Analytics”. Analytics Help. support.google.com/analytics/answer/13965727
- Kaplan, R. S. & Norton, D. P. (1992). The Balanced Scorecard: Measures that Drive Performance. Harvard Business Review, January–February 1992. hbr.org/1992/01/the-balanced-scorecard-measures-that-drive-p…
- Manheim, D. & Garrabrant, S. (2018). Categorizing Variants of Goodhart’s Law. arXiv:1803.04585. (Quotes Goodhart, C. A. E. (1975), Problems of Monetary Management: The U.K. Experience, Papers in Monetary Economics, Reserve Bank of Australia.) arxiv.org/abs/1803.04585
- Strathern, M. (1997). “Improving ratings”: audit in the British University system. European Review, 5(3), 305–321.