Interpreting results: charts, comparisons and causes · B01

Before you start

You will be able to

Read a chart critically, checking its axes, where its vertical axis starts, its scale and its encoding, and calculate how much a truncated axis exaggerates a change.
Compare like with like, adjusting for period length, base effects and seasonality.
Distinguish correlation from causation, and identify confounding variables and the evidence that would support a causal claim.
Explain Simpson’s paradox and regression to the mean, and recognise them in business data.
Diagnose a change in a KPI by decomposing it into drivers and segments and comparing it with an unaffected group.
Write a finding that states the change, the comparison, the evidence and the remaining uncertainty.

Review first

Summarising data: changes, averages and spread — Percentage changes, percentage points and weighted averages are the raw material of every interpretation.
Problem-solving frameworks — The driver tree and hypothesis-driven work structure a diagnosis.
Collecting data: sources, samples and bias — Experiments, samples and bias decide how far a result can be trusted.

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

September 2026 worried everyone at Lantern. Orders fell from 316 in August to 251, although visits rose from 10,200 to 11,400. Three explanations reach Ana within a day.

Fictional case

A colleague says the new website checkout, launched on 1 September, caused the fall. A friend who runs a café says September is always quiet after the summer. And a slide arrives with a bar chart on which September’s bar is barely a seventh the height of August’s.

Each is a claim about what the numbers mean. This lesson is about testing such claims before acting on them.

Reading a chart

A chart is an argument made with shapes, so read it as you would read an argument. Before looking at the shapes, check:

Title and source: what is measured, for which period, from which data.
Axes and units: what each axis shows, in what units, and whether time is evenly spaced.
Axis start: where the vertical axis starts. Is it zero, and if not, does the chart say so?
Scale: linear or logarithmic, and whether two different scales share the picture (a dual axis).
Encoding: whether values are shown by length (bars), position (lines and dots) or area (bubbles, icons).
Selection: which periods, groups or comparisons the author chose to show, and which they left out.

The same four months of Lantern’s orders drawn twice. On the left the axis starts at 240, so September’s bar looks a seventh the height of August’s; on the right it starts at zero and shows September at about four-fifths of August.

Bars encode values by their length, so a bar chart must start at zero: otherwise the lengths no longer compare the values. On the left chart August’s bar is drawn $316 - 240 = 76$ units long and September’s $251 - 240 = 11$ , so September looks $(76 - 11) \div 76 \approx 85.5\%$ lower. The real fall was $(316 - 251) \div 316 \approx 20.6\%$ . The chart exaggerates the change about four times.

Line charts are different. They encode values by position and change by slope, so a vertical axis that doesn’t start at zero can be honest, and is often necessary: a conversion rate moving between 2% and 4% is invisible on an axis running to 100%. The test is whether the axis is labelled clearly and whether the slope fairly reflects the size of the change. Dual axes deserve suspicion, because the author chooses both scales and so decides where the lines cross. Areas are hardest of all: an icon twice as tall and twice as wide covers four times the area.

Compare like with like

Any statement of change compares a result with a baseline, and the comparison must be fair. The definitions must match, as the KPIs lesson stressed. So must the periods: August has 31 days and September 30. Per day, orders fell from $316 \div 31 \approx 10.2$ to $251 \div 30 \approx 8.4$ , a fall of about 17.9%. The shorter month explains only about 8 of the 65 missing orders; the fall is real.

Seasons matter too. Many businesses have a regular yearly pattern, or seasonality: Lantern sells gift boxes in November and December, cafés sell more cold drinks in summer. Comparing a month with the month before mixes the change you care about with the seasonal pattern. The usual remedy is to compare a month with the same month a year earlier. Lantern began trading on 1 March 2026, so it has no earlier September: the café owner’s claim can’t be checked with Lantern’s own data, and needs other evidence, such as industry figures or the experience of similar shops.

Lantern’s gift-box orders show the other trap, the base effect from the summarising-data lesson: they rose from 9 in October to 88 in November, an increase of almost 880%, because the starting point was tiny and the season had arrived. The Explore section charts them month by month.

Correlation is not causation

Two quantities are correlated when they tend to move together: when one is high, the other tends to be high (or low). Causation means that changing one would change the other. Correlation is evidence worth following up, but it is not proof of causation, for three reasons.

A third factor drives both. A confounding variable affects both quantities and makes them move together. From October to December Lantern’s visits and its average rating both rose. Christmas is a likely confounder: it brings more visitors and more gift-box buyers, whose orders may be rated more warmly.
The causation runs the other way. Shops that advertise more may sell more, or shops that sell more may have more to spend on advertising.
Chance. With only six monthly values, two unrelated series can easily appear related, especially if both happen to trend upwards. Short series and many comparisons produce coincidences.

The checkout claim is a before-and-after comparison: something changed on 1 September, and orders fell in September. Timing is evidence, but weak on its own, because other things also changed in September. Stronger evidence comes from three places.

The first is a comparison group that did not get the change: Lantern’s app buyers never saw the new website checkout, so if app buyers carried on as before while website buyers stopped converting, the checkout is implicated. The second is a reversal: a fix for a payment-page bug on mobile browsers was released on 5 October, so if website conversion recovered after that date, the case strengthens. The third is a mechanism, such as error logs showing failed payments. The project asks you to look at the numbers; this lesson gives you the method.

In depth

What would establish a cause?

The strongest evidence for causation comes from a randomised experiment, such as the A/B test described in the data-collection lesson: if visitors are assigned at random to the old checkout (the control group) or the new one, the two groups differ, on average, only in the checkout, so a difference in their conversion can be put down to it, give or take chance. When an experiment wasn’t run, analysts build a case from several kinds of evidence, none conclusive alone:

Timing: the effect follows the supposed cause, and not before.
A comparison group: a similar group that did not get the change, and that shows what would probably have happened without it.
Reversal: undoing the change undoes the effect.
Dose and response: more of the cause gives more of the effect (more affected visitors, bigger fall).
Mechanism: a plausible route from cause to effect, ideally observed directly.
Alternatives ruled out: other explanations checked and found wanting.

The comparison-group idea has a name and a simple formula. The difference-in-differences estimate of an effect is the change in the affected group minus the change in the unaffected group over the same period: $(T_{\text{after}} - T_{\text{before}}) - (C_{\text{after}} - C_{\text{before}})$ . Subtracting the comparison group’s change removes whatever affected both groups, such as the season. It assumes that, without the change, the two groups would have moved in parallel; that assumption can be checked, roughly, by looking at how they moved in earlier months.

Simpson’s paradox

Overall figures mix groups, and the mix itself can move the total. Simpson’s paradox is the name for a relationship that holds within every group but disappears or reverses when the groups are combined. The classic case, analysed by Bickel, Hammel and O’Connell in 1975, concerns graduate admissions at the University of California, Berkeley: men were admitted at a higher rate than women overall, yet when the 85 departments were examined one by one, only a few showed a significant difference, and about as many favoured women as favoured men. Women had applied more often to the departments that admitted a smaller share of all applicants.

In business the commonest version involves channels. The challenging example below follows a fictional shop whose website and app conversion rates both rose while its overall conversion fell, because a campaign shifted visits towards the website, the channel with the lower rate. The overall rate is a weighted average of the channel rates, weighted by each channel’s share of visits, so it can change because the rates changed, because the shares changed, or both. Separating the two is called a mix analysis: recompute the new period’s overall rate using the old period’s shares, and compare.

Lantern’s app converts better than its website, so Lantern’s overall conversion depends on its mix of web and app visits as well as on each channel’s rate. Before blaming or crediting anything for a change in the overall rate, check each channel’s rate and each channel’s share. The project asks whether September’s mix changed enough to matter.

Regression to the mean

Measurements that are unusually high or low partly because of chance tend to be followed by measurements closer to the average. This is regression to the mean. Barnett, van der Pols and Dobson describe it as a pattern in which unusually large or small measurements tend to be followed by ones nearer the mean, and warn that it is most likely to mislead when units are chosen because they were extreme.

Suppose a fictional chain of twenty cafés sends the managers of its four lowest-rated cafés in March on a training course, and in April those four improve from an average of 3.6 stars to 3.9. The course may have worked. But the four highest-rated cafés, which had no training, fell from 4.7 to 4.5, and the chain as a whole stayed at 4.2. Part of every extreme month is luck, and luck doesn’t repeat reliably, so the worst tend to improve and the best tend to slip with no intervention at all. Only a comparison, for example training two of the four low-rated cafés and not the other two, can separate the course’s effect from regression to the mean.

Lantern’s September was its worst month for orders, and October was better. Some of that recovery may be regression to the mean, some may be the 5 October fix. A comparison group, the app, helps to tell them apart.

Small samples and noise

Every rate built on few cases is noisy, because sampling error shrinks only as the number of cases grows. A product page with 3 orders from 40 visits converts at 7.5%; another with 2 from 40 converts at 5.0%. The 2.5-point gap is one order and means almost nothing.

Survey researchers make the same point formally: Pew Research Center notes that estimates for subgroups have larger margins of error because they rest on fewer cases, and that a lead of one figure over another is less certain than either figure alone. Lantern’s October survey drew 79 responses; any conclusion about a subgroup, such as app buyers, rests on fewer still. B04 shows how to put a number on this uncertainty with confidence intervals; for now, report the counts and be cautious with small ones.

Diagnosing a change in a KPI

When a KPI moves unexpectedly, the work is diagnostic analytics: explaining why something happened. Work through the questions in order; each one can end the investigation.

Is it real? Check the data before the business: a tracking change, a definition change, missing days or duplicate records can all produce a “fall”. The data-quality lesson lists what to check.
Is it unusual? Compare it with normal month-to-month variation and, where possible, with the same period last year.
Which driver moved? Use the driver tree: revenue = visits × conversion × AOV. A fall in orders with rising visits points at conversion.
Where is it? Segment: by channel, device, product, new against returning customers. A change concentrated in one segment points at a cause that affects only that segment.
When did it start? Daily data around the suspected date. A step on the day of a launch is far more telling than a gradual drift.
Why? Form hypotheses and test them with comparison groups, reversals and mechanisms, as above.
How sure are you? Weigh the evidence, name the alternatives you couldn’t rule out, and say what would settle the question.

Writing a finding with its uncertainty

A finding is a short statement that someone else will act on. It should say what changed and compared with what, where, the likely cause with its evidence, what is still uncertain, and what to do or measure next. Match the strength of the words to the strength of the evidence:

Matching the words to the evidence
Evidence	Wording
A randomised experiment with an adequate sample	“The new checkout reduced conversion by …”
A clear comparison group, a reversal and a mechanism	“The evidence strongly suggests the checkout caused …”
Timing and one segment only	“The fall is consistent with a checkout problem, but …”
A correlation in a few data points	“We can’t tell yet whether …; to find out, we would …”

Two habits help. Give numbers with their denominators and comparisons (“orders fell 20.6%, from 316 to 251, while visits rose 11.8%”), and state the alternatives you checked and those you couldn’t. A reader who knows how sure you are can decide how much to bet on your conclusion.

Formulas

Rate per day

\text{Per day} = \dfrac{X}{d}

$X$: A count for the period, such as orders in a monthunits of X
$d$: Days in the perioddays

Removes differences in period length before comparing months of 28, 30 and 31 days.

Overall rate as a mix of group rates

r = \sum_{i} s_i\, r_i, \qquad s_i = \dfrac{V_i}{\sum_j V_j}

$r$: Overall rate, such as overall conversion%
$r_i$: Rate in group i, such as a channel’s conversion%
$s_i$: Group i’s share of the denominator; the shares add to 1
$V_i$: Group i’s denominator, such as its visitsvisits

The overall rate moves if the group rates change, if the shares change, or both.

Mix-adjusted rate

r_B^{\,\text{at A’s mix}} = \sum_i s_{A,i}\, r_{B,i}

$r_{B,i}$: Rate in group i in period B%
$s_{A,i}$: Group i’s share of the denominator in period A

Compared with period A’s overall rate, it shows the change due to the group rates alone; the rest of the change is due to the mix.

Difference in differences

\text{Effect} \approx (T_{\text{after}} - T_{\text{before}}) - (C_{\text{after}} - C_{\text{before}})

$T_{\text{before}},\ T_{\text{after}}$: The measure in the group that got the change, before and after itunits of the measure
$C_{\text{before}},\ C_{\text{after}}$: The same measure in a comparison group that did notunits of the measure

Assumes. Without the change, the two groups would have moved in parallel.

For rates, every term and the effect are in percentage points.

Worked examples

A first example: How much did September’s orders really fall?

Fictional figures

The slide Ana received shows Lantern’s orders for July to October (294, 316, 251 and 319) as bars on an axis that starts at 240. How large does the fall from August to September look, how large was it, and how large was it per day?

Read where the axis starts: 240. Bars are drawn from there, so August’s bar is $316 - 240 = 76$ long and September’s $251 - 240 = 11$ .
Apparent fall: $(76 - 11) \div 76 \approx 85.5\%$ . September’s bar is about a seventh of August’s.
Actual fall: $(316 - 251) \div 316 \approx 20.6\%$ . The chart exaggerates the change by a factor of about $85.5 \div 20.6 \approx 4.2$ .
Like with like: per day, $316 \div 31 \approx 10.19$ orders in August and $251 \div 30 \approx 8.37$ in September, a fall of about 17.9%.

Answer. The chart makes the fall look like 85.5%; it was 20.6% (17.9% per day). Redraw the bars from zero, or use a line chart with a clearly labelled axis.

What it means. A fall of a fifth is serious and worth diagnosing; a fall of six-sevenths would be a crisis. The truncated axis turned the first into the second. Check where the axis starts before reacting to the shape.

A harder example: Both channels improved, yet conversion fell

Fictional figures

Northwind Outfitters, a fictional online outdoor-gear shop, ran a social-media campaign that sent visitors to its website. Before the campaign (period A) the website had 6,000 visits and 150 orders and the app 4,000 visits and 160 orders. During it (period B) the website had 10,000 visits and 260 orders and the app 3,000 visits and 126 orders. The marketing director says the campaign hurt conversion. Did it?

Northwind Outfitters by channel (fictional)
Channel	Visits A	Orders A	Visits B	Orders B
Website	6,000	150	10,000	260
App	4,000	160	3,000	126
Total	10,000	310	13,000	386

Channel rates: the website rose from $150 \div 6{,}000 = 2.50\%$ to $260 \div 10{,}000 = 2.60\%$ ; the app from $160 \div 4{,}000 = 4.00\%$ to $126 \div 3{,}000 = 4.20\%$ . Both improved.
Overall: $310 \div 10{,}000 = 3.10\%$ in A and $386 \div 13{,}000 \approx 2.97\%$ in B, a fall of about 0.13 percentage points. This is Simpson’s paradox.
The mix: the website’s share of visits rose from $6{,}000 \div 10{,}000 = 60\%$ to $10{,}000 \div 13{,}000 \approx 77\%$ , and the website is the lower-converting channel.
Mix-adjusted rate: period B’s channel rates at period A’s mix give $0.60 \times 2.60\% + 0.40 \times 4.20\% = 3.24\%$ . The rates alone would have raised overall conversion by $3.24 - 3.10 = 0.14$ points; the shift in mix lowered it by $2.97 - 3.24 \approx -0.27$ points. Together: about −0.13.
Counts: orders rose from 310 to 386, by 76. Whether the campaign was worth its cost depends on that cost, but it did not make either channel convert worse.

Answer. No. Both channels converted better during the campaign (website 2.50% → 2.60%, app 4.00% → 4.20%). Overall conversion fell from 3.10% to 2.97% only because the campaign shifted the mix towards the lower-converting website: a rate effect of +0.14 points and a mix effect of −0.27 points. Orders rose by 76.

What it means. An overall rate can fall while every part of the business improves. Before acting on a change in a total, split it by the groups that make it up and separate the change in rates from the change in mix.

Applied: a business decision: Diagnosing a dip at Alderbank Outdoor

Fictional figures

Alderbank Outdoor, a fictional online shop, redesigned its product pages for desktop computers on 1 May; mobile pages were unchanged. Orders fell in May. On 3 June the desktop redesign was rolled back. Its AOV was $80 in every month. Diagnose the dip and write the finding.

Alderbank Outdoor by device (fictional)
Month	Desktop visits	Desktop orders	Mobile visits	Mobile orders
April	20,000	600	30,000	600
May	21,000	441	31,000	620
June	20,500	595	30,500	610

Is it real? The analytics team confirms that tracking and the definition of an order did not change on 1 May, and no days are missing.
Which driver moved? Visits rose from 50,000 to 52,000 (+4.0%) while orders fell from 1,200 to 1,061 (−11.6%), and AOV was flat. So conversion fell: $1{,}200 \div 50{,}000 = 2.40\%$ to $1{,}061 \div 52{,}000 \approx 2.04\%$ .
Where? Desktop conversion fell from $600 \div 20{,}000 = 3.00\%$ to $441 \div 21{,}000 = 2.10\%$ ; mobile stayed at $600 \div 30{,}000 = 2.00\%$ and $620 \div 31{,}000 = 2.00\%$ . The fall is entirely on desktop, the only device that got the redesign.
Comparison group: difference in differences $= (2.10 - 3.00) - (2.00 - 2.00) = -0.90$ percentage points on desktop. Anything that affected both devices, such as the season, cancels out.
Reversal: after the rollback, desktop conversion in June was $595 \div 20{,}500 \approx 2.90\%$ , close to April’s 3.00%.
Size: at April’s 3.00%, May’s 21,000 desktop visits would have produced 630 orders, not 441: about 189 orders, or $189 \times 80 = 15{,}120$ dollars of revenue.
Alternatives: a change in desktop traffic sources in May (a campaign aimed at desktop users, say) could also lower desktop conversion. The team checks and finds desktop traffic sources unchanged.

Answer. Finding: “Orders fell 11.6% in May (1,200 to 1,061) while visits rose 4.0%. The fall was confined to desktop, where conversion dropped from 3.00% to 2.10%; mobile, which kept the old pages, held at 2.00%. Desktop conversion recovered to 2.90% after the redesign was rolled back on 3 June. The evidence strongly suggests the desktop redesign cost about 189 orders ($15,120) in May. It was not a randomised test, so we can’t exclude another desktop-only change, but traffic sources were unchanged. Before relaunching, test the new design against the old with a randomised A/B test.”

What it means. Each step narrowed the question: from orders to conversion, from all visitors to desktop, from a coincidence in time to a pattern with a comparison group and a reversal. The finding says how strong the evidence is and how to make it stronger. The project asks you to do the same for Lantern’s September, using the app as the comparison group and the 5 October fix as the reversal.

Explore

Common mistakes

“The chart shows orders collapsed.”
Check where the axis starts. On an axis from 240, a 20.6% fall looks like 85.5%. Bars must start at zero; lines may not, but the axis must be labelled.
“It happened right after the launch, so the launch caused it.”
Timing alone can’t separate a cause from a coincidence. Look for a comparison group that didn’t get the change, a reversal and a mechanism, as in the Alderbank example.
“Every channel improved, so the total must have.”
Not if the mix shifted towards a weaker channel: Northwind’s two channels both improved while its overall conversion fell. Split totals by group and separate rate from mix.
“Our worst performers improved after the programme, so it works.”
Units picked because they were extreme tend to move back towards the average anyway: regression to the mean. Compare with similar units that didn’t get the programme.
“These two rose together, so one drives the other.”
A confounder may drive both, the causation may run the other way, or it may be chance in a short series. Correlation is a reason to investigate, not a conclusion.
“The data prove it.”
Data from an uncontrolled comparison rarely prove anything. Say what the evidence shows, how strong it is, what you couldn’t rule out and what would settle it.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading four questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading six questions…

In practice

Interpreting results is the step where analysis turns into decisions, so it is where professional standards matter most.

Official statistics agencies deal with seasonality as a matter of routine. The US Census Bureau explains that it seasonally adjusts its quarterly services figures to remove effects that recur in the same part of the calendar each year, such as tax season or travel season, because those swings can hide the underlying direction of recent change; adjusted figures make neighbouring quarters easier to compare. It also cautions that seasonal factors rest on present and past experience, so future patterns may differ. A business with years of history can do the same in a simpler way, by comparing each month with the same month a year earlier.

In medicine, regression to the mean is a standard caution. Barnett and colleagues point out that patients are often selected for treatment because a measurement was unusually high, and some of the later improvement would have happened anyway; well-designed studies include a comparison group for exactly that reason. A business that sends its worst-performing stores, salespeople or campaigns for “turnaround” faces the same trap.

In an online business the analyst’s diagnosis often decides whether a launch is rolled back. A fictional travel site’s analyst found that bookings fell 8% after a redesign; segmenting showed the fall only among returning customers on one browser, and the error logs showed a broken saved-card feature. Rolling back the whole redesign would have thrown away improvements elsewhere; fixing one feature was enough.

Fictional case

For Lantern, the method is now clear. Ana’s September question goes through the diagnosis in order: confirm the data, find the driver (visits rose, so conversion fell), split by channel, use app buyers as the comparison group for the new website checkout, check whether website conversion recovered after the 5 October fix, look at the mix, and write the finding with its uncertainty. The survey she sent on 1 October can add customers’ own views, but its small sample and its question about a checkout that app buyers never saw limit what it can show. The project asks you to do the calculations and write the finding.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.

Loading six questions…

Summary

Read a chart’s title, axes, axis start, scale and encoding before its shape. Bars must start at zero; a truncated axis exaggerated September’s 20.6% fall into an apparent 85.5%.
Compare like with like: the same definitions, the same length of period (per day if months differ) and, for seasonal businesses, the same period a year earlier.
Correlation is not causation: a confounder, reverse causation or chance can make two series move together.
Evidence for a cause: a randomised experiment, or failing that timing, a comparison group, a reversal, a mechanism and alternatives ruled out. Difference in differences = change in the affected group − change in the comparison group.
Simpson’s paradox: every group can improve while the total worsens, because the mix shifted. Separate rate effects from mix effects.
Regression to the mean: extreme results tend to be followed by more ordinary ones, so improvements after selecting the worst need a comparison group.
Small samples are noisy; report counts and treat small differences with caution.
Diagnose in order (is it real, unusual, which driver, where, when, why, how sure) and write findings whose wording matches the strength of the evidence.

Terms introduced

Seasonality: A regular pattern that repeats at the same time each year (or week, or day), such as gift buying in December.
Correlated: Describes two quantities that tend to move together, so that high values of one tend to go with high (or low) values of the other. Correlation alone does not show that one causes the other.
Causation: A relationship in which changing one quantity would change another.
Confounding variable: A third variable that affects both of two quantities being compared, making them move together, or apart, without either causing the other. Also called a confounder.
Comparison group: A group that did not get a change, used to estimate what would probably have happened to the affected group without it.
Difference-in-differences: An estimate of an effect: the change in the affected group minus the change in a comparison group over the same period, assuming the two would otherwise have moved in parallel.
Simpson’s paradox: A relationship that holds within each of several groups but disappears or reverses when the groups are combined, usually because the groups’ sizes or mix differ.
Regression to the mean: The tendency of unusually high or low measurements, which are partly due to chance, to be followed by measurements closer to the average.

Sources

Illowsky, B. & Dean, S. (2023). Introductory Statistics 2e, section 12.3, “The Regression Equation” (correlation and causation). OpenStax. openstax.org/books/introductory-statistics-2e/pages/12-3-the…
Weinberger, N. Simpson’s paradox. The Stanford Encyclopedia of Philosophy, first published 24 March 2021, substantive revision 6 June 2026. plato.stanford.edu/entries/paradox-simpson
Bickel, P. J., Hammel, E. A. & O’Connell, J. W. (1975). Sex bias in graduate admissions: data from Berkeley. Science, 187(4175), 398–404. doi.org/10.1126/science.187.4175.398
Barnett, A. G., van der Pols, J. C. & Dobson, A. J. (2005). Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220. academic.oup.com/ije/article/34/1/215/638499
U.S. Census Bureau. “Quarterly Services: Seasonal Adjustment FAQs”. census.gov/services/qss/seasonally_adjusted_faqs.html
Mercer, A. (2016). 5 key things to know about the margin of error in election polls. Pew Research Center, 8 September 2016. pewresearch.org/short-reads/2016/09/08/understanding-the-mar…

Before you start

You will be able to

Review first

The idea

Reading a chart

Compare like with like

Correlation is not causation

In depth

What would establish a cause?

Simpson’s paradox

Regression to the mean

Small samples and noise

Diagnosing a change in a KPI

Writing a finding with its uncertainty

Formulas

Rate per day

Overall rate as a mix of group rates

Mix-adjusted rate

Difference in differences

Worked examples

Explore

Common mistakes

“The chart shows orders collapsed.”

“It happened right after the launch, so the launch caused it.”

“Every channel improved, so the total must have.”

“Our worst performers improved after the programme, so it works.”

“These two rose together, so one drives the other.”

“The data prove it.”

Guided practice

On your own

In practice

Knowledge check

Summary

Key formulas

Terms introduced

Sources