Before you start
You will be able to
- Identify, for a given correlation, the explanations other than cause: chance, confounding, reverse causation, common trends, selection and pooled groups.
- Detect and remove a common trend by correlating changes or by working within shorter periods, and explain what the result shows.
- Evaluate the evidence a causal claim needs (experiments, natural experiments, dose–response, timing, consistency and mechanism), using Bradford Hill’s considerations as considerations rather than a checklist.
- Decide which correlations a manager can act on for prediction and which need causal evidence before an intervention.
- Write a finding from correlational data that states the association, its size and its limits without overclaiming.
Review first
- Partial correlation — Confounders, mediators and colliders, and the barista-hours example this lesson returns to.
- Correlation matrices — The many-tests problem, one of the ways chance produces correlations.
- Groups and Simpson’s paradox — Pooled groups that reverse a correlation.
- Comparing two means — The two-sample t test that analyses a randomised trial, and why random assignment allows a causal reading.
Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.
The idea
Fictional case
Ana has three questions from her matrix and her partial correlations, and a new one from her assistant: Harbourside’s loyalty-app members and the day’s high temperature correlate at +0.87 from March to July. “Warm weather gets people to sign up,” the assistant says, “so we should push the app hardest in summer.” In August–December the same two numbers correlate at −0.90. Ana wants to know which of her correlations she can act on.
Why correlation is not causation
To say that x causes y is to say that changing x would change y: if Ana rostered one more barista on a given morning, the wait that morning would be shorter than it would otherwise have been. That “otherwise” can never be observed on the same morning, so causal claims are always inferences. A correlation shows only that x and y moved together in the data. OpenStax’s business statistics text puts it plainly: “Strong correlation does not suggest that X1 causes X2 or X2 causes X1.” Six other explanations can produce a correlation, and a causal claim has to rule them out.
Six explanations other than cause
- Chance. With few rows, sizeable correlations arise from noise; with many tests, some cells are “significant” by luck. The correlation-matrices lesson showed 18 of 28 June cells at Harbourside passing an unadjusted test, and only 10 to 16 surviving adjustment. Adjustment changes what can be claimed, not what is true: some cells that fail it, such as the high temperature and footfall, are links the case planted, real but too weak to establish on 30 days.
- Confounding. A third variable drives both. Busy days bring both more baristas and longer waits (r = +0.53), so the simple correlation said the opposite of the truth until transactions were held fixed (−0.72). Peak barista hours also correlate with sales at 0.75, because the managers roster for the trade they expect: anticipated trade drives both the roster and the till. The case was built so that baristas don’t bring customers in.
- Reverse causation. y drives x, not x drives y. Across a chain’s stores, price cuts go with falling sales, because the stores whose sales slipped are the ones told to cut prices. Read the other way, the correlation would say that cutting prices loses sales.
- Common trends. Two series that both drift over time correlate whatever their connection. Loyalty members only ever rise; temperature rises to July and falls after. Together they correlate at +0.87 in spring and −0.90 in autumn.
- Selection. The rows were chosen in a way connected to both variables. Studying only the cafés that survived, the customers who answered a survey, or the days someone complained creates links that the full population doesn’t have (the collider of the partial-correlation lesson).
- Pooled groups. Two groups with different levels, pooled, can show a correlation that neither has inside it, or the reverse: footfall and average ticket are −0.32 pooled but positive inside each café.
These are not exotic. In a daily log like Ana’s, every one of them is present somewhere, and a correlation that has not been checked against all six is a lead, not a finding. The word for a correlation produced by one of them rather than by cause is a spurious correlation.
Two trends: members and the weather
The assistant’s +0.87 comes entirely from time. Members are a running total, so they rise every day whatever the weather; in spring the temperature also rises, so the two correlate positively; in autumn it falls, so they correlate negatively. Over the whole year the two trends partly cancel (−0.13). A number that flips sign when the calendar moves is telling you about the calendar.
The fix is to ask the question of the right quantity. Ana’s question is whether warm days bring sign-ups, so compare each day’s new members (today’s total minus yesterday’s) with the day’s high. That correlation is +0.35 in spring and +0.39 in autumn: modest and positive in both halves, though not equal. Treat it as an upper figure. Daily sign-ups still drift with the season (holding the day number fixed, the correlation falls to 0.26 in spring and 0.11 in autumn), and the route the case planted runs from warm days to more customers to more sign-ups, about 4% of transactions: holding the day’s transactions fixed leaves only 0.09 in spring and 0.12 in autumn. Warm days go with somewhat more sign-ups at Harbourside because they bring customers. The running-total trend made the link look far stronger than it is in spring and reversed it in autumn.
What would show causation
The strongest evidence comes from changing x on purpose. In a randomised experiment the analyst decides which units get the change by chance, a coin or a random number for each, so that nothing else (busy days, weather, the manager’s judgement) can line up with it. OpenStax’s introductory text describes how random assignment spreads lurking variables evenly across the groups. B04’s A/B tests are randomised experiments. Ana could flip a coin for each weekday, before the roster is drawn up, to decide whether to add a barista hour at the peak, and after eight weeks compare waits on extra-barista days with the rest.
When an experiment is impossible or too costly, other kinds of evidence each rule out some of the six explanations:
- A natural experiment: something outside anyone’s choice changes x for some units and not others. Rain is not chosen by the cafés. On the 30 Harbourside weekdays with a quarter-inch of rain or more, sales averaged $206, against $296 on the 148 dry weekdays: a difference of $90 (95% interval $57 to $123, Welch’s t test). The comparison is fair only if wet days are otherwise like dry ones. Real rain clusters by season, so check: here the wet weekdays fall in every month, and their mean high, 66.6°F, is close to the dry weekdays’ 63.7°F.
- A dose–response relationship: more of x goes with more of y, step by step. Peak waits average about 1.6 minutes at loads under seven transactions per barista hour, 3.1 at nine to ten, and 9.4 at twelve or more. A gradient is harder for chance to produce than a single difference, though a confounder can produce one too.
- Timing: the cause comes first. A day’s weather is fixed before its sales, so sales can’t cause the weather; the roster is set before the day, from the expected trade, which is why the roster can follow expected sales rather than cause them.
- Consistency: the same link in different places, periods and data. The negative partial correlation of barista hours and waits holds in both cafés, in every band of trade and in every month with both cafés together (−0.54 to −0.84).
- Mechanism: a plausible route from x to y. More baristas serve more customers per minute, so queues clear faster; that is how queues work. A correlation with no conceivable mechanism, such as members and temperature, deserves extra suspicion.
What Ana can act on
Not every decision needs causation. To predict, a stable correlation is enough: warm days go with a high iced share (r = 0.94), so stocking ice and cups by the forecast works whether or not anyone understands why. To intervene, changing x to change y, the correlation must reflect cause. Rostering more baristas to shorten waits is an intervention, supported by the partial correlation, the gradient and the mechanism, and best confirmed by a trial. Pushing the loyalty app in summer “because warm weather drives sign-ups” rests on a trend, though daily sign-ups do rise modestly on warm, busy days.
In depth
Confounding and reverse causation in more detail
A confounding variable is a common cause; the partial-correlation lesson held one fixed. Holding a confounder fixed works only if it is measured, measured well and controlled in the right shape, and only for the confounders you thought of. That is why observational studies, however careful, leave room for doubt, and why random assignment, which balances unmeasured confounders too, is the stronger design.
Reverse causation means that y affects x. It is common where a decision responds to the outcome itself: stores whose sales slip are told to cut prices, and a firm whose complaints rise hires more support staff. A close relative is a decision made from a forecast of y. Airlines add flights on routes they expect to fill, councils grit roads when they forecast ice, and Lantern’s managers roster for the trade they expect. There the conditions behind the forecast drive both x and y, which is confounding by anticipated demand; in medicine, where the sickest patients get the most intensive treatment, it is called confounding by indication. In every case the “treatment” correlates with the outcome it was chosen to meet. Timing helps (what was decided before what), and so does asking how x was set.
Trends and other time patterns
Any two series that trend over the same period correlate, as the members show: −0.13 over the year, +0.87 and −0.90 within the halves. Three remedies are common. Correlate changes (today minus yesterday, this week minus last) rather than levels, as with daily sign-ups. Hold time fixed with a partial correlation, as long as the trend is close to a straight line: within spring, members and temperature correlate at 0.20 holding the day number fixed, and within autumn at −0.03. Or compare like periods (each Monday with the previous Monday, this June with last June). Seasonal patterns that are not straight lines, like a year of temperature, defeat a straight-line control for time: over the whole year, holding the day fixed leaves 0.61, which is still mostly season. Time-series methods that model trend and season properly belong to a forecasting course.
Selection
Selection bias arises when which rows are in the data depends on the variables being studied. Studying only successful cafés to find what makes cafés succeed is survivorship bias, one kind; studying only customers who answered a survey, who may be the most and least satisfied, is another. Selecting on a common effect of two variables is selecting on a collider, which creates a link between them. The remedy is to ask, before computing anything, how each row got into the data, and whether that process could depend on x, on y, or on both.
Experiments and their limits
A randomised experiment answers a causal question for the units, period and version of the change that were tried. Ana’s eight-week barista trial would tell her what an extra peak hour did to waits in those weeks, at those cafés; whether the effect holds in the December rush is a further question. Experiments also need enough units to detect the effect (the power ideas of B04 and of the testing-correlation lesson), a measure fixed in advance, and a plan to run the change as assigned. Where randomising is unfair or impractical, a comparison group and difference-in-differences (the change in the treated group minus the change in a comparison group) can stand in, at the cost of an assumption that the two would otherwise have moved in parallel.
Bradford Hill’s considerations
In a 1965 lecture on how to judge whether an environmental exposure causes a disease, the epidemiologist Austin Bradford Hill listed nine aspects of an association to weigh before deciding: strength, consistency, specificity, temporality, biological gradient, plausibility, coherence, experiment and analogy. They are widely quoted as “criteria”, but Hill was explicit that they were not a checklist: in his words they were viewpoints, none of which could on its own prove or disprove a cause, and none of which was a necessary condition.
| Consideration | What it asks | Ana’s evidence |
|---|---|---|
| Strength | Is the association large? | Partial r = −0.72 at a given level of trade. |
| Consistency | Is it seen repeatedly, in different settings? | Both cafés, every band of trade, every month (cafés together). |
| Specificity | Is it tied to one outcome? | Weak here: busy days affect many measures at once. |
| Temporality | Does the cause come first? | The roster is set before the queue forms. |
| Gradient | Does more of the cause go with more effect? | Waits rise step by step with transactions per barista hour. |
| Plausibility | Is there a mechanism? | More baristas serve more customers per minute. |
| Coherence | Does it fit what else is known? | It fits basic queueing: service rate against arrival rate. |
| Experiment | Does changing it change the outcome? | Not yet: the barista trial would supply this. |
| Analogy | Are there similar known effects? | Extra checkout staff shorten supermarket queues. |
Hill also cautioned against letting tests of significance decide the question. A significance test answers the first of the six explanations, chance, and none of the other five.
Writing a finding without overclaiming
The words in a finding should match the evidence. Before a causal claim is earned, describe the association: “goes with”, “is associated with”, “was higher on days when”. Give its size, its interval or p-value, the rows it covers and what was held fixed. Name the explanations you could not rule out. Then, if the evidence supports it, say how strongly it points to cause and what would settle it.
| Evidence | Wording that fits | Wording that overclaims |
|---|---|---|
| A correlation in observational data | “Days with more X tended to have more Y (r = 0.40).” | “X drives Y.” |
| A partial correlation holding a confounder fixed | “At a given level of Z, more X went with less Y.” | “X reduces Y.” |
| A gradient, consistent across settings, with a mechanism | “The evidence points to X reducing Y; a trial would confirm it.” | “We have proved X reduces Y.” |
| A randomised experiment | “Adding X reduced Y by … in the trial (95% interval …).” | “X will always reduce Y.” |
Formulas
Change (first difference) of a series
- The series’ value in period t (members at the end of day t)units of y
- Its value in the period beforeunits of y
- The change in period t (the day’s new members)units of y per period
Assumes. Periods are equally spaced and in order.
Correlating changes instead of levels removes a shared trend; the first period has no change.
Expected chance findings among many tests
- Number of tests run when no relationship is real
- Probability that at least one test is significant by chance
- Significance level of each test
- Average number of “significant” results by chance
Assumes. Every null hypothesis true; the second formula also assumes independent tests.
The correlation-matrices lesson shows how to adjust for this.
Worked examples
Fictional figures
Seaside Cinemas (fictional) correlates weekly ticket sales with 40 other weekly measures from its systems, from the number of trailers shown to the car-park temperature, and reports the two that are significant at 5% as “drivers of attendance”. Could chance alone explain them?
- If none of the 40 measures were related to ticket sales, each test would still be significant with probability 0.05, so on average would pass.
- The chance of at least one passing by luck, if the tests were independent, is .
- Two significant results out of 40 is therefore exactly what pure noise would produce. Even if they are real correlations, “drivers” claims cause, which needs the other five explanations ruled out.
Answer. Yes: chance alone would give about 2 significant results among 40 tests, and at least one with probability about 0.87.
What it means. The cinema should treat the two measures as leads, adjust for the 40 tests (correlation-matrices lesson), check them on new weeks, and ask whether each could be a confounder, a reverse effect or a trend before calling it a driver.
Fictional figures
Harbourside’s loyalty members and the day’s high correlate at +0.87 from March to July and −0.90 from August to December. Does warm weather go with sign-ups, and how strongly?
- Diagnose: members are a running total, correlated with the day number at 0.99 in both halves. The high temperature correlates with the day number at 0.87 in spring and −0.90 in autumn. The members–temperature correlation simply copies the temperature’s trend.
- Hold time fixed: within spring, the partial correlation of members and temperature holding the day number fixed is 0.20; within autumn, −0.03. Most of each half’s correlation disappears.
- Ask about changes: the day’s new members (today’s total minus yesterday’s) against the day’s high correlate at +0.35 in spring and +0.39 in autumn, and +0.37 over the year. Holding the day number fixed within each half lowers them to 0.26 and 0.11: some seasonal drift remains.
- Interpret: the change-based correlation is positive in both halves, unlike the levels’ flip, but 0.37 is an upper figure. The route the case planted runs from warm days to more customers to more sign-ups: holding the day’s transactions fixed leaves only 0.09 in spring and 0.12 in autumn, so warmth adds little beyond the customers it brings.
Answer. The ±0.9 correlations are trends. Warm days go with somewhat more sign-ups, at most r ≈ 0.37 between the day’s new members and its high, and mainly because they bring more customers.
What it means. The assistant’s plan rests on the trend, not on the daily link. A better reading is that sign-ups follow customers, and customers follow warm weather at the waterfront. To grow the app, Ana should ask what share of customers sign up and whether a promotion raises that share, which is something she can test by running it on randomly chosen days.
Fictional figures
The log says that, at a given level of trade, more peak barista hours go with shorter waits (partial r = −0.72). Ana wants to know how many minutes an extra barista hour saves before she changes the roster at both cafés. Design a study that answers that causally, and say what she can claim before it runs.
- State why the log can’t settle it: the managers chose the roster from the trade they expected (confounding by anticipated trade), and anything that made them add staff, such as a planned event, may also affect waits.
- Randomise: for eight weeks, for each weekday at each café, a random draw made before the roster is set decides whether the roster gets one extra barista hour at the 7–11 a.m. peak. The managers can’t pick the days, so expected trade, weather and events are spread evenly across extra and normal days.
- Fix the measure and the analysis in advance: the peak wait as the log records it; compare mean waits on extra-hour and normal days with Welch’s t test (B04), within each café, and report the difference with its 95% interval.
- Plan the size: with about 40 weekdays per café, roughly 20 extra-hour days each, check with B04’s power tools that the trial can detect the smallest saving worth the wage cost.
- Write the interim finding in the language of association: “At the same level of trade, mornings with more barista hours had shorter waits (partial r = −0.72 over 612 café-days). Waits rise steeply above about seven transactions per barista hour. We are running a randomised trial to measure the effect of an extra hour.”
Answer. A randomised trial assigning an extra peak barista hour by chance on weekdays at each café for eight weeks, analysed by comparing mean waits with Welch’s t test; until it reports, Ana can claim an association at a given level of trade, consistent with a causal effect, not the effect itself.
What it means. The log has done its job: it identified a likely lever and gave reasons (strength, consistency, gradient, mechanism) to think it is causal. The trial turns that into a number Ana can put against the cost of the extra hour, which is the decision she actually has to make.
Explore
Common mistakes
“The correlation is significant, so the effect is real.”
A significance test addresses chance, the first of the six explanations. It says nothing about confounding, reverse causation, trends, selection or pooling. A tiny p-value on 612 days can belong to an entirely spurious correlation, such as members and temperature in spring.
“We can’t prove cause without an experiment, so correlations are useless.”
Correlations are enough for prediction (stock ice for hot days) and are how good causal questions are found. Natural experiments, gradients, timing, consistency and mechanism can make a causal reading strong even without a trial. The skill is matching the claim to the evidence, not refusing to claim anything.
“Both series went up this year, so one must be driving the other.”
Any two trending series correlate. Correlate the changes, compare like periods, or hold time fixed; if the link vanishes, it was the trend. Members and temperature flip from +0.87 to −0.90 when the calendar moves.
“More barista hours go with higher sales, so baristas sell coffee.”
The roster is set from the trade the managers expect, so anticipated trade drives both barista hours and sales (r = 0.75): confounding. In the case, baristas don’t bring customers in; the generator makes the roster follow expected trade. Always ask how x was set.
“I’ll write that X drives Y, since everyone knows what I mean.”
Readers act on words. “Drives”, “leads to” and “causes” tell a manager that changing X will change Y. If the evidence is a correlation, write “goes with” or “is associated with”, give the size, and say what would settle the causal question.
Guided practice
Work through these with hints when you need them. Every hint and every worked solution is there to be used.
Loading four questions…
On your own
No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.
Loading five questions…
In practice
Every analyst who reports correlations to managers faces the same pressure: the manager wants to know what to change, and a correlation seems to say. Airlines see the most flights on the routes that carry the most passengers, operations teams see staffing and incidents rise together on busy shifts, product teams see that users of a feature are more loyal. In each case the professional habit is the same: name the alternative explanations, check the ones the data can check, say clearly which remain, and propose the test that would settle the question when the decision is large enough to justify one.
Fictional case
Ana’s year-end summary for her managers reads: “1. Warm days go with a higher share of iced drinks (r = 0.94); we will stock by the forecast. 2. At the same level of trade, mornings with more barista hours had shorter waits (r = −0.72); waits climb steeply above about seven transactions per barista hour. We are testing an extra peak hour on randomly chosen weekdays from February. 3. Members and temperature are two trends; daily sign-ups rise only modestly on warm, busy days (r = 0.37 at most), so we won’t time app promotions by the weather. 4. Harbourside weekdays with a quarter-inch of rain or more took about $90 less than dry ones; plan staffing for wet forecasts.”
Her summary uses the distinction this lesson rests on. The first and fourth items are predictions: she will act on them without needing to know the cause, and the fourth rests on a natural experiment, since nobody chose the rain. The second is a probable cause, stated as an association with the evidence that supports a causal reading, and she has planned the experiment that will measure it. The third is a correlation she has explained away. None of the items uses “drives” where the evidence is a correlation.
The same reasoning scales up. Many large organisations run randomised experiments routinely precisely because correlations in their data are so often confounded by who chooses what; regulators ask for trials before approving medicines for the same reason. When a trial is impossible, the considerations in this lesson are how analysts, economists and epidemiologists argue from observational data, and how readers should judge their arguments.
Knowledge check
Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.
Loading six questions…
Summary
- Causation means that changing x would change y; a correlation only shows that x and y moved together.
- Six explanations other than cause produce correlations: chance (including many tests), confounding, reverse causation, common trends, selection and pooled groups. Check each before reading a correlation causally.
- Trends: correlate changes, compare like periods or hold time fixed. Harbourside’s members and temperature flip from +0.87 to −0.90 with the season, while daily sign-ups and temperature correlate at about +0.35 to +0.39 in the two halves, an upper figure for a link that runs mainly through the customers warm days bring.
- Randomised experiments are the strongest evidence; natural experiments, dose–response gradients, timing, consistency and mechanism each rule out some alternatives. Hill’s nine considerations help weigh the evidence but are not a checklist.
- Prediction needs only a stable correlation; intervention needs cause. Ana can stock by the weather forecast, should trial extra barista hours, and shouldn’t time promotions by temperature.
- Write findings in the language of association, with size, interval, rows and controls, name what wasn’t ruled out, and say what would settle the causal question.
Key formulas
Terms introduced
- Spurious correlation
- A correlation between two variables that arises from something other than one affecting the other, such as a confounder, a shared trend, selection or chance.
- Reverse causation
- A correlation in which the supposed effect actually causes the supposed cause, such as stores whose sales have fallen being the ones that cut their prices.
- Randomised experiment
- A study in which the analyst assigns the change to units by chance, so that nothing else can systematically line up with it and a difference in outcomes can be attributed to the change.
- Natural experiment
- A situation in which something outside the control of the people being studied, such as the weather or a rule change, alters a possible cause for some units and not others, allowing a comparison close to an experiment.
- Dose–response relationship
- A pattern in which larger amounts of a possible cause go with steadily larger effects, considered evidence (though not proof) of a causal link.
- Selection bias
- Systematic error that arises when the rows in the data were chosen by a process related to the variables being studied, creating or distorting associations.
Sources
- Hill, A. B. (1965). “The environment and disease: association or causation?” Proceedings of the Royal Society of Medicine, 58(5), 295–300. pmc.ncbi.nlm.nih.gov/articles/PMC1898525
- Holmes, A., Illowsky, B. & Dean, S. (2023). Introductory Business Statistics 2e, section 13.1, “The Correlation Coefficient r”. OpenStax. openstax.org/books/introductory-business-statistics-2e/pages…
- Illowsky, B. & Dean, S. Introductory Statistics 2e, section 1.4, “Experimental Design and Ethics”. OpenStax. openstax.org/books/introductory-statistics/pages/1-4-experim…
- Cole, S. R., Platt, R. W., Schisterman, E. F., Chu, H., Westreich, D., Richardson, D. & Poole, C. (2010). “Illustrating bias due to conditioning on a collider”. International Journal of Epidemiology, 39(2), 417–420. pmc.ncbi.nlm.nih.gov/articles/PMC2846442