Sunflower Intelligence
Business Analytics & Data Science

B06 · Intermediate

Regression Analysis

10 lessons

B06 · Module 1, lesson 1 · Lesson 1 of 10

The least-squares line

Turn a scatterplot into a line you can read: the least-squares criterion, the slope b₁ = r·s_y ÷ s_x and the intercept b₀ = ȳ − b₁x̄. Find what an extra transaction is worth at each café, see why Harbourside’s line crosses zero at −$76.58 and why that number means nothing, and meet the two facts that surprise everyone: the line of y on x is not the line of x on y, and extreme days are followed by ordinary ones.

About 85 minutes

Before you start

You will be able to

  • Explain the least-squares criterion and why it measures misses vertically and squares them.
  • Calculate the slope and intercept of the least-squares line from r, the standard deviations and the means, by hand and in Excel.
  • Interpret a slope and an intercept in context with their units, and recognise when the intercept has no sensible meaning.
  • Distinguish the regression of y on x from the regression of x on y, and calculate both slopes from the same summary statistics.
  • Explain regression toward the mean with ẑ_y = r·z_x, and recognise it when cases are chosen because they were extreme.

Review first

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

B05 ended with a correlation of 0.99 between transactions and sales at Harbourside, the waterfront café, over the 306 days from 1 March to 31 December 2027. That says the days lie close to a straight line. It doesn’t say which line: how many dollars of sales go with one more transaction, or what sales to expect on a day with 80 transactions. Ana needs the line itself.

From a scatterplot to a line

A straight line through a scatterplot is written y^=b0+b1x\hat{y} = b_0 + b_1 x. Here xx is the explanatory variable (transactions), y^\hat{y} (“y-hat”) is the value the line gives for it, the fitted value, b1b_1 is the slope, the change in y^\hat{y} for each extra unit of x, and b0b_0 is the intercept, the value of y^\hat{y} where x = 0. The actual value yy (the day’s sales, the response variable) is rarely on the line, and the gap e=y−y^e = y - \hat{y} is the day’s residual: positive when the day sold more than the line says, negative when it sold less.

Letters matter here. The b’s are calculated from Ana’s 306 days; module 2 treats them as estimates of a line for all the days the café might trade, written with Greek letters β₀ and β₁. This module is about the b’s: how to find them and how to read them.

Which line? The least-squares criterion

You could draw many plausible lines through a cloud of points, and people drawing by eye disagree. A rule is needed, and the standard one is least squares: choose the intercept and slope that make the sum of the squared residuals as small as possible.

SSE=∑(y−y^)2=∑(y−b0−b1x)2as small as possible\text{SSE} = \sum (y - \hat{y})^2 = \sum \bigl(y - b_0 - b_1 x\bigr)^2 \quad \text{as small as possible}

The line found this way is the least-squares line, also called the regression line of y on x. Three choices are built into the rule. Misses are measured vertically, in y’s units, because the line’s job is to give y for a known x. They are squared, so misses above and below the line can’t cancel and each is counted as a positive amount. And squaring makes a large miss cost far more than a small one: a residual of $200 counts 100 times as much as one of $20. That is why the line has simple formulas, and also why a few extreme points can pull it, as outliers pulled r in B05.

Harrier Cycle Hire (fictional): six days of the day’s high and bikes hired, with the least-squares line. Each dashed segment is a residual, measured vertically. The line passes through the point of means, (19, 48), circled.

The slope and intercept

Calculus finds the minimum (In depth shows where it comes from), and the answer is two short formulas. The slope is the sum of cross-products of deviations from the means divided by the sum of squared deviations of x; equivalently, it is the correlation scaled by the two standard deviations:

b1=∑(x−xˉ)(y−yˉ)∑(x−xˉ)2=r sysxb0=yˉ−b1xˉb_1 = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2} = r\,\frac{s_y}{s_x} \qquad b_0 = \bar{y} - b_1\bar{x}

At Harbourside, transactions have mean 65.46 and standard deviation 33.57 a day; sales have mean $358.17 and standard deviation $224.68; and r = 0.9922. So b1=0.9922×224.68÷33.57=6.64b_1 = 0.9922 \times 224.68 \div 33.57 = 6.64 dollars per transaction, and b0=358.17−6.641×65.46=−76.58b_0 = 358.17 - 6.641 \times 65.46 = -76.58 dollars. The fitted line is sales^=−76.58+6.64×transactions\widehat{\text{sales}} = -76.58 + 6.64 \times \text{transactions}, the line Excel’s SLOPE and INTERCEPT, or a linear trendline, would give (B03).

The second formula says the line always passes through the point of means (xˉ,yˉ)(\bar{x}, \bar{y}): a day with average transactions is given average sales. The first says something just as useful. Measured in standard deviations, the slope is r itself: a day one standard deviation above average in x is placed r standard deviations above average in y. Because r is never larger than 1 in size, the line never predicts y to be more unusual than x is. The section on regression toward the mean below makes this a central idea.

Millbrook’s line

At Millbrook, the office-district café, the same calculation gives sales^=−34.36+3.60×transactions\widehat{\text{sales}} = -34.36 + 3.60 \times \text{transactions}. A transaction there goes with $3.60 of sales against $6.64 at Harbourside, which fits what B05 found: Millbrook’s customers buy smaller orders. The two cafés need two lines. Pooling all 612 rows gives a slope of $3.35 and an intercept of +$67.37, a line that runs between the two bands and describes neither café, for the reason B05’s Simpson’s-paradox lesson gave: the gap between the groups is mixed up with the relationship inside each.

In depth

Where the formulas come from

SSE is a smooth bowl-shaped function of b0b_0 and b1b_1; at its lowest point both partial derivatives are zero. Setting them to zero gives the two normal equations:

∑e=∑(y−b0−b1x)=0∑x e=∑x (y−b0−b1x)=0\sum e = \sum (y - b_0 - b_1 x) = 0 \qquad \sum x\,e = \sum x\,(y - b_0 - b_1 x) = 0

The first gives b0=yˉ−b1xˉb_0 = \bar{y} - b_1\bar{x}; substituting it into the second gives the slope formula. Two facts follow that hold for every least-squares line with an intercept: the residuals add up to zero, so the line is not on average too high or too low; and the residuals have zero correlation with x, so whatever straight-line pattern x carries has been taken into the line. The next lesson uses both to split the variation in y into a part the line accounts for and a part it doesn’t.

Why squares rather than, say, the sum of absolute misses? Least absolute deviations is a legitimate method, more resistant to outliers, but it has no closed-form answer and can have several equally good lines. Least squares gives one line, by formula, and its theory (standard errors, tests, intervals) is the one module 2 builds on. The price is sensitivity to extreme points, which the transformations-and-influence lesson measures.

Units, and what rescaling does

The slope carries units: dollars of sales per transaction, people per °F, °F per person. Change the units and the slope changes with them, though r does not. Per 10 transactions Harbourside’s slope is $66.41; for temperatures in °C instead of °F a slope per degree is multiplied by 1.8, since one °C is 1.8 °F. The intercept is in y’s units. Always write both with their units; a bare “6.64” invites the reader to guess.

The intercept, and when it means nothing

The intercept is the fitted value at x = 0. Harbourside’s is −$76.58: on a day with no transactions, the line predicts negative sales. That is impossible, and no one should read it as a forecast. There are two reasons it comes out this way, and both are worth recognising.

First, x = 0 is far outside the data. Harbourside’s quietest day had 13 transactions, and the line was fitted to days with 13 to 293. Reading the intercept is extrapolation to a place with no evidence. Second, the line is steeper than an average sale. Over the year Harbourside took $109,600.64 from 20,032 transactions, $5.47 per transaction. Because the line passes through the means, b0=xˉ (yˉ/xˉ−b1)=65.46×(5.47−6.64)=−76.58b_0 = \bar{x}\,(\bar{y}/\bar{x} - b_1) = 65.46 \times (5.47 - 6.64) = -76.58: the intercept is negative exactly when the slope is larger than sales per transaction. And the slope is larger because tickets are bigger on busy days (B05 found that inside each café busier days go with larger average tickets; it is planted in the case’s data). So the line says something true: moving to a busier day adds a transaction and also raises the average ticket on all of that day’s transactions, so sales rise by more than one average ticket. Its intercept is just where that straight line happens to cross zero. The festival days don’t cause it: without them the slope is $5.91 against $5.38 per transaction, still above, and the intercept is still negative.

The $5.47 is total sales divided by total transactions. The average of the 306 daily tickets, the figure B05 quotes (about $5.34), is lower, because it gives a quiet day the same weight as a busy one and quiet days have smaller tickets. At Millbrook the line’s $3.60 is likewise above $3.32 a transaction (B05’s average daily ticket there is about $3.27), and its intercept is −$34.36.

An intercept does mean something when x = 0 is a real, observed situation. Regress Harbourside’s footfall on the day’s rain and the line is footfall^=146.6−81.1×rain (in)\widehat{\text{footfall}} = 146.6 - 81.1 \times \text{rain (in)}. Rain is 0 on 209 of the 306 days, so 146.6 people is the fitted footfall on a dry day, close to the 146.6 that the dry days actually averaged (they agree because the wet days lie close to a straight line; if rain’s effect were curved they would not). When 0 isn’t meaningful, you can make the intercept meaningful by centring x: fit sales on (transactions − 65.46), and the intercept becomes the fitted sales on a day with average transactions, $358.17, while the slope is unchanged.

One slope, two lines

Which variable is x matters. At Harbourside the day’s high (mean 64.18 °F, standard deviation 15.00) and footfall (mean 139.69, standard deviation 69.29) correlate at r = 0.436. The regression of footfall on the high is footfall^=10.33+2.016×high\widehat{\text{footfall}} = 10.33 + 2.016 \times \text{high}: about 2.0 more people for each degree warmer. The regression of the high on footfall is high^=50.99+0.0944×footfall\widehat{\text{high}} = 50.99 + 0.0944 \times \text{footfall}. If the two were the same line, turning the second round would give 1 ÷ 0.0944 = 10.6 people per degree, five times the first slope. They are different lines.

by⋅x=r sysxbx⋅y=r sxsyby⋅x bx⋅y=r2b_{y\cdot x} = r\,\frac{s_y}{s_x} \qquad b_{x\cdot y} = r\,\frac{s_x}{s_y} \qquad b_{y\cdot x}\, b_{x\cdot y} = r^2

Here 2.016×0.0944=0.190=0.43622.016 \times 0.0944 = 0.190 = 0.436^2. The lines coincide only when r=±1r = \pm 1; the weaker the correlation, the further apart they swing, both still passing through the point of means. Neither is wrong: the regression of y on x makes the vertical misses smallest and answers “what footfall goes with this temperature?”; the regression of x on y makes the horizontal misses smallest and answers “what temperature goes with this footfall?” Choose by the question. Ana plans staff from the forecast high, so she wants footfall on the high. Swapping the arguments of SLOPE, =SLOPE(high,footfall) instead of =SLOPE(footfall,high), silently gives the other line.

Regression toward the mean

Write each variable in standard units, zx=(x−xˉ)/sxz_x = (x - \bar{x})/s_x and likewise for y. Then the least-squares line becomes simply

z^y=r zx\hat{z}_y = r\,z_x

A Harbourside day one standard deviation warmer than average, about 79 °F, is given footfall only 0.436 standard deviations above average: 139.69+0.436×69.29=169.9139.69 + 0.436 \times 69.29 = 169.9 people. Run it the other way and a day with footfall one standard deviation above average (209 people) is given a high only 0.436 standard deviations above average, 70.7 °F. In both directions the predicted value is closer to its mean, in standard units, than the value it was predicted from. This is regression to the mean, and it is where regression got its name: Francis Galton’s 1886 paper on heights described the children of very tall or very short parents as less extreme than their parents, “regression towards mediocrity”.

Nothing pulls the values back. Whenever two measurements are imperfectly correlated, the cases that are extreme on one are, on average, less extreme on the other, because part of what made them extreme was particular to that measurement: the day’s rain, a passing coach party, chance. The trap is in selection. Pick the quietest days, the worst stores or the patients with the highest readings, and their next measurement will tend to be better whatever you do. A change made in between gets the credit unless you compare with a group that was chosen the same way and left alone.

Ana’s log shows it plainly. Take Harbourside’s weekdays from 1 May to 30 November (152 days). The ten quietest averaged 74.6 people; the same weekday a week later averaged 113.9, most of the way back to the 126.2 of all 152 weekdays. Nine of the ten quiet days had rain, and rain is drawn afresh each day in this log, so the next week was usually drier. The ten busiest averaged 172.4 and the same weekday a week later 145.5. Nothing was done to either group. The correlation between a weekday’s footfall and the next week’s is only 0.370, so a week later the line expects most of the gap to have closed.

In Excel

B03 covered =SLOPE(known_y's,known_x's), =INTERCEPT(known_y's,known_x's), the chart trendline and LINEST. Two formulas make this lesson’s algebra visible in a worksheet, with transactions in B2:B21 and sales in C2:C21: the slope from r and the standard deviations, =CORREL(B2:B21,C2:C21)*STDEV.S(C2:C21)/STDEV.S(B2:B21), and the intercept from the means, =AVERAGE(C2:C21)-SLOPE(C2:C21,B2:B21)*AVERAGE(B2:B21). Each returns exactly what SLOPE and INTERCEPT return. The regression of x on y is =SLOPE(B2:B21,C2:C21): the same function with the ranges swapped. =LINEST(C2:C21,B2:B21) spills the slope and the intercept side by side, slope first.

What a line can’t tell you yet

A least-squares line can be fitted to any scatterplot, curved or not, and it always passes through the means. Whether a straight line is a fair summary, and how far the points typically lie from it, is the next lesson’s question (residuals, R² and the residual standard error). Whether the slope could be chance, and how precisely it is known, is module 2’s. And a slope from one explanatory variable mixes in everything that travels with x; holding other variables fixed is module 3’s.

Formulas

The fitted line and residual

y^=b0+b1x,e=y−y^\hat{y} = b_0 + b_1 x, \qquad e = y - \hat{y}
y^\hat{y}
Fitted value of the response for a given xy’s units
b0b_0
Intercept: the fitted value at x = 0y’s units
b1b_1
Slope: the change in ŷ per unit of xy’s units per x’s unit
xx
Explanatory variableits units
yy
Observed responseits units
ee
Residual: observed minus fittedy’s units

Population counterparts, used from module 2: β₀ and β₁.

The least-squares criterion

SSE=∑i=1n(yi−b0−b1xi)2   is minimised\text{SSE} = \sum_{i=1}^{n} (y_i - b_0 - b_1 x_i)^2 \;\text{ is minimised}
SSE\text{SSE}
Sum of squared residuals (error sum of squares)y’s units squared
nn
Number of observations
xi,yix_i, y_i
The i-th observationtheir units
b0,b1b_0, b_1
Intercept and slope being chosenas above

Assumes. None to fit the line; the line is a description. Inference about it (module 2) needs conditions.

Least-squares slope

b1=∑(x−xˉ)(y−yˉ)∑(x−xˉ)2=r sysxb_1 = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2} = r\,\frac{s_y}{s_x}
b1b_1
Slopey’s units per x’s unit
x,yx, y
The two variablestheir units
xˉ,yˉ\bar{x}, \bar{y}
Means of x and ytheir units
rr
Pearson correlation of x and y
sx,sys_x, s_y
Sample standard deviations of x and ytheir units

Excel: =SLOPE(known_y's,known_x's), y first.

Least-squares intercept

b0=yˉ−b1 xˉb_0 = \bar{y} - b_1\,\bar{x}
b0b_0
Intercepty’s units
b1b_1
Slopey’s units per x’s unit
xˉ,yˉ\bar{x}, \bar{y}
Means of x and ytheir units

So the line passes through (x̄, ȳ). Excel: =INTERCEPT(known_y's,known_x's).

The regressions of y on x and of x on y

by⋅x=r sysx,bx⋅y=r sxsy,by⋅x bx⋅y=r2b_{y\cdot x} = r\,\frac{s_y}{s_x}, \qquad b_{x\cdot y} = r\,\frac{s_x}{s_y}, \qquad b_{y\cdot x}\,b_{x\cdot y} = r^2
by⋅xb_{y\cdot x}
Slope for predicting y from xy’s units per x’s unit
bx⋅yb_{x\cdot y}
Slope for predicting x from yx’s units per y’s unit
rr
Correlation of x and y
sx,sys_x, s_y
Standard deviationstheir units

The two lines coincide only when r = ±1.

The line in standard units

z^y=r zx,zx=x−xˉsx\hat{z}_y = r\,z_x, \qquad z_x = \frac{x - \bar{x}}{s_x}
zxz_x
x in standard deviations from its mean
x,xˉx, \bar{x}
A value of x and the mean of xx’s units
sxs_x
Standard deviation of xx’s units
z^y\hat{z}_y
Fitted y in standard deviations from its mean
rr
Correlation of x and y

With |r| below 1 the fitted value is always nearer its mean than x is: regression toward the mean.

Worked examples

A first example: Six days of bike hire

Fictional figures

Harrier Cycle Hire (fictional), a seaside kiosk, recorded the day’s high (°C) and bikes hired on six days. Find the least-squares line by hand and interpret its slope.

High (°C)Bikesx − x̄y − ȳ(x − x̄)(y − ȳ)(x − x̄)²
1230-7-1812649
1538-4-104016
1845-1-331
20501221
23584104016
266771913349
Sum: 11428800344132
  1. Means: x̄ = 114 ÷ 6 = 19 °C and ȳ = 288 ÷ 6 = 48 bikes.
  2. Deviations and their products are in the table. The deviations each add to 0, as they always do.
  3. Slope (the least-squares formula): b1=344÷132=2.606b_1 = 344 \div 132 = 2.606 bikes per °C.
  4. Intercept (the line passes through the means): b0=48−2.606×19=−1.52b_0 = 48 - 2.606 \times 19 = -1.52 bikes.
  5. Line: bikes^=−1.52+2.61×high\widehat{\text{bikes}} = -1.52 + 2.61 \times \text{high}. Check one point: at 20 °C it gives −1.52+2.606×20=50.6-1.52 + 2.606 \times 20 = 50.6, against 50 actually hired.

Answer. ŷ = −1.52 + 2.61x: about 2.6 more bikes hired for each degree warmer, on these six days.

What it means. The slope is the useful number: a forecast of 25 °C rather than 20 °C goes with about 13 more hires. The intercept, −1.5 bikes at 0 °C, is meaningless: the kiosk’s days ran from 12 to 26 °C, and a negative number of hires is impossible. Six days is also very little evidence; module 2 shows how uncertain such a slope is.

A harder example: What a transaction is worth at each café

Fictional figures

From the year’s summary statistics, find the least-squares line of sales on transactions at Harbourside and at Millbrook, explain why both intercepts are negative, and say what the slopes do and don’t mean.

HarboursideMillbrook
Mean transactions65.46124.38
SD of transactions33.5745.65
Mean sales ($)358.17413.07
SD of sales ($)224.68167.22
r0.99220.9821
  1. Harbourside slope: b1=0.9922×224.68÷33.57=6.641b_1 = 0.9922 \times 224.68 \div 33.57 = 6.641 dollars per transaction; intercept b0=358.17−6.641×65.46=−76.58b_0 = 358.17 - 6.641 \times 65.46 = -76.58 dollars.
  2. Millbrook slope: b1=0.9821×167.22÷45.65=3.597b_1 = 0.9821 \times 167.22 \div 45.65 = 3.597; intercept b0=413.07−3.597×124.38=−34.36b_0 = 413.07 - 3.597 \times 124.38 = -34.36 dollars.
  3. Sales per transaction over the year (ȳ ÷ x̄): $5.47 at Harbourside, $3.32 at Millbrook. Each slope is above its café’s figure, so each line, passing through the means, must cross x = 0 below zero: b0=xˉ(yˉ/xˉ−b1)b_0 = \bar{x}(\bar{y}/\bar{x} - b_1).
  4. Why the slopes are steeper than an average sale: inside each café, busier days go with larger average tickets (B05; planted in the data), so sales rise faster than in proportion to transactions.
  5. The festival days: Harbourside’s five festival days had 242 to 293 transactions, among the busiest of the year. Leave them out and the slope is $5.91, the intercept −$32.56. Squared misses give such far-out points a strong pull, which the transformations-and-influence lesson measures.

Answer. Harbourside: sales = −76.58 + 6.64 × transactions. Millbrook: sales = −34.36 + 3.60 × transactions. Both intercepts are negative because each slope exceeds the café’s sales per transaction.

What it means. Across Harbourside’s days, each extra transaction went with $6.64 more in sales, on average; at Millbrook, $3.60. These are comparisons between days, and days with more transactions also differ in other ways (bigger groups, warmer weather, festivals), so the slope is not the price of one more sale. Neither intercept is a forecast: no day had fewer than 13 transactions, and sales can’t be negative.

Applied: a business decision: A promotion after the quietest days?

Fictional figures

Ana is considering a “we’ve missed you” push through the loyalty app, sent after any unusually quiet weekday at Harbourside, and plans to judge it by comparing footfall on the quiet day with footfall on the same weekday a week later. Before she runs it, what does the log say would happen with no push at all? Use Harbourside’s 152 weekdays from 1 May to 30 November.

  1. Choose the cases the way the plan would: the ten quietest weekdays, which averaged 74.6 people.
  2. Look a week later, with nothing done: the same weekdays averaged 113.9 people, a rise of 39.3 people, or 53%.
  3. Compare with the regression line of next week’s footfall on this week’s: next^=79.86+0.369×this\widehat{\text{next}} = 79.86 + 0.369 \times \text{this} (r = 0.370). At 74.6 it predicts 107.4: most of the rise is what imperfect correlation implies.
  4. Explain it: nine of the ten quiet days had rain, and in this log rain on one day says nothing about rain a week later. The quiet days were extreme partly by bad luck that didn’t repeat.
  5. Check the other tail: the ten busiest weekdays averaged 172.4, and a week later 145.5. The fall is the same effect.

Answer. With no promotion, the quietest weekdays were followed a week later by footfall 53% higher (74.6 to 113.9 people). A before-and-after comparison would credit the push with a rise that happens anyway.

What it means. If Ana runs the push, she should compare quiet days that got it with quiet days, chosen the same way, that didn’t: for example, send it after a randomly chosen half of the quiet days. Only the difference between those groups is evidence about the push. The same trap catches bonuses for the best month, extra training for the worst stores and any intervention aimed at extremes.

Explore

Thirty June days at Harbourside: transactions and sales. Move the intercept and slope and watch the residuals and their sum of squares; try to beat the least-squares line, then press the button to see it. Notice that a line through the middle of the cloud can still be too flat or too steep.

Loading the interactive…

The line in standard units. Set r, the two standard deviations and how unusual a day is in x; read both slopes, the fitted value and how far it has regressed toward the mean. Try r = 0.436 with Harbourside’s standard deviations, then r = 0.99.

Loading the interactive…

All 306 Harbourside days, footfall against the day’s high, with the least-squares line of footfall on the high (tick “Show the least-squares line”). The cloud is wide (r = 0.436), which is exactly when the regressions of y on x and of x on y differ most.

Loading the interactive…

The June days in a worksheet: SLOPE and INTERCEPT, the slope rebuilt from CORREL and STDEV.S, the intercept from the means, and the regression the other way round. Change a day’s sales and every result updates.

Loading the interactive…

Common mistakes

  • “The intercept is the sales on a day with no customers.”

    Only if x = 0 is inside the data. Harbourside’s quietest day had 13 transactions, and its line crosses zero at −$76.58, an impossible value. The intercept is where a straight line fitted to busy and quiet days happens to meet the axis; read it only when x = 0 is a real, observed situation, or centre x so the intercept describes a typical day.

  • Swapping known_y’s and known_x’s

    =SLOPE(footfall,high) and =SLOPE(high,footfall) are different lines, not reciprocals: 2.016 people per °F against 0.0944 °F per person, whose product is r² = 0.190. Put the variable you want to predict first.

  • “The slope is what one more customer adds.”

    A slope compares days that differ in x, and everything that travels with x comes along: Harbourside’s busier days are also warmer, weekend and festival days with bigger tickets. That is why its slope, $6.64, is above the $5.47 an average transaction brings in. Read it as “goes with”, not “adds”.

  • Fitting one line to two groups

    All 612 café-days give sales = 67.37 + 3.35 × transactions, a line that runs between Harbourside’s band and Millbrook’s and describes neither. When the scatterplot shows separate groups, fit each, or (module 3) add the group to the model.

  • Crediting an action with regression toward the mean

    Choose cases because they were extreme and they will tend to be less extreme next time, with or without your action: Harbourside’s ten quietest weekdays were 53% busier a week later with nothing done. Compare with a group chosen the same way that didn’t get the action.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading four questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading six questions…

In practice

Fictional case

Ana budgets the cafés’ till takings for 2028 from the transactions each café expects. Her note: “Across 2027, each extra transaction went with $6.64 of sales at Harbourside and $3.60 at Millbrook (least-squares lines of daily sales on transactions). Use these for days in the range we have seen, 13–293 transactions at Harbourside and 38–231 at Millbrook. Don’t use the intercepts: they are negative and describe no real day. Harbourside’s slope leans on its five festival days; without them it is $5.91, so for an ordinary day I use the line fitted without them and treat festivals separately.”

The note does what a careful analyst does with any slope: it gives the number with its units and its source, keeps each café on its own line, states the range the line was fitted to, refuses to read the intercept, and flags the handful of days that move the answer. It also chooses the direction of the regression from the decision: Ana knows the transactions she expects and wants sales, so sales is y.

The same reading turns up across businesses. A hotel regresses room-service revenue on occupied rooms and quotes dollars per occupied room, not the intercept at zero rooms. A battery-storage installer regresses installation hours on system size and uses the slope in quotes, knowing that a tiny system still needs a site visit the line doesn’t see. A retailer regresses online sales on site visits; marketing wants visits on sales to set traffic targets, which is a different line, not the first turned round.

Regression toward the mean deserves its own place in practice. A sales director who sends the bottom ten branches on a training course and sees them improve, or rewards last quarter’s top performers and watches them slip, is often watching regression toward the mean. The defence is to plan the comparison before acting: pick the extremes, give the action to some and not others, and compare the groups.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.

Loading six questions…

Summary

  • The least-squares line ŷ = b₀ + b₁x makes the sum of squared vertical misses, SSE = Σ(y − ŷ)², as small as possible.
  • b₁ = Σ(x − x̄)(y − ȳ) ÷ Σ(x − x̄)² = r·s_y ÷ s_x and b₀ = ȳ − b₁x̄, so the line passes through the point of means.
  • Read a slope in y’s units per unit of x, as a comparison between cases (“goes with”), not as the effect of changing x. Harbourside: $6.64 of sales per transaction; Millbrook: $3.60.
  • The intercept is the fitted value at x = 0; read it only when x = 0 is inside the data. Harbourside’s −$76.58 means nothing: no day had fewer than 13 transactions.
  • The regression of y on x and that of x on y are different lines; their slopes multiply to r². Choose the one whose y you want to predict.
  • In standard units ẑ_y = r·z_x: predictions are less extreme than the values they come from. Cases chosen for being extreme tend to be less extreme next time, with or without an intervention.
  • Excel: SLOPE and INTERCEPT take y first; =CORREL(x,y)*STDEV.S(y)/STDEV.S(x) gives the same slope.

Key formulas

Terms introduced

Fitted value
The value of the response that a fitted line or model gives for a case’s values of the explanatory variables, written ŷ.
Slope
In a fitted line ŷ = b₀ + b₁x, the coefficient b₁: the change in the fitted value for each one-unit increase in x, in y’s units per unit of x.
Intercept
In a fitted line ŷ = b₀ + b₁x, the coefficient b₀: the fitted value when x is 0. It has a sensible meaning only when x = 0 is within or close to the data.
Least squares
The rule for fitting a line or model that chooses the coefficients making the sum of squared residuals as small as possible.
Centring
Subtracting a fixed value, often the mean, from a variable before fitting, so that the intercept describes a typical case rather than x = 0. The slope is unchanged.

Sources

  1. Illowsky, B. & Dean, S. (2023). Introductory Statistics 2e, section 12.3, “The Regression Equation”. OpenStax. openstax.org/books/introductory-statistics/pages/12-3-the-re…
  2. Microsoft Support. “SLOPE function”. support.microsoft.com/en-us/excel/functions/slope-function
  3. Wikipedia. “Regression toward the mean” (on Galton’s 1886 paper and the standardised regression line). en.wikipedia.org/wiki/Regression_toward_the_mean