Residuals and R² · B06 · Sunflower Intelligence

Before you start

You will be able to

Calculate residuals and interpret their sign and size in the units of the response.
Calculate the total, regression and residual sums of squares, show that the first is the sum of the other two, and find R² from them.
Interpret R² with careful wording, and show that it equals r² when there is one explanatory variable.
Calculate and interpret the residual standard error, by formula and with STEYX.
Read a residual plot for curvature, and explain why a high R² doesn’t show a line is right, a low R² doesn’t show it is useless, and R² depends on the range of x.

Review first

The least-squares line — The least-squares line, its slope and intercept, and the normal equations that make residuals add to zero.
Curves, thresholds and restricted ranges — The S-shaped iced-share curve and the way a restricted range shrinks r, both met again here through R².
Describing distributions — Sums of squares, variance and standard deviation, which R² and the residual standard error are built from.
Descriptive statistics and regression in Excel — RSQ, STEYX and the ToolPak’s R Square and Standard Error lines, read here more closely.

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

Ana now has a line for Harbourside: sales = −76.58 + 6.64 × transactions. On Wednesday 14 July the café made 81 transactions and took $484.45; the line says $461.35. On Saturday 29 May, a festival day, it made 293 and took $2,108.92; the line says $1,869.26. Is a line that misses by $23 on one day and $240 on another a good line? She needs ways to say how close the days lie to it, in dollars and in proportion.

Residuals

A residual is the observed value minus the fitted value, $e = y - \hat{y}$ , in y’s units. On 14 July it is $484.45 - 461.35 = +23.10$ dollars: the café took $23.10 more than the line expects for 81 transactions. On 29 May it is +$239.66. A negative residual means a day below the line. Each residual is the part of that day’s sales the line doesn’t account for: the ticket mix, a coach party, the till rounding, chance.

Two facts from the least-squares line’s normal equations hold for every fit with an intercept: the residuals add up to exactly zero, and they have zero correlation with x. So a residual’s sign and size carry information about the day, not about which side of the line the method favours.

Splitting the variation

Without any line, the best single guess for a day’s sales is the mean, $358.17, and the misses from it measure how much sales vary. The total sum of squares adds up their squares: $\text{SST} = \sum (y - \bar{y})^2$ . Each day’s deviation from the mean splits into two pieces, the part the line accounts for and the residual:

y - \bar{y} = (\hat{y} - \bar{y}) + (y - \hat{y})

Otterburn Laundry (fictional), five weeks. For the week with 1,000 leaflets, the distance from the mean line (dashed, ȳ = 13) to the point splits into the part the fitted line accounts for, ŷ − ȳ = 6.4, and the residual, y − ŷ = 0.6.

Square both sides and add up over the days, and for a least-squares line with an intercept the cross-term vanishes (In depth shows why), leaving

\underbrace{\sum (y - \bar{y})^2}_{\text{SST}} = \underbrace{\sum (\hat{y} - \bar{y})^2}_{\text{SSR}} + \underbrace{\sum (y - \hat{y})^2}_{\text{SSE}}

SSR, the regression sum of squares, is the variation in the fitted values; SSE, the residual (error) sum of squares, is what least squares made as small as it could. For Harbourside’s 306 days, SST = 15,397,215, SSR = 15,157,236 and SSE = 239,979 (all in dollars squared). Almost all of the day-to-day variation in sales lies along the line.

R², the share of variation accounted for

R^2 = \frac{\text{SSR}}{\text{SST}} = 1 - \frac{\text{SSE}}{\text{SST}}

For Harbourside $R^2 = 15{,}157{,}236 \div 15{,}397{,}215 = 0.984$ . Say it carefully: “the line accounts for 98.4% of the variation in Harbourside’s daily sales around their mean.” Textbooks and software often say “explains”, and that is the conventional term, but it is a statement about squared deviations, not about causes. It is not “98.4% accurate”, and not “transactions cause 98.4% of sales”. R squared runs from 0 (the line does no better than the mean) to 1 (every point on the line).

With one explanatory variable, R² is the square of the correlation: $0.9922^2 = 0.984$ . It is also the squared correlation between y and the fitted values, which is how the idea carries over to models with several explanatory variables in module 3. Because squaring loses the sign, a line with a negative slope has $r = -\sqrt{R^2}$ .

The residual standard error

R² is a proportion. Ana also needs to know how far off the line typically is, in dollars. The standard error of the regression, or residual standard error, is

s = \sqrt{\frac{\text{SSE}}{n - 2}}

For Harbourside $s = \sqrt{239{,}979 \div 304} = 28.10$ dollars: the line’s fitted sales are typically about $28 away from a day’s actual sales. Compare it with the standard deviation of sales, $224.68, the typical miss from the mean alone. In Excel it is =STEYX(known_y's,known_x's), the “Standard Error” line in the ToolPak’s regression output (B03). On 298 of the 306 days (97%) the residual is within two residual standard errors, about ±$56, a rough guide that module 4 refines.

In depth

Why the sums of squares add up

Squaring the split gives $\sum (y - \bar{y})^2 = \text{SSR} + \text{SSE} + 2\sum (\hat{y} - \bar{y})\,e$ . The fitted value is $\hat{y} = b_0 + b_1 x$ , so the cross-term is $2\sum (b_0 - \bar{y})\,e + 2b_1 \sum x\,e$ , and both sums are zero by the normal equations ( $\sum e = 0$ and $\sum x e = 0$ ). The identity holds for least squares with an intercept, which is what Excel’s SLOPE, INTERCEPT, trendline and ToolPak fit. For a line forced through the origin, or a line fitted any other way, SST ≠ SSR + SSE and “1 − SSE ÷ SST” loses its meaning; Excel then reports an R² computed about zero instead of about the mean, which is why the least-squares-line lesson warned against comparing it.

The same three sums appear in the ToolPak’s ANOVA table: SS for Regression, Residual and Total. Dividing each by its degrees of freedom gives mean squares, and their ratio is the F statistic; that is module 2’s test.

Why n − 2

The residual standard error divides by $n - 2$ , not n. Two coefficients were estimated from the same data, and the residuals satisfy two constraints (the normal equations), so only $n - 2$ of them are free to vary. Dividing by n would understate the typical miss, slightly with 306 days and badly with 6. More generally the divisor is $n - p$ , with p coefficients counting the intercept, which is why STEYX and LINEST’s third row agree with the ToolPak.

A first residual plot

B05 found a correlation of 0.94 between the day’s high and the share of drinks served iced, over both cafés’ 612 days. The least-squares line is iced share = −0.747 + 0.0176 × high, with R² = 0.883 and s = 0.096 (9.6 percentage points). By R² this is an excellent line. Plot the residuals against the high and it isn’t.

Average residual of the iced-share line by band of the day’s high (both cafés, 612 days)
High (°F)	Days	Average residual
below 45	58	+0.143
45–54	124	−0.034
55–64	128	−0.095
65–74	120	−0.001
75–84	130	+0.062
85 and above	52	+0.003

The residuals are positive on cold days, negative from 45 to 64 °F, near zero around 70 and positive again from 75 to 84: a wave. A straight line through an S-shaped relationship misses in a pattern: the line runs below the curve on the coldest days, above it through the 50s and early 60s, and below it again in the late 70s and early 80s. Worse, the line gives a negative iced share on the coldest days: it crosses zero at 42.5 °F, and on the 32 café-days below 40 °F it fits −0.13 on average where the observed share was +0.06. The data were generated with a logistic curve in temperature, centred near 69 °F, so this pattern is the truth showing through.

Read a residual plot by looking for structure. If the line is the right shape, the residuals scatter evenly around zero with no trend, curve or funnel. A curve says the relationship bends; a funnel says the spread changes with x; isolated large residuals say some days don’t fit. Harbourside’s sales line, R² = 0.984, has a pattern too: its residuals average +$26.52 on days with fewer than 40 transactions, −$11.21 and −$22.98 in the 60–79 and 80–119 bands, and +$32.92 on the 11 days with 120 or more. Five of the 11 busiest days are festival days; the other six sit below the line (−$27 on average), because the festivals pull the line steeper. Fitted without them, the residuals still curve upward at the top (+$24 on the six busiest days), so the bend is real: tickets are larger on busy days, so sales grow a little faster than in proportion to transactions. The assumptions-and-diagnostics lesson reads such plots in full; the transformations-and-influence lesson shows what to do about curves.

What R² can’t tell you

Whether the line is the right shape. Penn State’s regression course puts it bluntly: a high R² does not mean the line fits well, and a perfect curved relationship can have an R² near zero. The iced-share line, at 0.883, describes a curve with a straight line and predicts impossible values. Always plot.

Whether the line is useful. At Harbourside, footfall on the day’s high gives R² = 0.190, a figure many people would dismiss. Yet the slope is 2.0 people per °F, and March days averaged 53.1 °F against 83.5 °F in July, so the line puts July’s typical footfall about 61 people above March’s, about one extra barista hour at the morning peak, even though on any single day the line misses by about s = 62 people, because weekends, rain and festivals matter as much as temperature. Rain is weaker still, R² = 0.052, with a slope of −81 people per inch. A low R² says the line leaves most day-to-day variation unexplained; it doesn’t say the slope is unimportant. Part of that 2.0 is the cafés’ opening build-up, which ran through the cold spring; module 3 holds it fixed and finds a smaller slope (1.79 with the build-up alone held fixed, 1.44 once weekends, rain and festivals are added).

Anything apart from the range of x it was computed on. R² compares the spread along the line with the total spread of y, and the total spread depends on how widely x varies. Fit Harbourside’s sales line only on the 121 days with 50 to 70 transactions and R² falls from 0.984 to 0.859, while the typical miss falls too, from $28.10 to $13.51. The line fits those days at least as closely; there is simply less variation in sales for it to account for. This is B05’s restricted-range effect on r, squared. Comparing R² across data sets with different ranges of x compares the ranges as much as the fits.

Causes, or accuracy on new data. R² measures association in the data used to fit the line. It doesn’t show that x causes y, and it is not predictive accuracy: a line fitted to one year can do worse on the next, especially when many explanatory variables are tried (the multicollinearity-and-model-choice lesson tests models on months they weren’t fitted to). Nor can R² be compared between models of different responses, such as sales and the logarithm of sales.

In Excel

With transactions in B2:B21, sales in C2:C21, the fitted values in E2:E21 and the residuals in F2:F21: =RSQ(C2:C21,B2:B21) gives R²; =STEYX(C2:C21,B2:B21) gives s; =DEVSQ(C2:C21) is SST; =SUMSQ(F2:F21) is SSE; and =1-SUMSQ(F2:F21)/DEVSQ(C2:C21) rebuilds R² from them. =LINEST(C2:C21,B2:B21,TRUE,TRUE) returns R² and s in its third row, and SSR and SSE in its fifth. A residual plot is a scatter chart of column F against column B (or against E); the ToolPak’s “Residual Plots” box draws it for you.

Formulas

Residual

e = y - \hat{y}

$e$: Residualy’s units
$y$: Observed valuey’s units
$\hat{y}$: Fitted value from the liney’s units

For least squares with an intercept, Σe = 0 and the residuals are uncorrelated with x.

Sums of squares

\text{SST} = \sum (y - \bar{y})^2 = \underbrace{\sum (\hat{y} - \bar{y})^2}_{\text{SSR}} + \underbrace{\sum (y - \hat{y})^2}_{\text{SSE}}

$\text{SST}$: Total sum of squares: variation of y around its meany’s units squared
$\text{SSR}$: Regression sum of squares: variation of the fitted valuesy’s units squared
$\text{SSE}$: Residual (error) sum of squaresy’s units squared
$y, \hat{y}, \bar{y}$: Observed, fitted and mean values of the responsey’s units

Assumes. Least squares with an intercept.

Excel: DEVSQ(y) is SST; SUMSQ(residuals) is SSE. The ToolPak’s ANOVA table lists all three.

R² (coefficient of determination)

R^2 = \frac{\text{SSR}}{\text{SST}} = 1 - \frac{\text{SSE}}{\text{SST}} = r^2 \;\text{(one explanatory variable)}

$R^2$: Share of the variation of y around its mean that the line accounts for
$\text{SSR}, \text{SSE}, \text{SST}$: Regression, residual and total sums of squaresy’s units squared
$r$: Correlation of x and y

Excel: =RSQ(known_y's,known_x's). Adjusted R², for several explanatory variables, is module 3’s.

Residual standard error

s = \sqrt{\frac{\text{SSE}}{n - 2}}

$s$: Typical size of a residual (also s_e)y’s units
$\text{SSE}$: Residual sum of squaresy’s units squared
$n$: Number of observations

Excel: =STEYX(known_y's,known_x's). With p coefficients the divisor is n − p.

Worked examples

A first example: Leaflets and new customers

Fictional figures

Otterburn Laundry (fictional) posted leaflets (in hundreds) for five weeks and counted new customers. Its least-squares line is new customers = 3.4 + 1.6 × leaflets. Calculate the residuals, the three sums of squares, R² and the residual standard error.

Leaflets (hundreds)	New customers	Fitted	Residual
2	7	6.6	0.4
4	9	9.8	−0.8
6	14	13.0	1.0
8	15	16.2	−1.2
10	20	19.4	0.6

Fitted values: 3.4 + 1.6 × 2 = 6.6, and so on down the table. Residuals: 7 − 6.6 = 0.4, 9 − 9.8 = −0.8, 1.0, −1.2, 0.6. They add up to 0.
SSE: $0.4^2 + 0.8^2 + 1.0^2 + 1.2^2 + 0.6^2 = 3.6$ .
SST, with $\bar{y} = 13$ : $(-6)^2 + (-4)^2 + 1^2 + 2^2 + 7^2 = 106$ . So SSR = 106 − 3.6 = 102.4, which matches $\sum (\hat{y} - \bar{y})^2$ computed directly.
R² $= 1 - 3.6 \div 106 = 0.966$ .
Residual standard error: $s = \sqrt{3.6 \div (5 - 2)} = 1.10$ customers.

Answer. Residuals 0.4, −0.8, 1.0, −1.2, 0.6; SST = 106, SSR = 102.4, SSE = 3.6; R² = 0.966; s = 1.10 new customers.

What it means. The line accounts for 96.6% of the week-to-week variation in new customers, and its fitted values are typically about one customer away from the counts. Five weeks is little evidence, and weeks with more leaflets may differ in other ways (the season, a competitor closing), so this describes the association, not what an extra hundred leaflets would bring.

A harder example: Harbourside’s sales line in full

Fictional figures

For Harbourside’s 306 days, the line sales = −76.58 + 6.64 × transactions has SSE = 239,979 and the sales have SST = 15,397,215 (dollars squared). Find SSR, R², r and s, interpret them, and say what the R² of the same line fitted only to the days with 50 to 70 transactions shows.

SSR = SST − SSE = $15{,}397{,}215 - 239{,}979 = 15{,}157{,}236$ .
R² $= 15{,}157{,}236 \div 15{,}397{,}215 = 0.984$ ; the slope is positive, so $r = +\sqrt{0.984} = 0.992$ , the correlation B05 reported as 0.99.
$s = \sqrt{239{,}979 \div 304} = 28.10$ dollars, against a standard deviation of sales of $224.68.
Residuals by band of transactions: on average +$26.52 below 40, +$7.28 from 40 to 59, −$11.21 from 60 to 79, −$22.98 from 80 to 119 and +$32.92 at 120 or more. Five of those 11 days are festivals; the other six average −$27.00. Refitted without the festivals, the bands are +$6.12, −$0.08, −$5.46, +$1.36 and +$24.44: a gentle curve, upward at the top.
Restricted to the 121 days with 50–70 transactions: R² = 0.859 and s = $13.51.

Answer. SSR = 15,157,236; R² = 0.984; r = 0.992; s = $28.10. On the 50–70 days alone, R² = 0.859 with s = $13.51.

What it means. The line accounts for 98.4% of the variation in daily sales and is typically about $28 off. The residuals show it is not quite the right shape: sales rise a little faster than in proportion to transactions. Narrowing the range lowers R² while the typical miss halves, which is the clearest demonstration that R² depends on how widely x varies, not only on how well the line fits.

Applied: a business decision: Ordering iced-drink cups from the forecast

Fictional figures

Ana orders iced-drink cups a week ahead from the forecast high. A supplier’s rep suggests the line iced share = −0.747 + 0.0176 × high, “R² = 0.88, so it’s 88% accurate.” Should Ana use it?

Correct the claim: R² = 0.883 means the line accounts for 88% of the variation in the iced share around its mean. It is not a measure of accuracy. The typical miss is s = 0.096, almost 10 percentage points of the drinks.
Plot the residuals against the high (the table in In depth): positive below 45 °F, negative from 45 to 64 °F, positive from 75 to 84 °F. The line is the wrong shape.
Translate the pattern into cups: in the 55–64 °F band, typical of spring and autumn (128 café-days, about a fifth of the year), the line overstates the iced share by 0.095 on average, so it would over-order iced cups on exactly the days she has most of.
Check the cold end: below 42.5 °F the line gives a negative share, an impossible order.
Use something that follows the curve: the average iced share in each 5 °F band of the high from last year, or a curved model (the transformations-and-influence lesson fits one).

Answer. No. The high R² hides a systematic pattern: the line over-orders iced cups in the 55–64 °F band by about 9.5 percentage points of drinks and gives impossible negative orders on cold days.

What it means. R² summarises how much variation a line follows, not whether it follows the right shape. A one-minute residual plot turned an apparently excellent model into one Ana would not use for ordering. The banded averages would also need checking against the new year as it comes in.

Explore

Common mistakes

“R² = 0.88, so the model is 88% accurate.”
R² is the share of the variation of y around its mean that the line accounts for, not a hit rate or an error percentage. The iced-share line has R² = 0.883 and still misses by about 10 percentage points on a typical day. Report the residual standard error for accuracy in y’s units.
Trusting a high R² without a residual plot
A straight line through a curve can have a high R²: the iced-share line’s residuals run positive, negative, positive across temperatures, and it predicts negative shares below 42.5 °F. Plot the residuals before believing any line.
“R² is low, so x doesn’t matter.”
Harbourside’s footfall on the high has R² = 0.190, yet its slope of 2.0 people per °F puts July about 61 people above March. A low R² says much of the variation comes from elsewhere (weekends, rain), not that the slope is negligible.
Comparing R² across different ranges of x
Restricting Harbourside’s days to 50–70 transactions drops R² from 0.984 to 0.859 while the typical miss halves. A study with a wider spread of x will show a higher R² for the same relationship.
Dividing SSE by n for the residual standard error
Two coefficients have been estimated, so the residuals have n − 2 degrees of freedom: s = √(SSE ÷ (n − 2)), which is what STEYX computes. Dividing by n understates the typical miss, badly in small samples.
Reading “explained” as “caused”
“Transactions explain 98% of the variation in sales” is a statement about squared deviations. It says nothing about what causes sales, and the same R² would come from regressing transactions on sales.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading three questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading five questions…

In practice

Fictional case

Ana’s review of the café models has one line per model. “Sales on transactions, Harbourside: R² = 0.984, typical miss $28 (STEYX), residuals bend gently upward at the top even without the festival days; fine for budgeting inside the usual range. Iced share on the forecast high: R² = 0.883, typical miss 10 points, but the residuals wave, so I order from last year’s averages by 5 °F band instead. Footfall on the high, Harbourside: R² = 0.19, typical miss 62 people; useless for any single day, useful for the season (once module 3 has separated the build-up).”

Each line gives three things: R² for the share of variation, s in the response’s units for the size of a typical miss, and a sentence about the residual plot. That combination is what lets a reader decide whether a model is good enough for a purpose. A budget can live with $28 a day; a barista roster for one particular Saturday can’t live with ±62 people, but a seasonal hiring plan can.

In other businesses the same habits apply. A retailer forecasting store revenue from floor area might find R² = 0.6 and a typical miss of $400,000 a year: the R² sounds respectable, the miss decides whether it can set targets with it. A battery-storage installer modelling installation hours on system size may see residuals fanning out for larger systems, a sign that a single s overstates precision on small jobs and understates it on big ones (module 4 deals with this). And a delivery firm comparing two cities’ models by R² should first check whether the cities’ routes cover the same range of distances.

Two cautions travel with every R² Ana reports. It is computed on the days used to fit the line, so it flatters how well the line will do on next year’s days. And it describes association: a high R² between baristas rostered and sales (B05 found they go together because the roster follows expected trade) would not mean that adding baristas raises sales.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.

Loading six questions…

Summary

A residual is e = y − ŷ, in y’s units; for least squares with an intercept the residuals add to zero and are uncorrelated with x.
SST = SSR + SSE: the total variation of y around its mean splits into the part along the line and the residual part.
R² = SSR ÷ SST = 1 − SSE ÷ SST is the share of the variation the line accounts for; with one explanatory variable it equals r². It is not accuracy and not causation.
The residual standard error s = √(SSE ÷ (n − 2)) (STEYX) is the typical miss in y’s units. Report it with R².
A residual plot shows whether a line is the right shape: the iced-share line has R² = 0.883 but its residuals wave and it predicts negative shares below 42.5 °F.
A low R² can go with a slope that matters (footfall: R² = 0.19, 2.0 people per °F).
R² depends on the range of x: Harbourside’s sales line on 50–70 transactions has R² = 0.859 against 0.984 for all days, with a smaller typical miss.

Terms introduced

Total sum of squares: SST = Σ(y − ȳ)²: the variation of the response around its mean, before any explanatory variable is used. Excel’s DEVSQ.
Regression sum of squares: SSR = Σ(ŷ − ȳ)²: the part of the total sum of squares that the fitted values account for. With least squares and an intercept, SSR = SST − SSE.
Residual sum of squares: SSE = Σ(y − ŷ)²: the sum of the squared residuals, the quantity least squares makes as small as possible. Also called the error sum of squares.
Residual plot: A scatterplot of a model’s residuals against an explanatory variable, the fitted values or the order of the observations. Residuals with no pattern around zero suggest the model’s form fits; a curve, a funnel or a trend says it doesn’t.

Sources

Illowsky, B. & Dean, S. (2023). Introductory Statistics 2e, section 12.3, “The Regression Equation” (residuals, SSE and the coefficient of determination). OpenStax. openstax.org/books/introductory-statistics/pages/12-3-the-re…
Penn State Eberly College of Science. STAT 501 Regression Methods, section 1.8, “R² Cautions”. online.stat.psu.edu/stat501/book/export/html/893
Microsoft Support. “STEYX function”. support.microsoft.com/en-us/excel/functions/steyx-function

Before you start

You will be able to

Review first

The idea

Residuals

Splitting the variation

R², the share of variation accounted for

The residual standard error

In depth

Why the sums of squares add up

Why n − 2

A first residual plot

What R² can’t tell you

In Excel

Formulas

Residual

Sums of squares

R² (coefficient of determination)

Residual standard error

Worked examples

Explore

Common mistakes

“R² = 0.88, so the model is 88% accurate.”

Trusting a high R² without a residual plot

“R² is low, so x doesn’t matter.”

Comparing R² across different ranges of x

Dividing SSE by n for the residual standard error

Reading “explained” as “caused”

Guided practice

On your own

In practice

Knowledge check

Summary

Key formulas

Terms introduced

Sources