Sunflower Intelligence
Learn

Plate 10 of 19 · Business analytics

Regression: what the coefficients and R² say

About 7 minutes to read, then three questions that count toward your mastery.

A regression line turns “these move together” into “this much more of y for each unit of x”. Its numbers come with uncertainty, and the diagnostics say when to doubt them.

The idea

Ordinary least squares (OLS) draws the line that makes the squared vertical distances from the points, the residuals, as small as possible. Each slope is the expected change in the outcome for one more unit of that predictor, holding the other predictors fixed.

R² is the share of the outcome’s variation the model accounts for: 1 minus the leftover squared error over the total. With one predictor it equals the squared correlation. Adjusted R² charges for each extra predictor, so adding noise columns cannot raise it for free.

Each coefficient has a standard error, a t statistic (estimate ÷ standard error) and a p-value: how often a slope this far from zero would turn up by chance if the true effect were zero. If the residuals fan out (uneven spread), use robust HC3 standard errors; if two predictors overlap heavily (variance inflation factors above about 10), their separate effects can’t be pinned down.

The formula

ŷ = b0 + b1x, b1 = Σ (x − x̄)(y − ȳ)Σ (x − x̄)2, b0 = ȳ − b1x̄, R2 = 1 − SSESST, t = b1SE(b1)

b1
the slope: change in y for one more unit of x
b0
the intercept: the predicted y when x is 0
SSE
sum of squared residuals, Σ (y − ŷ)²
SST
total sum of squares around the mean, Σ (y − ȳ)²
t
the estimate in standard errors; its p-value comes from a t distribution with n − 2 degrees of freedom here

A worked example

Plate 10

Over eight weeks a shop spent $2k to $9k on advertising and sold $14k, $13k, $18k, $15k, $20k, $17k, $22k and $19k. How strongly are they linked?

  1. Means: advertising x̄ = 5.5, sales ȳ = 17.25. Σ (x − x̄)² = 42 and Σ (x − x̄)(y − ȳ) = 41.0.
  2. Slope b1 = 41.0 ÷ 42 = 0.976: each extra $1k of advertising goes with about $0.98k more sales. Intercept b0 = 17.25 − 0.976 × 5.5 = 11.88.
  3. R² = 1 − 27.5 ÷ 67.5 = 0.59: the line accounts for 59% of the week-to-week variation in sales.
  4. Standard error of the slope 0.330, so t = 0.976 ÷ 0.330 = 2.96, with a p-value of 0.025 on 6 degrees of freedom.
  5. Predicted sales for a $10k week: 11.88 + 0.976 × 10 = $21.64k.

The data support a positive link: about $0.98k of sales per $1k of advertising, unlikely to be chance (p = 0.025). Eight weeks is little evidence, and a third factor such as season could drive both.

Try it

Sales = 10 + true slope × week + noise. The same fixed noise is reused, so only your settings change the sample.

0.5
3.0
12

How much can a sample tell you?

0.54estimated slope (true 0.5)
95% interval for the slope
0.11 to 0.97
p-value
0.02
R²
0.44

The interval excludes zero, so the data can tell the slope from no effect.

The common mistake

Reading a p-value as the chance the effect is real

Suppose the slope’s p-value had been 0.30. That does not mean a 70% chance that advertising works. It means that if advertising had no effect at all, a slope at least this far from zero would appear in about 30% of samples like this one. The right reading is “the data can’t tell the effect apart from zero”, not “probably real”.

Remember

  • A slope is the change in y per unit of x, with the other predictors held fixed, and it describes association, not cause.
  • R² is the share of variation explained; with one predictor it is the correlation squared (R² = 0.64 means r = 0.8).
  • A p-value is not the probability the effect is real. Fanning residuals call for robust (HC3) standard errors; high VIFs mean overlapping predictors.

Check yourself

Three questions from the Learn room’s bank. Your answers update your mastery of this skill, the same as in Practise.

Use it on real numbers

Open Regress, in Business analytics › Explore your data and try the idea on your own numbers.