The idea
The mean adds everything up and divides by the count; the median is the middle value once the data are sorted. When a few values are far out on one side (skew), the mean follows them and the median barely moves. For incomes, house prices and order sizes, report the median, or both.
Spread has the same split. The standard deviation uses every value’s distance from the mean, so one extreme value can inflate it. The interquartile range (IQR), the distance between the 25th and 75th percentiles, describes the middle half and ignores the extremes.
Pearson’s correlation r measures how closely two columns follow a straight line, from −1 to +1. Spearman’s is Pearson’s on the ranks, so it resists outliers and catches any steadily rising or falling pattern. Neither says anything about cause: a third factor can drive both.
The formula
x̄ = Σ xin, s = √[ Σ (xi − x̄)2 ÷ (n − 1) ], r = Σ (xi − x̄)(yi − ȳ)√[Σ (xi − x̄)2 × Σ (yi − ȳ)2]
- x̄
- the mean of the values x1 … xn
- s
- the sample standard deviation; dividing by n − 1 corrects for measuring spread around an estimated mean
- IQR
- Q3 − Q1, the 75th minus the 25th percentile
- r
- Pearson’s correlation; Spearman’s ρ is the same formula applied to the ranks of x and y
Percentiles here interpolate between neighbouring sorted values, as spreadsheets and NumPy do by default.
A worked example
Seven households report monthly incomes of $3k, $4k, $4k, $5k, $6k, $7k and $30k. Describe the typical household and the spread.
- Mean: (3 + 4 + 4 + 5 + 6 + 7 + 30) ÷ 7 = 59 ÷ 7 = $8.43k. Median: the fourth of seven sorted values, $5k. Six of the seven households earn less than the mean.
- Standard deviation: $9.61k, larger than the mean itself. IQR: Q3 − Q1 = 6.5 − 4 = $2.5k.
- Drop the $30k household and look again: the mean falls to $4.83k and the median to $4.5k; the standard deviation shrinks to $1.47k, the IQR only to $1.75k.
- One value moved the mean by $3.6k and the standard deviation by $8.1k, but the median by $0.5k and the IQR by $0.75k.
The typical household earns about $5k a month, and the middle half spans $2.5k. The mean and standard deviation describe the one outlier more than the other six.
Try it
What the two correlations see
- Spearman’s ρ (on ranks)
- +0.99
- r², share of y’s variation on a straight line
- 99%
A strong, steady relationship: both measures agree.
The common mistake
Reading a correlation as a cause
Over twelve months of invented data, ice-cream sales and drownings correlate at r = 0.97. Concluding that ice cream causes drowning, or that banning it would save swimmers, is wrong: hot weather raises both. A strong correlation says two things move together, not why.
Remember
- Skewed data: the mean chases the long tail. Report the median, and the IQR for spread.
- Pearson’s r measures straight-line association only; a strong curve can score 0. Spearman’s ρ handles outliers and steady curves.
- Correlation is not cause. Look for a third factor that could drive both columns.
Check yourself
Three questions from the Learn room’s bank. Your answers update your mastery of this skill, the same as in Practise.
Use it on real numbers
Open Describe and Correlate, in Business analytics › Explore your data and try the idea on your own numbers.