Testing a correlation

In depth

Conditions

The t test of r rests on assumptions. OpenStax lists them in regression terms: a linear relationship; for each x, y values roughly normal around the line with the same spread; independent errors; and data from a random sample or a randomised experiment. In practice: plot the data first (a curve or a slip can make r, and its test, misleading), and ask whether the pairs are independent.

Independence is the condition most often broken by business data. Ana’s days come in sequence, and some of her columns follow the season: a warm day is usually followed by another warm day, and footfall at the waterfront rises and falls with the summer. At Harbourside the correlation between one day’s high and the next day’s is 0.93. Days that resemble their neighbours carry less independent information than their number suggests, so the test treats 306 days as more evidence than they are and its p-values come out too small; for temperature against footfall the effective number of days is far below 306. Rain, by contrast, is close to independent from one day to the next in this log (the same day-to-day correlation is −0.08), so its tests are less affected. Pooling the cafés adds a different problem: each day’s rain appears twice, once for each café, so the 612 rows hold only 306 different weather days. Treat a p-value from a time series as a rough guide, and be most cautious when it is near α or when both variables follow the season.

One-sided and two-sided tests

If the question, fixed before looking at the data, is whether rain goes with fewer customers, the alternative is one-sided, $H_1: \rho < 0$, and the p-value is the lower tail, =T.DIST(t,n-2,TRUE). For Millbrook that is 0.037, half the two-sided 0.074, and below 0.05. This is exactly the temptation the hypothesis-testing lesson warned about: switching to one-sided after seeing the two-sided p-value doubles the real false-alarm rate. Ana’s question when she started the log was “does rain make any difference?”, so she keeps the two-sided test.

Critical values of r

Solving $t = r\sqrt{n-2}/\sqrt{1-r^2}$ for r at the critical t gives the critical value of r directly:

OpenStax’s Method 2 uses a printed table like this one instead of a p-value. The table makes the trade-off plain: with a handful of pairs only very strong correlations can be told apart from chance, and with hundreds almost any correlation can.

A confidence interval for ρ

A confidence interval says how large the population correlation plausibly is, which a p-value doesn’t. The sampling distribution of r is skewed when $\rho$ is far from 0 (r can’t go above 1), so the usual estimate ± margin doesn’t work. Fisher’s z transformation fixes this: $z = \operatorname{atanh}(r) = \tfrac{1}{2}\ln\frac{1+r}{1-r}$ is close to normal with standard error $1/\sqrt{n-3}$, whatever $\rho$ is. Build the interval for z, then transform its ends back with $\tanh$.

For Harbourside’s rain: $z = \operatorname{atanh}(-0.2276) = -0.2316$, $SE = 1/\sqrt{303} = 0.0574$, so the 95% interval for z is $-0.2316 \pm 1.96 \times 0.0574$, from −0.3442 to −0.1190. Back-transformed, the interval for $\rho$ runs from −0.331 to −0.118. In Excel: =FISHERINV(FISHER(B1)-NORM.S.INV(0.975)/SQRT(B2-3)) and =FISHERINV(FISHER(B1)+NORM.S.INV(0.975)/SQRT(B2-3)), with r in B1 and n in B2.

The interval is not symmetric about r: it stretches further on the side away from ±1. It also agrees with the test: it excludes 0, as a two-sided test at 5% that rejects ρ = 0 requires (approximately, since the test uses t and the interval Fisher’s z). Millbrook’s interval, −0.212 to 0.010, includes 0. For the eight April days it runs from −0.120 to 0.926: anything from a weak negative association to a near-perfect positive one. That width is the honest summary of what eight days can say.

Testing a value other than 0

Sometimes the question is not whether $\rho$ is 0 but whether it exceeds some value that matters, such as 0.3. The t test above works only for $\rho_0 = 0$; Fisher’s z handles any value: $z = (\operatorname{atanh} r - \operatorname{atanh}\rho_0)\sqrt{n-3}$, compared with the standard normal. A hardware chain (fictional) finds r = 0.42 between store floor area and the number of product lines sold, across 200 stores, and wants to know whether the correlation is above 0.3. Then $z = (0.448 - 0.310) \times \sqrt{197} = 1.94$, and the two-sided p-value is 0.052: not quite significant at 5%, which matches the 95% interval for $\rho$, 0.299 to 0.528, whose lower end sits just below 0.3.

Tests of rank correlations

Spearman’s ρ is usually tested with the same t statistic, applied to the ranks, and Kendall’s τ with a normal approximation to its distribution. Both are approximations, good for moderate samples; for small samples without ties, exact tables or permutation tests are better. For rain and footfall at Harbourside, Spearman’s ρ = −0.226 and τ-b = −0.173, both with p below 0.001: the same conclusion as r, without relying on normality.

Power and sample size

Power, from the hypothesis-testing lesson, is the chance of rejecting $H_0$ when a particular alternative is true. Fisher’s z gives a simple approximation: with true correlation $\rho$ and n pairs, the test of $\rho = 0$ rejects (two-sided, α = 0.05) with probability about $\Phi(\operatorname{atanh}|\rho|\sqrt{n-3} - 1.96)$. Turned round, the pairs needed for power $1 - \beta$ are

Halving the correlation to detect roughly quadruples the pairs needed. Millbrook’s 306 days had about a 42% chance of detecting a true correlation of −0.1 between rain and footfall: its non-significant result is weak evidence of anything. And eight days had only an 18% chance of detecting a correlation as large as the year’s 0.44 between Harbourside’s high and its footfall.

Comparing two correlations

Harbourside’s rain correlation is significant and Millbrook’s is not. It is tempting to conclude that rain matters more at the waterfront, but “significant here, not significant there” is not a test of a difference. Compare the correlations directly. For two independent samples, transform each with Fisher’s z and use

A cinema chain (fictional) has 90 sites in one country, where local population and annual ticket sales correlate at r = 0.48, and 110 sites in another, where r = 0.22. The sites are different cinemas, so the samples are independent. Then z = (atanh 0.48 − atanh 0.22) ÷ √(1/87 + 1/107) = 2.07, and the two-sided p-value is 0.038: at 5%, population goes with ticket sales more closely in the first country.

Ana’s two cafés are not independent samples, and the formula above doesn’t apply to them. Both correlations are measured on the same 306 days, they share the same rain (or temperature) variable, and the cafés’ daily footfalls are themselves correlated (−0.39: weekends are busy at Harbourside and quiet at Millbrook). Correlations that overlap like this need a test for dependent correlations; the usual one is Steiger’s (1980) Z, which allows for the correlation between the two footfall columns. It is beyond this course to derive, but its results are easy to read. For rain, −0.228 at Harbourside against −0.102 at Millbrook gives Z = −1.34, p = 0.18: no convincing evidence that the correlations differ. For the day’s high, 0.436 against 0.157 gives Z = 3.19, p = 0.0014. The independent-samples formula would have given z = −1.59 and 3.81: different numbers, the same conclusions.

The temperature result needs one more caution. Temperature and Harbourside’s footfall both follow the season, so neighbouring days are far from independent (see Conditions) and every p-value involving temperature here, including 0.0014, is much too small. The direction of the difference (warm weather goes with more customers at the waterfront than in the office district) is plausible for the two cafés, but its strength of evidence is overstated.

Statistical and practical significance

A significant correlation can be too small to matter, and an important one can fail to reach significance in a small sample. Judge size by r, its interval and r², in the context of the decision. Rain’s pooled r of −0.14 is significant but explains under 2% of the variation in footfall; Ana would not reschedule staff on rain forecasts on that evidence. How many customers an inch of rain costs is a slope, a question for regression in B06, not for r.

Finally, every test here is one test. Ana could test all 28 pairs of eight columns in her log, and at α = 0.05 more than one would be significant by chance even if nothing were related. The correlation-matrices lesson shows how to adjust for that.