One-way analysis of variance (ANOVA)

In depth

The one-way ANOVA model

There are k groups. Group i has $n_i$ observations with mean $\bar{x}_i$ and sample variance $s_i^2$; there are $N = n_1 + \dots + n_k$ observations in all, with grand mean $\bar{x}$. The model says each observation is its group’s population mean plus a random error, $x_{ij} = \mu_i + \varepsilon_{ij}$, where the errors are independent, normally distributed and have the same variance $\sigma^2$ in every group. OpenStax lists the conditions the same way: normal populations, random and independent samples, equal standard deviations, a categorical factor and a numerical response.

Sums of squares

ANOVA splits the total variation into its two parts. The total sum of squares, SST, adds up every observation’s squared distance from the grand mean. The sum of squares between groups, SSB, adds up each group mean’s squared distance from the grand mean, weighted by the group’s size. The sum of squares within groups, SSW, adds up every observation’s squared distance from its own group’s mean, which is the same as $\sum (n_i - 1) s_i^2$. The three always satisfy SST = SSB + SSW.

For Lantern’s 76 events, SSB = 35.687, SSW = 230.759 and SST = 266.446, in dollars squared. Excel calculates each without the ToolPak: =DEVSQ(range) returns the sum of squared deviations from the range’s own mean, so SST is DEVSQ of all the values, SSW is the sum of each group’s DEVSQ, and SSB = SST − SSW. The worksheet in the Explore section builds a whole ANOVA table this way.

Degrees of freedom, mean squares and F

Each sum of squares has its degrees of freedom: $k - 1$ between groups (k group means, tied together by the grand mean), $N - k$ within groups (N observations, k group means estimated from them) and $N - 1$ in total; the first two add up to the third. A mean square is a sum of squares divided by its degrees of freedom: $\text{MSB} = \text{SSB} \div (k - 1)$ and $\text{MSW} = \text{SSW} \div (N - k)$. The F statistic is their ratio, $F = \text{MSB} \div \text{MSW}$.

For Lantern, $\text{MSB} = 35.687 \div 4 \approx 8.922$, $\text{MSW} = 230.759 \div 71 \approx 3.250$ and $F = 8.922 \div 3.250 \approx 2.75$. MSW is the pooled variance of the pooled t test extended to k groups, and its square root, $1.80, is the typical distance of an event from its type’s mean. F is large only when MSB is large relative to MSW: when the group means are spread out more than the within-group scatter explains.

The F distribution

If the null hypothesis and the model’s conditions hold, F follows an F distribution with k − 1 and N − k degrees of freedom, the distribution of the ratio of two independent chi-square variables, each divided by its degrees of freedom. It is never negative and is skewed to the right. Its mean, df₂ ÷ (df₂ − 2), is close to 1 when the denominator df is large: about 1.03 for Lantern’s 71. Only a large F is evidence against the null hypothesis, so the test uses the right tail; OpenStax calls one-way ANOVA “always right-tailed”.

In Excel, =F.DIST.RT(x,deg_freedom1,deg_freedom2) returns the right-tail probability, the p-value: =F.DIST.RT(2.745,4,71) ≈ 0.035. =F.INV.RT(probability,deg_freedom1,deg_freedom2) returns the critical value: =F.INV.RT(0.05,4,71) ≈ 2.50, and Lantern’s F of 2.75 lies just beyond it. Microsoft’s examples: =F.DIST.RT(15.2068649,6,4) returns 0.01 and =F.INV.RT(0.01,6,4) returns 15.20686. The numerator df comes first, and swapping the two gives a different distribution; both functions truncate a df that isn’t a whole number.

With two groups, one-way ANOVA is the pooled two-sample t test of Comparing means: F equals t², with the same p-value. Microsoft’s ToolPak page says that with only two samples you can use T.TEST instead; with type 2, the pooled test, it gives the ANOVA’s two-tailed p-value. ANOVA earns its keep with three groups or more.

Excel’s Anova: Single Factor

With the Analysis ToolPak loaded (B03 showed how), choose Data > Data Analysis > Anova: Single Factor, which Microsoft describes as a simple analysis of variance on data for two or more samples. Put each group in its own column with its name in the first row, select the block as the Input Range, set Grouped By to Columns, tick Labels in first row, and leave Alpha at 0.05 unless you want another level. The groups needn’t be the same size: leave the cells below a shorter column empty, since a 0 would count as an observation. Like B03’s Descriptive Statistics and Regression tools it writes values, not formulas, so run it again when the data change.

The output has two tables. SUMMARY lists each group’s Count, Sum, Average and Variance (the sample variance, as VAR.S gives it). ANOVA has a row each for Between Groups, Within Groups and Total, with the columns SS, df, MS, F, P-value and F crit. Lantern’s, with the event types in five columns:

P-value is F.DIST.RT of F, and F crit is F.INV.RT of Alpha, both with the Between Groups and Within Groups df. F is above F crit (2.75 against 2.50), which is the same as P-value below Alpha (0.035 against 0.05), so the null hypothesis of equal means is rejected at the 5% level. The Explore section reproduces both tables.

Checking the conditions

Independence. The observations should be independent: ideally random samples, or units assigned to groups at random. Lantern’s 76 events are not a random sample; they are every event of 2026, and the analysis treats them as a sample of the catering Lantern will go on doing. They also come from only 18 clients, Harbor Tech alone booking 11 events, and events for one client may resemble each other (the same office, menu and budget), so they are not fully independent. That is a caveat to state, not something a calculation can fix.

Normality. Each group should come from a normal population. ANOVA tolerates moderate departures, especially as samples grow (Penn State’s STAT 500 notes call its test statistic fairly robust), but outliers matter. Lantern’s meetings are right-skewed, with two invoices far above the rest ($14.62 and $17.14 per guest-day), and one team day earned $16.11.

Equal variances. The groups should share one population variance. A common rule of thumb, given in Penn State’s STAT 500 notes, is that the largest sample standard deviation should be no more than about twice the smallest. Lantern’s groups fail it: meetings have a standard deviation of $2.15 and launches $0.68, a ratio of 3.2, because launches are priced very consistently.

When the conditions are doubtful, check whether the conclusion depends on them. Welch’s ANOVA drops the equal-variance assumption by weighting each group by nᵢ ÷ sᵢ², and Delacre and colleagues (2019) recommend it as the default for comparing means. It isn’t in the ToolPak, but statistical software runs it: for Lantern it gives F ≈ 4.13 on 4 and 30.3 df, p ≈ 0.009. Here the largest group, meetings, also has the largest spread, which makes the ordinary F conservative; when the smallest group has the largest spread it errs the other way and finds differences too often.

The Kruskal–Wallis test compares the groups’ ranks instead of their values, so it doesn’t assume normality: for Lantern H ≈ 13.0 on 4 df, p ≈ 0.011. A log transformation (B02) is a third option for right-skewed groups. Here every route points the same way, so the conclusion that the types differ doesn’t rest on the equal-variance assumption.

How big are the differences?

A p-value says whether the means differ, not by how much. The simplest effect size for ANOVA is eta squared, $\eta^2 = \text{SSB} \div \text{SST}$, the share of the total variation that lies between the groups. For Lantern $\eta^2 = 35.687 \div 266.446 \approx 0.134$: 13.4% of the variation in invoice per guest-day goes with event type, and 86.6% is variation among events of the same type. Report it with the means themselves: the gap between the highest and lowest types is $2.25 per guest-day.

What a significant F does and doesn’t tell you

A significant F says the group means are not all equal. It doesn’t say which differ, or that they all differ: one group out of line is enough. A non-significant F is not proof of equal means, only a lack of evidence against them, as with any test.

To find which groups differ, compare pairs in a way that keeps the family-wise error rate in check. Tukey’s HSD (honestly significant difference) compares every pair and is the usual choice when all pairs matter; NIST’s handbook says it has the edge when all pairwise comparisons are of interest. It needs the studentised range distribution, for which Excel has no function, so it is run in statistical software. The Bonferroni correction works anywhere: to make m comparisons with a family-wise error rate of at most α, test each one at α ÷ m.

For Lantern’s five types m = 10, so each pair is tested at 0.05 ÷ 10 = 0.005. Because the variances differ, use Welch’s t test for each pair: with the types in columns A to E in the SUMMARY’s order, =T.TEST(D2:D12,E2:E12,2,3) compares launches with team days. Only that pair clears the bar, with p ≈ 0.004 and team days $2.25 higher; the next smallest p-value, conferences against launches, is 0.018. Tukey’s HSD, run in statistical software, also finds only this pair; its usual form assumes equal variances, but the Games–Howell version, which doesn’t, agrees.

Comparisons planned before seeing the data, such as each type against one standard type, need fewer adjustments than all pairs. Picking the biggest gap after looking and testing only that one is the multiple testing trap in disguise: the biggest of ten gaps is not a fair draw. The challenging example shows it happening.

Beyond one factor

Two-way ANOVA handles two categorical factors at once, for example event type and season, and tests whether their effects interact; the ToolPak’s Anova: Two-Factor With Replication needs the same number of observations for every combination (its dialog takes a single Rows per sample). ANOVA is also a special case of regression on group indicators, which B06 Regression Analysis develops. And the Kruskal–Wallis test above is the rank-based alternative when the groups are far from normal.