Categorical predictors and interactions

In depth

Categories with more than two levels

A categorical predictor with k levels needs $k - 1$ dummies, one for each level except the reference. Millbrook’s day of the week has seven levels; with Monday as the reference, Ana creates six dummies (Tue, …, Sun). Fitted with the high and rain over Millbrook’s 306 days, Tuesday to Friday’s coefficients are between −6.8 and +0.8 people, none significant at 5%, while Saturday’s is −168.0 and Sunday’s −162.6: each compares that day with a Monday of the same weather. The five weekdays behave alike, and so do the two weekend days, which is why one weekend dummy does almost as well. Its adjusted R² is 0.8456, against 0.8442 for the six weekday dummies, and it is one coefficient instead of six. Whether a group of dummies is needed is a question for a single test of the group, the nested F test of the multicollinearity-and-model-choice lesson, not for six separate t tests.

The choice of reference changes the coefficients but not the model. Make Millbrook the reference, with a Harbourside dummy, and the cell-means model becomes $271.8 - 157.4\,\text{Harbourside} - 161.6\,\text{Weekend} + 251.8\,(\text{Harbourside} \times \text{Weekend})$: the same four fitted means, the same R², the same residuals. Choose the reference that makes the comparisons you want to report direct, usually the largest or most familiar group.

The dummy-variable trap

Why not a dummy for every level? With an intercept in the model, the k dummies of one variable always add up to 1, the intercept’s own column, so one of them is an exact combination of the others and least squares has no unique answer. This is the dummy-variable trap, a case of perfect collinearity. Software reports it rather than fitting: the B06 engine, given all seven weekday dummies with an intercept, says the terms “are perfectly collinear with the others, so their effects can’t be separated”. The same happens with less obvious duplicates: Millbrook’s Saturday and Sunday dummies add up to its weekend column, so a model with all three can’t be fitted either. Excel’s LINEST, given collinear columns, sets one coefficient to 0 and carries on, which is easy to miss.

An interaction with a numeric predictor

An interaction can also let a slope differ between groups. Add $\text{High} \times \text{Millbrook}$ to the case’s footfall model: the product equals the high on Millbrook’s rows and 0 on Harbourside’s. The slope on the high, 1.65 people per °F, is now Harbourside’s; the interaction, −0.76 (t = −5.27), is the difference, so Millbrook’s slope is $1.65 - 0.76 = 0.89$ people per °F. In the generator, footfall rises 1.1% per °F of Harbourside’s expected trade and 0.2% at Millbrook, so in people the waterfront café is far more sensitive to the weather.

Read the main effect of the café with care. In this model the Millbrook coefficient, 206.1, is the cafés’ difference on a weekday when the high is 0°F, which never happened in 2027. Centring the temperature, using $\text{High} - 60$ in both terms, changes only that: the Millbrook coefficient becomes 160.6, the difference on a 60°F weekday, while the slopes, the fitted values and R² stay exactly the same. Centre a numeric predictor at a typical value whenever it enters an interaction, so the main effects describe something that exists.

The build-up of the multiple-regression lesson matters here too, and differs by café. Add the build-up and its interaction with the café, and the slopes become 1.43 people per °F at Harbourside and 0.45 at Millbrook; fitted to the generator’s planted mean footfall, the same model gives 1.46 and 0.44. Without the build-up, Millbrook’s slope of 0.89 is twice the planted one, because its spring was both cold and still building up.

Simpson’s paradox, with a café dummy

B05 found that across all 612 café-days footfall and the average ticket correlate at −0.324, while within each café they correlate positively (0.73 at Harbourside, 0.52 at Millbrook). In regression terms, the pooled slope of the average ticket on footfall is −$0.40 per 100 people. Add the Millbrook dummy and the slope on footfall becomes +$0.25 per 100 people, with Millbrook’s tickets $2.30 smaller at the same footfall. The dummy holds the café fixed: the slope now compares days within the same café, and the negative pooled slope came from the café difference alone (Millbrook has more people and smaller tickets).

Add the interaction $\text{Footfall} \times \text{Millbrook}$ and each café gets its own slope: $0.36 per 100 people at Harbourside and $0.36 - 0.18 = 0.18$ at Millbrook. The generator planted $0.40 and $0.22 before sales were scaled to the year’s $236,000 total, which is about $0.33 and $0.18 in the log’s dollars, so the fitted slopes are close to the truth. The model with one shared slope, $0.25, is between the two, an average weighted towards the café whose footfall varies more.

Testing an interaction

An interaction’s t test asks whether the two groups’ slopes (or effects) differ, holding the rest of the model fixed. The café × temperature interaction’s t of −5.27 says the cafés’ temperature slopes clearly differ. When a categorical predictor has more than two levels, its interaction with a numeric predictor is several product columns, and the question “do the slopes differ at all?” needs one test of all of them together: the nested F test in the next lesson.