Encoding categorical variables · B02 · Sunflower Intelligence

Before you start

You will be able to

Explain why most models need categorical variables converted to numbers, and how arbitrary codes mislead linear and distance-based models.
Encode a nominal variable with one-hot encoding or dummy coding, and calculate how many columns an encoding creates.
Explain the dummy-variable trap and decide when dropping a reference category is necessary and when keeping every column is better.
Apply ordinal, frequency and target encoding, and state the assumption each makes.
Calculate smoothed and out-of-fold target encodings that avoid data leakage.
Handle rare categories, categories unseen at prediction time and high-cardinality variables.

Review first

Normalisation and standardisation — You need training and test sets and the rule that every data-dependent step is fitted on the training set only.
Datasets: rows, columns, records and variables — You need the difference between nominal and ordinal variables.
Text cleaning and consistent categories — Spelling variants must be consolidated before encoding, or each variant becomes a category of its own.

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

Lantern Coffee Co. wants to know which first-time customers are likely to order again within 60 days, so that it can send a welcome offer to those who probably won’t. Lantern’s analyst, Jun Park, has the online-shop export, with each order’s region, channel and subscription status, and he knows which customers came back. His first model is a logistic regression, which multiplies each input by a coefficient, adds the results and turns the total into a probability. It stops at the first column: it can multiply 1.5 by a dollar amount, but not by “North”.

Most models do arithmetic on their inputs. Linear and logistic regression multiply and add; k-nearest neighbours and k-means measure distances. A categorical variable, whose values are labels, must therefore be turned into numbers before these models can use it. That conversion is categorical encoding. The difficulty is that numbers carry meaning of their own, order and distance, which the labels may not have.

Numbers that invent an order

The quickest encoding numbers the categories, usually in alphabetical order: once the spelling variants are consolidated, Lantern’s five regions become Central 0, East 1, North 2, South 3 and West 4. This is label encoding. Now look at what a model does with it. A logistic regression gives region a single coefficient, so the effect of region has to rise or fall in equal steps from Central through East, North and South to West, an order that comes from the spelling of the names. A nearest-neighbour model concludes that a Central customer is four times as far from a West customer as from an East one. None of that is true of the regions. Region is a nominal variable: its categories have no order, so its code numbers must not have one either.

One-hot encoding

One-hot encoding replaces the column with one new 0/1 column per category. Each row has a 1 in the column for its own category and 0 everywhere else.

Five Lantern orders with region one-hot encoded (fictional)
Region	Central	North	South	West
North	0	1	0	0
West	0	0	0	1
North	0	1	0	0
Central	1	0	0	0
South	0	0	1	0

Now each region gets its own coefficient, free to take any value, and every pair of regions is the same distance apart: two rows from different regions differ in exactly two columns, so their Euclidean distance is $\sqrt{1^2 + 1^2} = \sqrt{2} \approx 1.41$ . No order has been invented. The price is width: five columns instead of one.

A variable with only two categories needs only one column. Lantern’s subscription flag becomes 1 for “yes” and 0 for “no”, and its channel becomes 1 for app and 0 for web. A second column would add nothing, because it would always equal 1 minus the first.

Ordered categories

Some categories do have an order. Lantern’s catalogue describes each coffee’s roast as light, medium or dark, which is an ordinal variable. Ordinal encoding numbers the categories in their true order, light 1, medium 2 and dark 3, and keeps them in one column. It preserves the order, but it also asserts something the labels don’t: that the step from light to medium is the same size as the step from medium to dark.

Many categories

Some variables have hundreds or thousands of categories: postcodes, product codes, travel-agent codes. This is high cardinality (the cardinality of a variable is the number of distinct values it takes). One-hot encoding would create thousands of columns, mostly zeros, with many categories seen only once or twice. The alternatives replace each category with a single number learned from the training data, such as how common the category is or the average outcome for that category. They are compact and often effective, and the second kind can leak the answer into the model if it is computed carelessly.

In depth

One-hot encoding in detail

A variable with $k$ categories becomes $k$ columns, and in every row they add up to exactly 1. Geometrically, each category sits at its own corner, and all the corners are $\sqrt{2}$ apart, which is why one-hot encoding suits distance-based methods. Most of the entries are 0, so software often stores the result in a sparse format that records only where the 1s are.

Two practical points. First, clean before you encode. The region column of the lantern-orders export spells regions several ways (different capitals, stray spaces, abbreviations, typing errors), and an encoder treats every variant as a separate category, so “North”, “north” and “North ” would each get a column. Consolidate the variants first, as in the text-and-categories lesson. Second, know your tool’s defaults. In pandas, pd.get_dummies(df, columns=['region'], dtype=int) keeps all $k$ columns unless you set drop_first=True, and without dtype=int it returns True/False columns. In scikit-learn, OneHotEncoder learns the categories when it is fitted and keeps all of them unless you set its drop option.

The dummy-variable trap

Most regression models include an intercept, which is a coefficient multiplied by a column of 1s. The $k$ one-hot columns of a variable also add up to 1 in every row, so together they exactly reproduce the intercept’s column. One column is a perfect combination of the others: perfect multicollinearity. This is the dummy-variable trap. The data can no longer decide between infinitely many sets of coefficients that make identical predictions, and least squares has no unique answer, because the matrix it must invert is singular.

An example makes it concrete. In a sample of off-season bookings, the fictional Harborview Hotel’s average nightly rates by room type are Standard $140, Deluxe $185, Family $210 and Suite $320. With room type as the only predictor, a regression predicts each type’s average rate. Both rows of the next table do that exactly, and so does any row in which a constant is added to the intercept and subtracted from all four room coefficients.

Two different coefficient sets, identical predictions (fictional)
Intercept	Standard	Deluxe	Family	Suite
0	140	185	210	320
100	40	85	110	220

Each prediction is the intercept plus the coefficient of the booking’s room type: 0 + 140 = 100 + 40 = 140 for a Standard room, and so on.

The cure is to drop one of the $k$ columns. The category without a column becomes the reference category (also called the baseline), and the $k - 1$ remaining columns are called dummy variables. The intercept is then the reference category’s prediction, and each dummy’s coefficient is the difference between its category and the reference.

Dummy coding of room type with two different reference categories (fictional)
Reference	Intercept	Standard	Deluxe	Family	Suite
Standard	140	—	+45	+70	+180
Deluxe	185	−45	—	+25	+135

A Suite is predicted at 140 + 180 = 185 + 135 = $320 either way.

Changing the reference changes every coefficient and no prediction. Choose a reference that makes the comparisons useful: the largest category, which gives the most stable comparisons, or a natural baseline such as the standard room, customers without a subscription, or the web channel. Statistics packages usually create dummy variables automatically and pick a default reference, often the first category in alphabetical order; check which one they picked and say so when you report coefficients.

When to drop a column, and when not to

Drop one (use $k - 1$ dummies) for an unpenalised linear or logistic regression with an intercept, the classic statistical model whose coefficients you want to read. With several categorical variables, drop one category from each.
Keep all $k$ for penalised models such as ridge and lasso regression. The penalty already makes the solution unique, and dropping a column makes one category special: scikit-learn’s documentation warns that it breaks the symmetry of the representation and can bias penalised linear models.
Keep all $k$ for tree-based models, which never invert a matrix. A benchmark study by Pargent and colleagues (2022) found one-hot encoding performed slightly better than dummy coding with nonlinear regularised models such as random forests and gradient boosting.
Keep all $k$ for distance-based methods. With $k - 1$ dummies the reference category sits at the origin, 1 unit from every other category, while the others stay $\sqrt{2}$ apart from each other: the reference becomes artificially close to everything.

The defaults reflect this: both OneHotEncoder and get_dummies keep every column unless you ask them not to. Dropping a column is a requirement of one family of models, not a rule of encoding.

Ordinal encoding and its assumption

Ordinal encoding keeps an ordered variable in one column of integers. In a linear model that column gets one coefficient, so the model assumes equal steps: light to medium moves the prediction exactly as much as medium to dark. If that is implausible, use one-hot encoding instead, which gives each level its own effect and ignores the order, or use numbers with real meaning. Lantern’s size_g column is already numeric (250, 480, 750 or 1,000 grams); recoding it as 1, 2, 3 and 4 would throw away the actual gaps between pack sizes.

The order must come from you. Software that isn’t told the order usually numbers categories alphabetically: scikit-learn’s LabelEncoder, for instance, numbers “paris”, “paris”, “tokyo” and “amsterdam” as amsterdam 0, paris 1 and tokyo 2. For roast that gives dark 0, light 1 and medium 2, which scrambles the order. In scikit-learn, pass the order explicitly: OrdinalEncoder(categories=[['light','medium','dark']]).

Label encoding of nominal variables

Label encoding gives a nominal variable arbitrary integer codes, as in the region example. For linear models it forces one straight-line effect along a meaningless order; for distance-based models it invents distances. Tree-based models are harmed less, because a tree can isolate a category with several splits, but each split can only separate codes above a threshold from codes below it, so useful groupings may need many splits. Pargent and colleagues argue that integer encoding is acceptable, if at all, only for tree-based models, and in their benchmark it did not perform well even with gradient boosting. scikit-learn’s documentation states that LabelEncoder is meant for encoding target labels, not input features.

Frequency encoding

Frequency encoding replaces each category with its share of the training rows, $n_c \div n$ . It needs one column whatever the number of categories, and it captures how common a category is, which is sometimes predictive in itself (popular products may be returned less often). Two categories with the same frequency become indistinguishable, and the shares must be computed on the training set and reused unchanged. A benchmark summarised by Pargent and colleagues found that it performed well in many cases.

Target encoding

Target encoding replaces each category with the average value of the target variable for that category in the training data. For a yes/no target coded 1 and 0, that average is the proportion of 1s. It also needs one column however many categories there are, and it puts the category’s relationship with the outcome directly into the feature. Micci-Barreca (2001) set out the method as a preprocessing step for high-cardinality variables, based on empirical Bayes estimation, and scikit-learn has provided it as TargetEncoder since version 1.3.

The danger is data leakage. Each training row’s own outcome is part of its category’s average, so the feature partly contains the answer. In the extreme, a category with a single training row is encoded as that row’s own outcome, and the model learns to trust a feature that is perfect in training and useless on new data. Three safeguards work together:

Smoothing. Blend each category’s average with the overall average, giving the category more weight the more rows it has: $S_c = \lambda_c \bar{y}_c + (1 - \lambda_c)\bar{y}$ with $\lambda_c = n_c \div (n_c + m)$ . A category with few rows is pulled towards the overall average; the smoothing strength $m$ is chosen by the analyst or estimated from the data.
Out-of-fold encoding (cross fitting). Split the training set into folds and encode each fold using averages computed from the other folds only, so that no row’s own outcome enters its encoding. scikit-learn’s TargetEncoder does this inside fit_transform, with five folds by default; its documentation notes that fit followed by transform does not use cross fitting and can leak.
Test and new data are encoded with averages learned from the whole training set, never from their own outcomes, which are unknown at prediction time anyway. A category never seen in training gets the overall training average.

Rare and unseen categories

A category with only a handful of training rows gives an unreliable coefficient or encoding. A common remedy is to group rare categories into a single “Other” category, using a threshold set on the training data, such as fewer than 10 rows. scikit-learn’s encoders can do this with the min_frequency and max_categories options, which combine infrequent categories into one output.

Categories also appear after a model is built: a new delivery region, a new product, a new degree programme. Decide in advance what happens. scikit-learn’s OneHotEncoder raises an error by default; with handle_unknown='ignore' it encodes the new category as all zeros; with handle_unknown='infrequent_if_exist' it puts it in the “Other” column if there is one. Beware the combination of dummy coding and “ignore”: all zeros is exactly how the reference category is encoded, so an unseen region would silently be treated as the reference region.

High cardinality

Postcodes, product codes and customer numbers can have thousands of values. One-hot encoding them creates thousands of sparse columns, most categories are rare, and new values keep arriving. The usual strategies, often combined, are:

Move up a level. Postcode becomes district or region; product code becomes product line. Micci-Barreca’s scheme can also blend statistics across levels of such a hierarchy, for example ZIP codes.
Group the rare values into “Other”, then one-hot encode what remains.
Use frequency or target encoding, fitted on the training set with smoothing and out-of-fold estimates.
Leave identifiers out. customer_id and order_id are keys that pick out a row (a primary key identifies each row), not descriptions of it. A model given customer_id can memorise individual customers and has nothing to say about a new one.

Missing categories

A blank category can be encoded as a category of its own, such as “Not stated”, when the fact that it is missing may carry information. pandas’ get_dummies adds a column for missing values only if you set dummy_na=True; otherwise a missing value becomes a row of zeros. scikit-learn’s TargetEncoder treats missing values as another category.

Fit encoders on the training set

Every encoder learns something from data: the list of categories, the reference category, the frequencies, the target averages. Like a scaler, it is fitted on the training set and applied unchanged to the test set and to new data. A category list taken from all the rows would quietly tell the training process about categories that exist only in the test set; target averages taken from all the rows put the test outcomes into the training features, which is leakage of the most damaging kind.

Formulas

Columns created by one-hot and dummy coding

p_{\text{one-hot}} = \sum_{j} k_j, \qquad p_{\text{dummy}} = \sum_{j} (k_j - 1)

$k_j$: Number of categories of categorical variable j (after cleaning)categories
$p$: Number of columns the encoded variables occupycolumns

Assumes. A two-category variable needs only one 0/1 column, which is what both formulas give for it when k − 1 = 1.

Use dummy coding (k − 1) for unpenalised regression with an intercept; keep all k otherwise.

Dummy coefficients with one categorical predictor

\hat{y} = b_0 + \sum_{c \neq \text{ref}} b_c D_c, \qquad b_0 = \bar{y}_{\text{ref}}, \qquad b_c = \bar{y}_c - \bar{y}_{\text{ref}}

$D_c$: Dummy variable: 1 if the row is in category c, 0 otherwise
$\bar{y}_{\text{ref}},\ \bar{y}_c$: Mean of the target in the reference category and in category cunits of y
$b_0,\ b_c$: Intercept and the coefficient of category cunits of y

Assumes. Least squares with an intercept and this categorical variable as the only predictor; with other predictors, each coefficient is the difference from the reference holding the others constant.

Frequency encoding

f_c = \dfrac{n_c}{n}

$n_c$: Training rows in category crows
$n$: All training rowsrows
$f_c$: Frequency encoding of category c: its share of the training rows

Computed on the training set and reused unchanged for test and new data.

Target encoding (category mean)

\bar{y}_c = \dfrac{1}{n_c}\sum_{i \in c} y_i

$y_i$: Target value of training row i (1 or 0 for a yes/no target)units of y
$n_c$: Training rows in category crows

Assumes. Computed out-of-fold for training rows; test rows use the full training averages.

Unsmoothed, it leaks each row’s own outcome and is unreliable for small categories.

Smoothed target encoding

S_c = \lambda_c\,\bar{y}_c + (1 - \lambda_c)\,\bar{y}, \qquad \lambda_c = \dfrac{n_c}{n_c + m}

$S_c$: Smoothed target encoding of category cunits of y
$\bar{y}_c$: Mean of the target in category cunits of y
$n_c$: Training rows in category crows
$\bar{y}$: Overall mean of the target in the training dataunits of y
$\lambda_c$: Weight on the category’s own mean, between 0 and 1
$m$: Smoothing strength: the larger it is, the more small categories are pulled towards the overall meanrows

Assumes. m is chosen by the analyst or estimated from the data (scikit-learn’s default estimates it).

Equivalent form: $S_c = (n_c \bar{y}_c + m\bar{y}) \div (n_c + m)$ .

Worked examples

A first example: Encoding a small order table

Fictional figures

Lantern’s reorder model will use four categorical variables from each order: region (Central, East, North, South, West), channel (web or app), subscription (yes or no) and roast (light, medium or dark). How should each be encoded, how many columns result, and how many if the model is an unpenalised logistic regression with an intercept?

Region is nominal with 5 categories, so one-hot encode it: 5 columns.
Channel and subscription each have 2 categories, so each becomes one 0/1 column (app = 1, subscription = 1): 2 columns.
Roast is ordinal and plausibly evenly spaced, so encode it in its true order (light 1, medium 2, dark 3): 1 column.
Total with every region column kept: $5 + 1 + 1 + 1 = 8$ columns.
For an unpenalised logistic regression with an intercept, drop one region column to avoid the dummy-variable trap, for example North if it is the most common region: $4 + 1 + 1 + 1 = 7$ columns.

Answer. Eight columns if all regions are kept; seven with one region as the reference category.

What it means. An order from the South, placed in the app, without a subscription, for a dark roast becomes the row (0, 0, 0, 1, 0 | 1 | 0 | 3) with all regions kept. Every number now means something the model can use, and no region is “bigger” than another.

A harder example: Reading dummy coefficients at the Harborview

Fictional figures

In a sample of off-season bookings, the Harborview Hotel’s average nightly rates by room type are Standard $140, Deluxe $185, Family $210 and Suite $320. An analyst regresses rate on room type with an intercept. (a) Why can’t she include all four room-type columns? (b) What are the coefficients with Standard as the reference? (c) What is the Suite coefficient if Deluxe is the reference, and what does the model predict for a Suite either way?

The four one-hot columns add to 1 in every row, which duplicates the intercept’s column of 1s. Coefficients (0; 140, 185, 210, 320) and (100; 40, 85, 110, 220) give identical predictions, so least squares can’t choose: the dummy-variable trap.
Drop Standard. The intercept is the Standard mean, 140, and each coefficient is a difference from it: Deluxe $185 - 140 = 45$ , Family $210 - 140 = 70$ , Suite $320 - 140 = 180$ .
With Deluxe as the reference, the intercept is 185 and the Suite coefficient is $320 - 185 = 135$ .
Predictions: $140 + 180 = 320$ and $185 + 135 = 320$ . The reference changes the coefficients, not the predictions.

Answer. (a) Perfect multicollinearity with the intercept. (b) Intercept $140; Deluxe +$45, Family +$70, Suite +$180. (c) Suite +$135 relative to Deluxe; a Suite is predicted at $320 either way.

What it means. Each dummy coefficient answers “how much more than the reference room?”. With Standard as the reference, the revenue manager reads directly that a Suite earns $180 a night more than a Standard room, which is the comparison she uses when deciding which rooms to upgrade.

A harder example: Target encoding style codes without leaking

Fictional figures

Ashgrove Clothing, a fictional online clothing brand, wants to predict whether an order will be returned (1) or kept (0) using the style code, which has hundreds of values. The table shows ten training orders, split into two folds. Compute (a) the plain target encoding of each style, (b) smoothed encodings with m = 2, (c) out-of-fold encodings, and (d) the encoding of a test order for style A and for a new style E.

Ten training orders (fictional)
Order	Style	Fold	Returned
1	A	1	1
2	A	1	1
3	B	1	0
4	C	1	0
5	D	1	1
6	A	2	0
7	A	2	1
8	B	2	0
9	B	2	1
10	C	2	0

Plain encodings (category means): A $3 \div 4 = 0.75$ , B $1 \div 3 \approx 0.33$ , C $0 \div 2 = 0$ , D $1 \div 1 = 1$ . The overall return rate is $5 \div 10 = 0.5$ . Order 5 is the only style D order, so its encoding is its own outcome: pure leakage.
Smoothed with $m = 2$ , using $S_c = (n_c\bar{y}_c + m\bar{y}) \div (n_c + m)$ : A $(3 + 1) \div 6 \approx 0.67$ , B $(1 + 1) \div 5 = 0.4$ , C $(0 + 1) \div 4 = 0.25$ , D $(1 + 1) \div 3 \approx 0.67$ . The single style D order is pulled from 1 halfway towards 0.5.
Out-of-fold, fold 1 is encoded with fold 2’s averages: A $1 \div 2 = 0.5$ (orders 1 and 2), B 0.5 (order 3), C 0 (order 4), and D, absent from fold 2, gets fold 2’s overall rate, $2 \div 5 = 0.4$ (order 5).
Fold 2 is encoded with fold 1’s averages: A $2 \div 2 = 1$ (orders 6 and 7), B 0 (orders 8 and 9), C 0 (order 10). No order’s own outcome is in its encoding; with only two tiny folds the values are noisy, which is why tools combine folds with smoothing.
Test orders use encodings learned from the whole training set: style A gets the smoothed 0.67. Style E never appeared in training, so it gets the overall training rate, 0.5.

Answer. (a) A 0.75, B 0.33, C 0, D 1. (b) A 0.67, B 0.4, C 0.25, D 0.67. (c) Orders 1–10: 0.5, 0.5, 0.5, 0, 0.4, 1, 1, 0, 0, 0. (d) Style A 0.67; new style E 0.5.

What it means. The plain encoding hands the model a feature that is perfect for order 5 and meaningless for new style D orders. Out-of-fold encoding removes each order’s own outcome from its feature, and smoothing stops tiny styles from claiming certainty. With hundreds of styles and thousands of orders, these safeguards are what make target encoding usable rather than self-deceiving.

Applied: a business decision: Planning the encodings for Northfield’s student model

Fictional figures

Northfield University wants a ridge regression to predict students’ GPA. Candidate inputs: programme (six programmes), year of study (1–4), residence (on campus, off campus, family home), scholarship (yes or no), home postcode district (several hundred values) and student_id. Choose an encoding for each, count the columns, and say what changes if the analyst switches to an unpenalised regression with an intercept to read the coefficients.

Programme: nominal, 6 categories, one-hot. Ridge has a penalty, so keep all 6 columns.
Year: ordered numbers already; keep it as one numeric column if a steady effect per year is plausible (check by comparing the average GPA by year), otherwise one-hot encode it.
Residence: nominal, 3 categories, one-hot: 3 columns. Scholarship: two categories, one 0/1 column.
Postcode district: high cardinality. Target-encode it out-of-fold with smoothing (1 column), or move up to a coarser area and one-hot encode that. student_id: an identifier, left out.
Ridge regression: $6 + 1 + 3 + 1 + 1 = 12$ columns. Unpenalised with an intercept: drop one category from programme and from residence, $5 + 1 + 2 + 1 + 1 = 10$ columns, and state the two reference categories.

Answer. Twelve columns for the ridge regression; ten for an unpenalised regression with reference categories for programme and residence. student_id is not used.

What it means. The encodings are design decisions that should be written down with the model: a reader of the coefficients needs to know the reference programme and residence, and a future analyst needs to know that the district encoding was fitted out-of-fold on the training cohort. When a seventh programme opens next year, the plan should already say what happens to it.

Explore

Common mistakes

“Numbers are numbers: code the categories 1, 2, 3.”
Arbitrary codes for a nominal variable invent an order and distances. A linear model is forced to give the categories one straight-line effect along that order, and a nearest-neighbour model treats alphabetically distant categories as dissimilar. Use one-hot encoding, or frequency or target encoding when there are many categories.
“Always drop one dummy column.”
Dropping a column is needed for unpenalised regression with an intercept. Penalised models, tree-based models and distance-based methods work better with all $k$ columns, and dropping one makes the reference category special. Drop deliberately, for the model that needs it.
“Target encoding is just a group average, so compute it on all the data.”
The average contains each row’s own outcome, and computed on all rows it contains the test outcomes too. Fit it on the training set, encode training rows out-of-fold, smooth small categories, and encode test and new rows with the training averages.
“Ordinal encoding is safe for anything with an order.”
It assumes equal steps between levels, and tools that aren’t told the order use alphabetical order: dark, light, medium. Supply the order explicitly, and switch to one-hot encoding or meaningful numbers when equal steps are implausible.
“Encode every column, including the IDs.”
Identifiers such as customer_id pick out rows; they don’t describe them. Encoded, they let a model memorise individual customers and say nothing about new ones. Leave them out, and treat other high-cardinality variables by grouping or with fitted frequency or target encodings.
“New categories won’t turn up.”
New regions, products and programmes appear all the time. Decide beforehand: raise an error, map them to “Other”, or use the overall training average for target encoding. With dummy coding, encoding an unknown category as all zeros silently treats it as the reference category.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading four questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading six questions…

In practice

Jun finishes his reorder model by writing down an encoding plan, because the model is only as reusable as that plan. Region is consolidated to its five valid values and one-hot encoded; channel and subscription become single 0/1 columns; roast is ordinal-encoded in the order light, medium, dark after a quick check that reorder rates move steadily across the three. For a penalised logistic regression used for prediction, he keeps all five region columns. When Ana asks “how much more likely are Western customers to reorder than Northern ones?”, he fits an unpenalised version with North as the reference category, so that each coefficient answers a question in that form. Everything is fitted on the training months and stored in the pipeline, including an “Other” rule for any new delivery region.

At the Harborview Hotel, a cancellation model uses the booking channel (five values), the guest’s country (nine countries once codes and names are made consistent, several with only a handful of bookings) and, for travel-agent bookings, an agent code with hundreds of values. Channel is one-hot encoded. Countries with fewer than a set number of training bookings are grouped into “Other” before one-hot encoding, so that a country with two bookings can’t produce a wild coefficient. The agent code is target-encoded on the cancellation outcome, out-of-fold and smoothed, and new agents get the overall training cancellation rate. The analyst also reports how many test bookings had an agent never seen in training, because those predictions rest on the overall rate alone.

At Northfield University, the planning office uses programme in a model that forecasts how many students will need extra tutoring. When a new degree programme is approved, the encoder would raise an error on its first students, or, with dummy coding and unknown categories ignored, silently treat them as the reference programme. The office therefore decides in advance that new programmes are encoded as “Other” until a full year of data exists, records that decision in the cleaning log, and refits the encoder and the model at the start of each academic year.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.

Loading six questions…

Summary

Most models need numbers; categorical encoding converts labels into numbers without inventing an order or distances the categories don’t have.
One-hot encoding creates one 0/1 column per category and keeps every pair of categories equally far apart; a two-category variable needs only one column.
With an intercept, all k one-hot columns are perfectly collinear (the dummy-variable trap); unpenalised regression therefore uses k − 1 dummies and a reference category, while penalised, tree-based and distance-based methods usually keep all k.
Ordinal encoding suits ordered categories but assumes equal steps and needs the order supplied explicitly; label encoding of nominal variables misleads linear and distance-based models.
Frequency and target encoding use one column for any number of categories; target encoding must be fitted on the training set, computed out-of-fold and smoothed to avoid leakage.
Group rare categories, plan for categories unseen at prediction time, and leave identifiers out of the model.

Key formulas

Columns created by one-hot and dummy coding
Dummy coefficients with one categorical predictor
Frequency encoding
Target encoding (category mean)
Smoothed target encoding

Terms introduced

Categorical encoding: Converting the labels of a categorical variable into numbers that a model can use.
One-hot encoding: Encoding a categorical variable with k categories as k columns of 0s and 1s, with a 1 in the column of each row’s category.
Dummy variable: A 0/1 column that marks whether a row belongs to one category. With dummy coding, a variable with k categories is represented by k − 1 dummy variables plus a reference category.
Dummy-variable trap: The perfect multicollinearity that arises when all k one-hot columns of a variable are used together with an intercept, so the coefficients can’t be uniquely estimated.
Reference category: The category left without a dummy variable. The intercept describes it, and each dummy coefficient is a difference from it. Also called the baseline.
Ordinal encoding: Encoding an ordered categorical variable as integers that follow its order, in one column. It assumes equal steps between levels.
Label encoding: Giving each category an arbitrary integer code, often in alphabetical order. Misleading for nominal input variables in linear and distance-based models.
Frequency encoding: Replacing each category with its share of the training rows.
Target encoding: Replacing each category with the average value of the target variable for that category in the training data, ideally smoothed and computed out-of-fold.
High cardinality: A categorical variable with a very large number of distinct values, such as postcodes or product codes.

Sources

scikit-learn developers. Preprocessing data, user guide section 8.3.4, “Encoding categorical features” (one-hot and ordinal encoding, infrequent categories, target encoder with cross fitting and smoothing). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/preprocessing.html
scikit-learn developers. OneHotEncoder, API reference (the drop and handle_unknown options); see also the OrdinalEncoder, LabelEncoder and TargetEncoder pages. scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/generated/sklearn.preprocess…
pandas development team. pandas.get_dummies, API reference. pandas 3.1.0 documentation. pandas.pydata.org/docs/reference/api/pandas.get_dummies.html
Minitab. Coding schemes for categorical predictors (1, 0 coding with a reference level). Minitab support. support.minitab.com/en-us/minitab/help-and-how-to/statistica…
Micci-Barreca, D. (2001). A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems. ACM SIGKDD Explorations Newsletter, 3(1), 27–32. doi.org/10.1145/507533.507538
Pargent, F., Pfisterer, F., Thomas, J. & Bischl, B. (2022). Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features. Computational Statistics, 37, 2671–2692. Free preprint: arXiv:2104.00629. arxiv.org/abs/2104.00629

Before you start

You will be able to

Review first

The idea

Numbers that invent an order

One-hot encoding

Ordered categories

Many categories

In depth

One-hot encoding in detail

The dummy-variable trap

When to drop a column, and when not to

Ordinal encoding and its assumption

Label encoding of nominal variables

Frequency encoding

Target encoding

Rare and unseen categories

High cardinality

Missing categories

Fit encoders on the training set

Formulas

Columns created by one-hot and dummy coding

Dummy coefficients with one categorical predictor

Frequency encoding

Target encoding (category mean)

Smoothed target encoding

Worked examples

Explore

Common mistakes

“Numbers are numbers: code the categories 1, 2, 3.”

“Always drop one dummy column.”

“Target encoding is just a group average, so compute it on all the data.”

“Ordinal encoding is safe for anything with an order.”

“Encode every column, including the IDs.”

“New categories won’t turn up.”

Guided practice

On your own

In practice

Knowledge check

Summary

Key formulas

Terms introduced

Sources