Normalisation and standardisation · B02

Before you start

You will be able to

Explain why distance-based, penalised and gradient-based methods need features on comparable scales, and when scaling makes no difference.
Calculate min–max normalised, standardised and robust-scaled values, and convert scaled values back to the original units.
Compare how an extreme value affects each scaler and choose a suitable method for a given variable.
Distinguish the different meanings of “normalisation” and state precisely which transformation was used.
Apply scaling without data leakage: fit the parameters on the training set and apply them unchanged to the test set and to new data.
Interpret distances and regression coefficients computed from scaled variables.

Review first

Outliers and impossible values — You need z-scores, quartiles, the interquartile range and the idea of a robust statistic.
Skewed data and transformations — Scaling never changes the shape of a distribution; the log transformation from that lesson does, and it usually comes before scaling.
Calculating a mean, a median and a standard deviation for a short list of numbers.

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

Lantern Coffee Co.’s online-shop export covers orders from 1 March to 30 June 2026, and Ana Lim wants to send different emails to different kinds of customer. Lantern’s analyst, Jun Park, summarises each customer by two numbers for the period: total spend in dollars and the number of orders placed. A clustering method then groups customers whose two numbers are close together. The first run produces groups that differ only in spend: customers who order every fortnight land in the same group as customers who ordered twice. The method has not failed. The units have.

Many analytical methods decide how similar two observations are by measuring the distance between them. Plot each customer as a point, with spend on one axis and orders on the other. The usual measure is the straight-line or Euclidean distance: take the difference on each variable, square it, add the squares and take the square root. k-means clustering builds groups by putting each point with the nearest of k group centres, and k-nearest neighbours predicts something about a new observation from the k past observations closest to it. Both depend entirely on this distance.

Three Lantern customers, 1 March – 30 June 2026 (fictional)
Customer	Spend ($)	Orders
A	120	2
B	126	8
C	90	3

Customer B orders four times as often as A; C behaves much like A but spends $30 less. Measured in the raw units, the distance from A to B is $\sqrt{6^2 + 6^2} \approx 8.5$ and from A to C is $\sqrt{30^2 + 1^2} \approx 30.0$ , so B counts as A’s nearest neighbour. A gap of $6 weighs exactly as much as a gap of 6 orders, although $6 is a trivial difference in spend and 6 orders is a large difference in behaviour. Record spend in cents and the order counts would matter even less. The answer depends on the units somebody happened to choose, not on what matters to the business.

The fix is to measure each difference against the variable’s own typical variation. Suppose that across all of Lantern’s online customers in the period, spend has a standard deviation of $40 and orders a standard deviation of 2. Then A and B are $6 \div 40 = 0.15$ standard deviations apart on spend and $6 \div 2 = 3$ apart on orders, a distance of $\sqrt{0.15^2 + 3^2} \approx 3.00$ . A and C are 0.75 and 0.5 standard deviations apart, a distance of $\sqrt{0.75^2 + 0.5^2} \approx 0.90$ . On comparable scales, C is A’s nearest neighbour.

Converting numeric variables to comparable scales before they are used together is called feature scaling. (In machine learning, a feature is a variable used as an input to a model.) The scaled values have no units: dollars divided by dollars, or orders divided by orders, leave a pure number that says where a value sits relative to the others.

Three ways to rescale

Min–max normalisation: Subtract the smallest value and divide by the range (largest minus smallest). The smallest value becomes 0 and the largest becomes 1.
Standardisation: Subtract the mean and divide by the standard deviation. The result is the z-score you used to look for outliers: 0 is the mean and 1 is one standard deviation above it.
Robust scaling: Subtract the median and divide by the interquartile range (IQR). 0 is the median and 1 is one IQR above it.

All three are linear: they shift the values and stretch or shrink them by a constant. None changes the order of the values or the shape of their distribution. They differ only in which two numbers they use, a centre and a spread, and that choice decides how each one reacts to extreme values.

When scaling matters, and when it doesn’t

Scaling matters whenever a method lets variables with large numbers outweigh variables with small ones:

Distances. k-nearest neighbours, k-means and other clustering methods add up squared differences, so the variable with the widest numeric spread dominates. Principal component analysis, which looks for the directions of greatest variance, is affected in the same way.
Penalties. Ridge and lasso regression shrink coefficients towards zero, and support vector machines also penalise coefficient size. A coefficient’s size depends on its variable’s units (an effect per dollar is a hundred times smaller than the same effect per hundred dollars), so without scaling the penalty treats variables unequally for no business reason.
Gradient-based fitting. Neural networks and models fitted by stochastic gradient descent adjust coefficients in small steps; when features differ in scale by orders of magnitude, those steps make slow, uneven progress or fail to settle.
Comparing or combining variables measured in different units, for example an engagement index built from spend and order frequency.

Scaling makes essentially no difference to tree-based models (decision trees, random forests, gradient-boosted trees). A tree splits on one variable at a time at a threshold, such as “is spend above $95?”, and rescaling a variable doesn’t change which observations fall on each side. For ordinary least-squares regression without a penalty, rescaling changes the units of the coefficients but not the predictions or the fit. And in reports for people, keep the original units: a manager can act on “$140 in four months”, not on “0.85”.

In depth

Min–max normalisation

x' = \dfrac{x - x_{\min}}{x_{\max} - x_{\min}}

Here $x_{\min}$ and $x_{\max}$ are the smallest and largest values in the data the scaler is fitted on. Every fitted value lands between 0 and 1. For another range $[a, b]$ , such as −1 to 1, stretch the result: $x'' = a + (b - a)\,x'$ .

Two consequences follow from the formula. First, the bounds hold only for the fitted data. A new value above the training maximum scales above 1, and one below the minimum scales below 0. That is information, not a malfunction: it says the new value lies outside anything seen before. Some tools can clip such values back into range (scikit-learn’s MinMaxScaler has a clip option), but clipping hides how far outside the range a value lies. Second, the minimum and maximum are the two most extreme observations, so a single outlier sets the scale for everyone else.

Standardisation

z = \dfrac{x - \bar{x}}{s}, \qquad s = \sqrt{\dfrac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n - 1}}

After standardising, the fitted values have a mean of 0 and a standard deviation of 1. The formula is the z-score from the outliers lesson; there you computed it to flag unusual values, here you apply it to every value to create a new feature. Standardised values are unbounded: about two-thirds of the values of a roughly bell-shaped variable fall between −1 and 1, but nothing stops a value from scoring 6.

Standardising does not make a variable normally distributed. Because the standard normal distribution also has mean 0 and standard deviation 1, it is easy to assume otherwise, but a right-skewed variable stays exactly as right-skewed after standardising, with the same long tail. If the shape is the problem, transform first (for example with a log transformation) and scale afterwards.

Robust scaling

r = \dfrac{x - \tilde{x}}{Q_3 - Q_1}

Here $\tilde{x}$ is the median and $Q_1$ and $Q_3$ are the first and third quartiles, so the denominator is the IQR. Both the median and the IQR depend only on the middle of the data, so an extreme value barely moves them: they are robust statistics. scikit-learn’s RobustScaler uses exactly these two numbers by default.

Quartiles can be calculated in several ways, and the methods give slightly different answers on small datasets. This course and the site’s tools use linear interpolation between the ordered values, which is also NumPy’s default and Excel’s PERCENTILE.INC. Python’s built-in statistics.quantiles uses a different default, so even two Python functions can disagree. State the method when you report robust-scaled values.

Robust scaling doesn’t remove outliers or pull them in. It stops them from distorting everyone else’s scaled values, and it leaves the outliers themselves far from 0, where you can still see them.

What one extreme value does to each scaler

Take seven orders from one morning at Lantern’s online shop: $20, $22, $24, $25, $26, $28 and $30. Then an eighth arrives: $225, a genuine order from a café buying for its staff. The table shows what each scaler uses, and what happens to a typical $25 order, before and after the large order is included.

Scaling parameters for seven typical orders, then for all eight (fictional)
Scaler	Fitted on the seven	Fitted on all eight	A $25 order
Min–max	min 20, max 30	min 20, max 225	0.50 → 0.02
Standardisation	mean 25, SD 3.42	mean 50, SD 70.78	0 → −0.35
Robust	median 25, IQR 4	median 25.5, IQR 5	0 → −0.10

Sample standard deviations; quartiles by linear interpolation. Values rounded to two decimal places.

Min–max is hit hardest: the maximum jumps from 30 to 225, and all seven typical orders are squeezed into the narrow band from 0 to about 0.05, where a model can barely tell them apart. Standardisation shifts every typical order to between about −0.42 and −0.28, and something subtler happens to the large order itself: its z-score is only $(225 - 50) \div 70.78 \approx 2.47$ , below the common cut-off of 3, because the outlier has inflated the very standard deviation that is supposed to measure it. Robust scaling barely moves: the median shifts by 0.5 and the IQR by 1, the typical orders keep a spread from −1.1 to 0.9, and the large order scores $(225 - 25.5) \div 5 = 39.9$ , clearly visible as exceptional.

So the choice is a judgement about the variable. Min–max suits variables with natural, fixed bounds and no extreme values, such as percentages or ratings on a 1–5 scale. Standardisation is the usual default for roughly symmetric variables and for methods that expect centred inputs. Robust scaling suits variables with genuine extreme values that you have decided to keep.

Fit on the training set, apply everywhere

A predictive model is built on one part of the data and judged on another. The training set is the part used to fit the model and every preparation step; the test set is held back, untouched, to estimate how the model will perform on data it has never seen. The split only works if nothing from the test set influences the model. When it does, the result is data leakage:

“Data leakage occurs when information that would not be available at prediction time is used when building the model.”
scikit-learn, Common pitfalls and recommended practices

A scaler’s parameters (the minimum and maximum, the mean and standard deviation, or the median and IQR) are learned from data, just like a model’s coefficients. Computing them is called fitting the scaler. Fit it on all the rows before splitting, and the test rows help to shape the training features. Here is what that looks like for a small, fictional Northfield University model that uses students’ weekly study hours: six training students and two test students.

Weekly study hours standardised two ways (fictional)
Hours	Set	z, fitted on training (mean 10, SD 4)	z, fitted on all eight (mean 12, SD 7.33)
4	Training	−1.50	−1.09
8	Training	−0.50	−0.55
10	Training	0.00	−0.27
10	Training	0.00	−0.27
12	Training	0.50	0.00
16	Training	1.50	0.55
8	Test	−0.50	−0.55
28	Test	4.50	2.18

Sample standard deviations; z-scores rounded to two decimal places.

Two things go wrong in the right-hand column. The training features change because of students the model is supposed never to see: the most studious training student drops from 1.50 to 0.55. And the test student who studies 28 hours looks only moderately high (2.18) instead of far beyond anything in the training data (4.50). The test set no longer imitates the future, because when a real new student arrives, nobody can go back and re-fit the scaler that the model was trained with.

With large datasets, adding the test rows usually moves a mean or a standard deviation only a little, so the optimism caused by scaling alone is often small. The habit is what matters, because the same mistake in other preparation steps can be serious. In scikit-learn’s own demonstration, choosing features with the help of all the data produced an accuracy of 0.76 on labels that were pure random noise, where an honest workflow scored 0.5, the level of guessing. The safe workflow is the same for every data-dependent step:

Fix errors that don’t depend on other rows (wrong types, typing errors, duplicates).
Split the data into a training set and a test set.
Fit every data-dependent step on the training set only: imputation values, scaling parameters, encodings.
Apply the fitted steps, unchanged, to the training set and to the test set.
Store the fitted parameters and apply them unchanged to new data when the model is used.

Re-fitting on new data is the mirror-image mistake. If a batch of orders from a holiday promotion were standardised with that batch’s own mean, a $50 order that the model learned to treat as high would suddenly look average. The model expects inputs measured with the ruler it was trained on.

Clustering has no test set: Lantern’s segmentation simply groups the customers it has. There, fit the scaler on the customers being segmented. The rule about new data still applies: when a new customer arrives next month, standardise their numbers with the stored mean and standard deviation and compare them with the existing segment centres.

Converting back

Each scaler can be reversed by solving its formula for $x$ : $x = x_{\min} + x'(x_{\max} - x_{\min})$ , $x = \bar{x} + z\,s$ and $x = \tilde{x} + r(Q_3 - Q_1)$ . These inverse transforms turn results back into units people can use: a cluster centre at $z = 1.5$ on orders, with mean 4 and standard deviation 2, means $4 + 1.5 \times 2 = 7$ orders in the period. If a target variable was scaled before modelling, its predictions must be converted back the same way before anyone reads them.

Reading coefficients after standardising

In a linear regression with standardised predictors, each coefficient is the change in the prediction for a one-standard-deviation increase in that predictor, holding the others constant. Suppose a fictional Northfield regression predicts GPA from weekly study hours (standard deviation 6 hours) and attendance (standard deviation 12 percentage points), with standardised coefficients of 0.18 and 0.24. Per original unit, divide by the standard deviation: $0.18 \div 6 = 0.03$ GPA points per extra hour and $0.24 \div 12 = 0.02$ per percentage point. If the predictors were standardised with the training data, the least-squares intercept equals the training set’s mean GPA, the prediction for a student who is average on everything.

The two views rank the variables differently. Per unit, an hour of study is worth more than a point of attendance; per standard deviation, attendance is worth more, because attendance varies more in this group of students. Neither is a measure of importance in a causal sense: both describe associations in this sample, and both depend on how spread out each variable happens to be.

One word, several meanings

“Normalisation” is used for at least four different operations, and each changes the data differently:

Min–max scaling of a column to the range 0–1. This course uses the word only in this sense.
Scaling each observation to unit length. All the values in one row are divided by the row’s length, the square root of the sum of its squared values, so that every row has length 1. This works on rows, not columns; it is used when only the proportions within a row matter, as with word counts in documents. scikit-learn calls this “normalization”.
Transforming towards a normal distribution, with power transformations (Box–Cox, Yeo–Johnson) or quantile transformations. These change the shape of the distribution, which none of this lesson’s scalers do.
Z-score standardisation, which some authors and tools call “z-score normalisation”.

In database design the word means something else again: organising tables so that each fact is stored once. When someone says the data were “normalised”, ask which formula, and which rows it was fitted on.

Practical decisions

What to scale. Numeric features that enter a distance, a penalty or a gradient-based fit. Never scale identifiers such as customer_id: they are labels, not quantities. Columns of 0s and 1s that mark a category are usually left as they are (the next lesson builds them).
All features the same way? Standardising gives every variable equal weight per standard deviation. That is itself a choice. If the business decides that order frequency should count twice as much as spend in a segmentation, multiply its standardised values by 2, deliberately and on the record.
Constant columns. If every training value is the same, the range and standard deviation are 0 and the formulas divide by zero; scikit-learn’s StandardScaler leaves such a feature unscaled. Robust scaling also fails when the middle half of the values are identical, as with a column that is mostly zeros, because the IQR is then 0.
Order of operations. Correct errors, handle missing values, transform skewed variables, then scale, with every fitted step learned from the training set.

Formulas

Euclidean distance between two observations

d(\mathbf{p}, \mathbf{q}) = \sqrt{\sum_{j=1}^{k} (p_j - q_j)^2}

$d(\mathbf{p}, \mathbf{q})$: Straight-line distance between observations p and q
$p_j,\ q_j$: The two observations’ values on variable junits of variable j
$k$: The number of variables

Assumes. Every variable is measured on a comparable scale; otherwise the variable with the widest numeric spread dominates.

Used by k-nearest neighbours and k-means. Compute it on scaled values.

Min–max normalisation

x' = \dfrac{x - x_{\min}}{x_{\max} - x_{\min}}

$x$: The value to scaleoriginal units
$x_{\min},\ x_{\max}$: Smallest and largest values in the training dataoriginal units
$x'$: The scaled value: 0 at the training minimum, 1 at the training maximum

Assumes. The training range is representative; new values outside it scale below 0 or above 1.

For a range from a to b, use $a + (b - a)\,x'$ . Very sensitive to outliers.

Standardisation (z-score)

z = \dfrac{x - \bar{x}}{s}

$\bar{x}$: Mean of the training dataoriginal units
$s$: Standard deviation of the training data (sample formula, n − 1, in this course)original units
$z$: Standard deviations above (+) or below (−) the training mean

Assumes. The mean and standard deviation describe the bulk of the data; extreme values inflate both.

scikit-learn’s StandardScaler divides by n instead of n − 1; the difference is a constant factor of $\sqrt{n/(n-1)}$ .

Robust scaling

r = \dfrac{x - \tilde{x}}{Q_3 - Q_1}

$\tilde{x}$: Median of the training dataoriginal units
$Q_1,\ Q_3$: First and third quartiles of the training data (linear interpolation in this course)original units
$r$: Interquartile ranges above (+) or below (−) the training median

Assumes. The IQR is greater than zero.

Extreme values barely change the median and IQR, and they stay visible as large scaled values.

Converting scaled values back

x = x_{\min} + x'(x_{\max} - x_{\min}), \qquad x = \bar{x} + z\,s, \qquad x = \tilde{x} + r\,(Q_3 - Q_1)

$x$: The value in the original unitsoriginal units
$x',\ z,\ r$: A min–max, standardised or robust-scaled value, with the parameters it was fitted with
$x_{\min},\ x_{\max}$: Training minimum and maximum (min–max)original units
$\bar{x},\ s$: Training mean and standard deviation (standardisation)original units
$\tilde{x},\ Q_1,\ Q_3$: Training median and first and third quartiles (robust scaling)original units

Use the same stored parameters that produced the scaled value.

Coefficient per original unit from a standardised coefficient

b_j = \dfrac{b_j^{(z)}}{s_j}

$b_j^{(z)}$: Coefficient on the standardised predictor j: change in the prediction per standard deviation of the predictorunits of y
$s_j$: Standard deviation of predictor j used for the scalingunits of predictor j
$b_j$: Change in the prediction per one-unit increase in predictor junits of y per unit of predictor j

Assumes. Only the predictors were standardised, not the target.

Worked examples

A first example: One order, three scalers

Fictional figures

From its training data, Lantern’s analyst has stored these parameters for order value: minimum $12, maximum $60, mean $30, standard deviation $8, median $28 and IQR $10. Scale a $36 order with each method, then convert a standardised value of −1.25 back to dollars.

Min–max: subtract the minimum and divide by the range. $(36 - 12) \div (60 - 12) = 24 \div 48 = 0.5$ .
Standardisation: subtract the mean and divide by the standard deviation. $(36 - 30) \div 8 = 0.75$ .
Robust scaling: subtract the median and divide by the IQR. $(36 - 28) \div 10 = 0.8$ .
Inverse of standardisation: $x = \bar{x} + z\,s = 30 + (-1.25)(8) = 20$ .

Answer. Min–max 0.5, standardised 0.75, robust 0.8. A standardised value of −1.25 is a $20 order.

What it means. The three numbers describe the same order with three different rulers: it sits halfway along the training range, three-quarters of a standard deviation above the mean, and 0.8 IQRs above the median. None of them is in dollars, and none is “more correct”; what matters is using one scaler consistently, with the parameters fitted on the training data.

A harder example: An outlier that hides from its own z-score

Fictional figures

Kestrel Battery Installers, a fictional company that fits home battery-storage systems, records the hours each installation took. Nine jobs took 6, 7, 7, 8, 8, 9, 10, 11 and 42 hours; the 42-hour job was real, delayed by a roof repair. The analyst wants to scale installation time for a model and asks which scaler to use. Compare the scalers on the 42-hour job and on a typical 8-hour job.

Mean and sample standard deviation: the values add to 108, so $\bar{x} = 108 \div 9 = 12$ . The squared deviations add to 1,032, so $s = \sqrt{1{,}032 \div 8} = \sqrt{129} \approx 11.36$ .
Standardise the long job: $z = (42 - 12) \div 11.36 \approx 2.64$ . Below 3: the outlier has inflated the standard deviation that measures it, so a |z| > 3 rule would not flag it.
Median and quartiles by linear interpolation: with 9 ordered values, the quartile positions are $(9 - 1) \times 0.25 = 2$ and $(9 - 1) \times 0.75 = 6$ places after the first value, so $Q_1$ is the third value, 7, and $Q_3$ the seventh, 10. The median is the fifth value, 8, and the IQR is 3.
Robust-scale the long job: $r = (42 - 8) \div 3 \approx 11.33$ , far from the rest. A typical 8-hour job scores 0.
Min–max: the range is $42 - 6 = 36$ , so the 8-hour job scores $(8 - 6) \div 36 \approx 0.06$ . Eight of the nine jobs are squeezed into 0–0.14.

Answer. Standardisation gives the 42-hour job a z of about 2.64; robust scaling gives it about 11.33; min–max crushes the eight typical jobs into the bottom 14% of the range. Robust scaling is the best fit here.

What it means. The 42-hour job is genuine, so it stays in the data, but it shouldn’t decide the scale for every other job. Robust scaling keeps the typical jobs spread out and leaves the delayed job clearly visible, which is also what Kestrel’s operations manager would want to see.

A harder example: How fitting on all the data leaks

Fictional figures

A Northfield University analyst builds a model that uses weekly study hours. Six training students study 4, 8, 10, 10, 12 and 16 hours; two test students study 8 and 28 hours. Standardise the test student who studies 28 hours (a) correctly and (b) with the scaler fitted on all eight students, and say what the second version does to the training data.

Fit on the training set: mean $= 60 \div 6 = 10$ ; squared deviations $36 + 4 + 0 + 0 + 4 + 36 = 80$ ; $s = \sqrt{80 \div 5} = 4$ .
Apply to the test student: $z = (28 - 10) \div 4 = 4.5$ . Far above anything in training, which is exactly what the model will meet if such a student turns up.
Leaky fit on all eight: mean $= 96 \div 8 = 12$ ; squared deviations add to 376; $s = \sqrt{376 \div 7} \approx 7.33$ . The test student now scores $(28 - 12) \div 7.33 \approx 2.18$ .
Effect on the training data: the training student with 16 hours falls from $(16 - 10) \div 4 = 1.5$ to $(16 - 12) \div 7.33 \approx 0.55$ . The features the model learns from were reshaped by the test students.

Answer. Correctly fitted, the test student scores 4.5; with the leaky fit, 2.18. The leaky fit also changes every training value.

What it means. The correct version reports honestly that this student is far outside the training data, so the test result shows how the model copes with genuinely new cases. The leaky version quietly lets the test students set the ruler, so the evaluation is no longer a rehearsal for the future.

Applied: a business decision: Which segment should a new customer join?

Fictional figures

Lantern’s segmentation standardised spend (mean $100, SD $40) and orders (mean 4, SD 2) for 1 March – 30 June 2026 and found three segment centres, in standardised units: occasional (−0.6, −0.7), big basket (1.2, −0.2) and subscriber (0.5, 1.5). Customer F spent $150 over 7 orders. Describe the centres in dollars and orders for the marketing team, and assign F to a segment.

Convert the centres back with $x = \bar{x} + z\,s$ . Occasional: $100 - 0.6 \times 40 = 76$ dollars and $4 - 0.7 \times 2 = 2.6$ orders. Big basket: $148 and 3.6 orders. Subscriber: $120 and 7 orders.
Standardise F with the stored parameters, not refitted ones: spend $(150 - 100) \div 40 = 1.25$ , orders $(7 - 4) \div 2 = 1.5$ .
Distances in standardised units: to occasional $\sqrt{1.85^2 + 2.2^2} \approx 2.87$ ; to big basket $\sqrt{0.05^2 + 1.7^2} \approx 1.70$ ; to subscriber $\sqrt{0.75^2 + 0^2} = 0.75$ .
For comparison, raw distances: to big basket $\sqrt{2^2 + 3.4^2} \approx 3.94$ , to subscriber $\sqrt{30^2 + 0^2} = 30$ . Unscaled, the $30 spend gap to the subscriber centre outweighs everything, while the 3.4-order gap to the big-basket centre counts for almost nothing, so F would be called a big-basket buyer.

Answer. Centres: occasional $76 and 2.6 orders, big basket $148 and 3.6 orders, subscriber $120 and 7 orders. F is closest to the subscriber centre (distance 0.75).

What it means. F orders as often as Lantern’s subscribers and spends somewhat more, so F should get the subscription-retention emails rather than the bulk-discount offers. The result rests on a stated assumption: standardising treats one standard deviation of spend and one of orders as equally important. If Ana weighted spend more heavily, the assignment could change, and that would be a business decision to record, not a technical accident.

Explore

Common mistakes

“Standardising makes the data normal.”
Standardising subtracts a constant and divides by a constant, so the shape of the distribution doesn’t change at all: a right-skewed variable stays right-skewed, with mean 0 and standard deviation 1. If the shape matters, transform it first (for example with a log) and then scale.
“Scale the whole dataset first; it’s only arithmetic.”
The parameters of a scaler are learned from data. Fitted on all rows, they carry information from the test set into the training features, and the test result becomes optimistic. Split first, fit on the training set, then apply the same parameters to the test set and to new data.
“A min–max value above 1 is an error.”
Min–max values are bounded only for the data the scaler was fitted on. A new order larger than any training order scales above 1, and that tells you something useful: the model is being asked about territory it hasn’t seen. Don’t silently clip it, and don’t re-fit the scaler on the new data; check whether the inputs have shifted.
“Every model needs scaled inputs.”
Distance-based, penalised and gradient-based methods do. Tree-based models split one variable at a time at a threshold and are essentially unaffected, and ordinary least squares without a penalty gives the same predictions whatever the units. Scale because the method needs it, not by reflex.
“The bigger standardised coefficient is the more important cause.”
A standardised coefficient is the change in the prediction per standard deviation of the predictor in this sample. It depends on how spread out the variable happens to be and, like any regression coefficient, describes an association, not a cause. Report coefficients in original units as well, and say what was held constant.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading four questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading six questions…

In practice

Lantern’s segmentation project shows the whole sequence. Jun first fixes the export’s errors, then builds two customer features for 1 March – 30 June 2026: total spend and number of orders. He checks the shape of each first: had a few bulk-buying cafés stretched spend into a long right tail, he would take logs before scaling. He standardises both features using the customers being segmented, stores the four parameters (two means, two standard deviations) with the model, and runs k-means. For Ana’s marketing meeting he converts every segment centre back to dollars and orders, because “subscriber: about $120 over 7 orders” is something a marketer can act on and “(0.5, 1.5)” is not. When July’s new customers arrive, they are standardised with the stored parameters and assigned to the nearest centre; the segmentation itself is re-run, and re-fitted, only at the next planned review.

At the Harborview Hotel, a revenue manager uses a k-nearest-neighbours model to estimate how likely a booking is to be cancelled, from the lead time in days, the nightly rate in dollars and the number of nights. Unscaled, the two variables that run into the hundreds (the rate in dollars and the lead time in days) would decide almost every neighbour, and the number of nights, from 1 to 14, would hardly count. The analyst splits by time, training on the earlier months and testing on the most recent ones, which imitates how the model will be used. The scaler is fitted on the training months only and saved inside the model pipeline. Each week she also checks how many new bookings fall outside the training range (min–max values above 1, or very large z-scores): a rising share means the inputs have drifted, and the right response is to retrain and re-validate the whole pipeline on a new training period, not to quietly re-fit the scaler.

At Northfield University, an analyst presents a regression of GPA on study hours and attendance to the student-success committee. The standardised coefficients let the committee compare variables measured in hours and in percentage points, but the analyst also reports them per original unit (0.03 GPA points per weekly hour of study, 0.02 per percentage point of attendance) and states plainly that these are associations in one cohort, not the effect of making a student study more. Scaling made the comparison possible; it didn’t make it causal.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.

Loading six questions…

Summary

Feature scaling puts numeric variables on comparable, unitless scales so that none dominates a distance, a penalty or a gradient-based fit just because of its units.
Min–max normalisation maps the training range to 0–1; standardisation gives mean 0 and standard deviation 1; robust scaling centres on the median and divides by the IQR.
All three are linear: they keep the order and the shape of the data. Standardising does not make a variable normal.
An extreme value squeezes min–max values and inflates the standard deviation; robust scaling keeps typical values spread out and the extreme value visible.
Fit scaling parameters on the training set only and apply them unchanged to the test set and to new data; fitting on all the data is data leakage.
Tree-based models and unpenalised least squares don’t need scaling; reports for people should use original units, using inverse transforms where needed.
“Normalisation” has several meanings (min–max, unit-length rows, transformation towards normality, z-scores): always state the formula and the data it was fitted on.

Key formulas

Euclidean distance between two observations
Min–max normalisation
Standardisation (z-score)
Robust scaling
Converting scaled values back
Coefficient per original unit from a standardised coefficient

Terms introduced

Feature scaling: Converting numeric variables to comparable, unitless scales before they are used together in an analysis or model.
Min–max normalisation: Rescaling a variable as (x − minimum) ÷ (maximum − minimum), so the training data run from 0 to 1.
Standardisation: Rescaling a variable as (x − mean) ÷ standard deviation, so the training data have mean 0 and standard deviation 1. The result is a z-score.
Robust scaling: Rescaling a variable as (x − median) ÷ interquartile range, so that extreme values have little influence on the scale.
Data leakage: Using information when building a model that would not be available when the model makes real predictions, such as test-set values used to fit a preparation step. It makes performance estimates too optimistic.
Training set: The part of the data used to fit a model and every data-dependent preparation step, such as scaling parameters and encodings.
Test set: Data held back from all fitting and used only to estimate how a model will perform on data it has not seen.

Sources

scikit-learn developers. Preprocessing data, user guide section 8.3 (standardisation, scaling to a range, scaling data with outliers, non-linear transformation, normalisation). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/preprocessing.html
scikit-learn developers. StandardScaler, API reference (notes on the biased standard-deviation estimator). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/generated/sklearn.preprocess…
scikit-learn developers. MinMaxScaler, API reference (formula, feature range and the clip option). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/generated/sklearn.preprocess…
scikit-learn developers. RobustScaler, API reference (median and interquartile-range scaling). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/modules/generated/sklearn.preprocess…
scikit-learn developers. Common pitfalls and recommended practices, user guide section 12 (inconsistent preprocessing and data leakage). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/common_pitfalls.html
scikit-learn developers. Importance of Feature Scaling, example gallery (k-nearest neighbours and PCA with and without standardisation; tree-based models almost unaffected). scikit-learn 1.9.1 documentation. scikit-learn.org/stable/auto_examples/preprocessing/plot_sca…

Before you start

You will be able to

Review first

The idea

Three ways to rescale

When scaling matters, and when it doesn’t

In depth

Min–max normalisation

Standardisation

Robust scaling

What one extreme value does to each scaler

Fit on the training set, apply everywhere

Converting back

Reading coefficients after standardising

One word, several meanings

Practical decisions

Formulas

Euclidean distance between two observations

Min–max normalisation

Standardisation (z-score)

Robust scaling

Converting scaled values back

Coefficient per original unit from a standardised coefficient

Worked examples

Explore

Common mistakes

“Standardising makes the data normal.”

“Scale the whole dataset first; it’s only arithmetic.”

“A min–max value above 1 is an error.”

“Every model needs scaled inputs.”

“The bigger standardised coefficient is the more important cause.”

Guided practice

On your own

In practice

Knowledge check

Summary

Key formulas

Terms introduced

Sources