Sunflower Intelligence
Learn

Plate 11 of 19 · Business analytics

Classification: odds, thresholds and the confusion matrix

About 7 minutes to read, then three questions that count toward your mastery.

A churn model gives each customer a probability. Turning that into a decision means choosing a threshold, and every threshold trades missed churners against false alarms.

The idea

Logistic regression keeps predictions between 0 and 1 by modelling the log-odds as a straight line. Raise e to a coefficient and you get an odds ratio: the factor by which one more unit multiplies the odds of the outcome, holding the other predictors fixed. Odds ratios multiply odds, not probabilities.

To act, choose a threshold and flag everyone above it. The confusion matrix counts the four results. Precision is the share of flagged customers who really churn; recall is the share of churners you flag. Lower the threshold and recall rises while precision falls. The ROC curve traces every threshold, and its area (AUC) is the chance a random churner scores above a random stayer: 0.5 is a coin flip, 1.0 perfect.

Clustering has no outcome column. k-means groups rows by distance, so put the columns on a common scale first (standardise each to mean 0, standard deviation 1), or whichever column has the bigger numbers decides the groups.

The formula

log p1 − p = b0 + b1x, p = 11 + e−(b0 + b1x), odds ratio = eb1, precision = TPTP + FP, recall = TPTP + FN

p
the predicted probability of the outcome (here, churn)
p ÷ (1 − p)
the odds: 0.25 probability is odds of 1 to 3
eb1
the odds ratio for one more unit of x
TP, FP, FN, TN
true positives (flagged churners), false positives (flagged stayers), false negatives (missed churners), true negatives

A worked example

Plate 11

A churn model has log-odds = −2.0 + 0.405 × support calls. Of 1,000 customers, 200 churn.

  1. Odds ratio for a call: e0.405 = 1.50. Each extra call multiplies the odds of churning by 1.5.
  2. A customer with no calls: p = 1 ÷ (1 + e2.0) = 11.9%. With two calls the log-odds are −2.0 + 0.81 = −1.19, so p = 23.3%.
  3. Flag everyone at 50% or more: 120 churners caught, 80 stayers flagged by mistake, 80 churners missed. Precision = 120 ÷ 200 = 60%; recall = 120 ÷ 200 = 60%.
  4. Lower the threshold to 20%: 170 caught, 230 false alarms, 30 missed. Recall rises to 170 ÷ 200 = 85%; precision falls to 170 ÷ 400 = 42.5%.

Neither threshold is “right”. If a missed churner costs much more than a retention call, the 20% threshold is worth its 230 false alarms; if calls are expensive, 50% may be better.

Try it

400 customers, 100 of whom churn. Separation sets how much higher churners score than stayers.

1.5
50%

Move the threshold

51%of churners flagged (recall)
Precision: flagged who really churn
71%
Customers flagged
72 (21 false alarms)
Accuracy
83%
AUC (every threshold)
0.86

A middle setting: compare the cost of a false alarm with the cost of a missed churner to choose.

The common mistake

Reading an odds ratio as a ratio of probabilities

An odds ratio of 1.5 does not make churn 50% more likely. Two calls multiply the odds by 1.5² = 2.25, but the probability goes from 11.9% to 23.3%, only 1.96 times as high. The gap widens as probabilities grow: near certainty, the odds can keep multiplying while the probability barely moves.

Remember

  • ecoefficient is an odds ratio: it multiplies the odds per unit, holding the other predictors fixed.
  • Lowering the threshold raises recall and lowers precision. AUC summarises every threshold: 0.5 is chance.
  • Standardise columns before k-means, or the one with the biggest numbers dominates the distances.

Check yourself

Three questions from the Learn room’s bank. Your answers update your mastery of this skill, the same as in Practise.

Use it on real numbers

Open Logistic regression in depth, in Business analytics › Supervised learning and try the idea on your own numbers.