Probability

In depth

The rules of probability

Every way of assigning probabilities must obey the same rules. For any event, $0 \leq P(A) \leq 1$, and the whole sample space has probability 1. The complement rule follows: $P(A') = 1 - P(A)$. The addition rule gives “or”, the union $A \cup B$: $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$. Here “and” is the intersection $A \cap B$, the outcomes in both events, and its probability is 0 for mutually exclusive events.

Conditional probability is defined from the joint and marginal probabilities, $P(A \mid B) = P(A \text{ and } B) \div P(B)$, for any $B$ with $P(B) > 0$. Rearranged, it is the multiplication rule: $P(A \text{ and } B) = P(B) \times P(A \mid B)$. With the table, $P(\text{app and sub}) = 0.429 \times 0.576 \approx 0.247$, the same joint probability as before.

For independent events the multiplication rule simplifies to $P(A \text{ and } B) = P(A) \times P(B)$. Three checks are equivalent, and any one of them shows independence: $P(A \mid B) = P(A)$, $P(B \mid A) = P(B)$, or $P(A \text{ and } B) = P(A)P(B)$. For channel and subscription, $P(\text{app})P(\text{sub}) = 0.429 \times 0.450 \approx 0.193$, against the joint probability of 0.247.

Probabilities in Excel

A PivotTable gives the counts, and B03’s % of Grand Total, % of Row Total and % of Column Total turn them into joint and conditional probabilities. The same numbers come from formulas on the order sheet, with channel in column K and subscription in column L: =COUNTIFS(K2:K232,"app",L2:L232,"yes")/COUNT(J2:J232) is the joint probability 0.247, and =COUNTIFS(K2:K232,"app",L2:L232,"yes")/COUNTIF(K2:K232,"app") is the conditional probability 0.576. The pattern is always the same: the count of orders that meet both conditions, divided by the count of orders that meet the condition after the bar (or by all orders, for a joint probability).

Formulas have two advantages over a PivotTable for this work. They update when the data change, and they make the condition visible: anyone reading COUNTIF(K2:K232,"app") in the denominator can see that the probability is conditional on an app order. When a report quotes a percentage, the first check is the denominator, because a share of all orders, a share of app orders and a share of subscription orders are three different probabilities.

Tree diagrams

A tree diagram shows events in sequence. Each branch carries the probability of an event given everything to its left, so the first branches are marginal probabilities and later ones conditional. Multiplying along a path gives the joint probability of that path (the multiplication rule), and the paths’ probabilities add to 1.

Adding the paths that end in “subscription” recovers the marginal probability: $0.203 + 0.247 = 0.450$. This is the law of total probability: $P(A) = P(B)P(A \mid B) + P(B')P(A \mid B')$. It is the denominator of Bayes’ rule.

Sampling with and without replacement

Draw two spring orders at random. If the first is put back before the second is drawn (with replacement), the draws are independent and $P(\text{both subscriptions}) = (104 \div 231)^2 \approx 0.2027$. If it isn’t (without replacement), the second draw depends on the first: $104 \div 231 \times 103 \div 230 \approx 0.2016$. Surveys and audits sample without replacement, but when the sample is a small part of the population the two answers are almost the same. That is why the next lesson can treat a few draws from a large population as independent; OpenStax’s rule of thumb is a sample of fewer than 10% of the population.

Bayes’ rule

Tillwise screens every card payment and flags some as possible fraud. Suppose (all rates fictional) that 0.4% of orders are fraudulent, the base rate; that the screen flags 95% of fraudulent orders; and that it also flags 2% of genuine ones, its false positive rate. If an order is flagged, what is the probability it is fraudulent? The screen’s 95% is $P(\text{flag} \mid \text{fraud})$; the question asks for $P(\text{fraud} \mid \text{flag})$. Bayes’ rule turns one into the other:

Only about 16% of flagged orders are fraudulent. Natural frequencies make the reason visible. Of 10,000 orders, 40 are fraudulent and 38 of them are flagged; of the 9,960 genuine orders, 2%, about 199, are flagged too. So 38 of about 237 flagged orders are fraud. When the base rate is low, even a small false-positive rate produces more false alarms than real cases.

Confusing $P(\text{flag} \mid \text{fraud})$ with $P(\text{fraud} \mid \text{flag})$ is sometimes called the confusion of the inverse; it usually comes from ignoring the base rate (base-rate neglect), and it lies behind many bad decisions about screening tests, alarms and audits. The reverse question gives a reassuring answer: an order that is not flagged is fraudulent with probability about 0.0002, two in ten thousand. A screen like this is good at clearing orders and poor at convicting them.

Counting outcomes: COMBIN and PERMUT

Classical probability needs the number of equally likely outcomes, and counting them is often the hard part. The number of ways to choose $k$ items from $n$ when order doesn’t matter is the number of combinations, $\binom{n}{k} = n! \div (k!\,(n - k)!)$, Excel’s =COMBIN(n,k); when order matters it is the number of permutations, $n! \div (n - k)!$, =PERMUT(n,k). Here $n!$ (“n factorial”) is $n \times (n - 1) \times \dots \times 1$.

Ana could draw 5,247,527,901 different samples of 5 orders from the 231: =COMBIN(231,5). A simple random sample makes each of them equally likely. Choosing 3 of 5 trainees for a photo can be done in =COMBIN(5,3) = 10 ways, but lining 3 of them up in order in =PERMUT(5,3) = 60 ways, because each group of 3 can be arranged in $3! = 6$ orders. Combinations return in the next lesson, where they count the ways a binomial outcome can happen.