Bayes' rule

A screening test for a rare cancer is right 90% of the time when the disease is present, a sensitivity of 0.90, and wrong 5% of the time when the disease is absent, a false-positive rate of 0.05. One woman in a hundred in the clinic has the disease, so the prevalence, the share who have it before testing, is 0.01 [S:5:713]. Her test comes back positive, and the belief that she has the disease is about 15 in 100, not 90 [S:5:715]. The counting below shows where the 15 comes from.

How do you turn a likelihood into a belief? A hypothesis is a candidate explanation; here, that she has the disease. The likelihood is the belief the evidence would receive if the hypothesis were true. You weigh it against how common the hypothesis was before the test, and against every other way the same result could have arisen. The likelihood states how well the hypothesis predicts the evidence; it does not state how probable the hypothesis is. !Each square is one woman in a clinic of 10,000. The 100 sick women occupy the first row, and 90 of them test positive. Of the 9,900 healthy women, 495 also test positive. The 585 positive results are therefore mostly false, and the share who are sick is 90/585, about 0.154. Status: synthetic.

The two quantities are not interchangeable. A likelihood of 0.90 says that nine in ten sick women test positive, and it says nothing about the share of positive women who are sick. The previous section showed that the two conditional beliefs can differ by a wide margin, and Bayes' rule is the accounting that converts one into the other. Write P for a belief, so P(positive | cancer) is the likelihood and P(cancer | positive) is the posterior, the belief after the evidence.

One woman in a hundred has the disease before the test, so the prior is 0.01. The test catches ninety of every hundred sick women, so the likelihood is 0.90. The posterior, the share of positive women who are sick, is what Bayes' rule returns from those two numbers and the overall probability of a positive result. That overall probability is found by adding the routes that produce one [S:5:716].

$$P(\text{cancer}\mid \text{positive})=\frac{P(\text{positive}\mid \text{cancer})P(\text{cancer})}{P(\text{positive})}$$

That line says: multiply the likelihood by the prior, then divide by the overall rate of positive results. [F01]

Worked example. Imagine ten thousand women in the clinic, which keeps the arithmetic in whole people [S:5:715]. One in a hundred has the disease, so one hundred women are sick, and the test finds ninety of them. On the same hundred, the test misses ten sick women, the share that the sensitivity leaves out.

The other nine thousand nine hundred are healthy, and 5% of them receive a false positive, a positive result in someone without the disease. That is four hundred and ninety-five women. The positive results therefore number 90 + 495 = 585. The posterior is the share of positive results that belong to sick women: 90/585, which is approximately 0.154.

About fifteen in a hundred women with a positive result have the disease, and about eighty-five in a hundred do not. The denominator is the count of positive tests rather than the count of sick women, and that substitution is the whole content of the rule. A larger clinic with the same rates returns the same share, because the factor of ten cancels between numerator and denominator.

The formula returns the same number as the counts. The numerator is 0.90 × 0.01 = 0.009. The denominator collects both routes to a positive result: 0.90 × 0.01 + 0.05 × 0.99 = 0.009 + 0.0495 = 0.0585. Dividing 0.009 by 0.0585 gives 0.1538, which matches 90/585.

The same rule admits an odds form, which some people find quicker. Odds compare two beliefs as a ratio, so odds of 1 to 99 mean one case in favour for every ninety-nine against. A prevalence of 0.01 is one case in a hundred, which gives prior odds of 1 to 99.

The likelihood ratio is 0.90/0.05, which equals 18. Multiplying gives posterior odds of 18 to 99, and 18/117 is 0.1538, because 117 is 18 + 99. The odds form makes the two ingredients visible: one factor for what you believed, one factor for what the evidence did.

The test behaves like a sieve with two openings. It catches most of the sick women, and it also catches a share of the healthy ones. When healthy women vastly outnumber sick women, the second catch supplies most of what lands in the pan [S:5:715]. The comparison stops at the mesh, because a sieve keeps the same holes everywhere while a test can behave differently in a different clinic.

The result is not a defect of the test. The ninety sick women who test positive are real, and so are the four hundred and ninety-five healthy women who test positive. The test reports a positive result, and the clinic supplies the prevalence; only the two together determine the posterior. A report that quotes the sensitivity alone has left out the ingredient that dominates the answer.

Accuracy alone can mislead in the other direction as well. A test that always returns negative is right for every healthy woman and wrong only for the 1% who are sick, so its accuracy, the share of all results that are correct, is about 99%, and it detects nothing [S:5:715]. A test that answers no to everything scores well and knows nothing. A single accuracy figure hides both error rates and the prevalence, and the rule needs all three.

The idea has a cost. The posterior inherits the prior, so a wrong prevalence yields a wrong belief however accurate the test is. The error rates themselves are measured in some population, and they can shift when the test moves to another one. The next section isolates the prior and shows how far it can move an answer.

The formal statement. Let the hypotheses form a finite list that is mutually exclusive and exhaustive, and let the evidence have positive probability. Then

$$P(\text{hypothesis}\mid \text{evidence})=\frac{P(\text{evidence}\mid \text{hypothesis})P(\text{hypothesis})}{P(\text{evidence})}.$$

That line says: multiply the likelihood by the prior, then divide by the overall probability of the evidence [S:62:42]. The denominator expands, by the law of total probability, into the sum of the same product over every hypothesis in the list [S:62:42].

The derivation. Two lines. The product rule writes the joint belief in hypothesis and evidence in two orders, and the two orders must agree. Divide both sides by the probability of the evidence. That is the rule. The only extra step is the denominator: the evidence can arise through any hypothesis, so its probability is the sum of the routes. Bayes' rule is a theorem about conditional probabilities, not a new axiom.

The odds form. Divide the rule for one hypothesis by the rule for a second. The common denominator cancels, and the posterior odds equal the prior odds multiplied by the likelihood ratio. The denominator therefore normalises; it does not decide the comparison between two hypotheses.

The edge case. A continuous parameter. The prior is a density, the denominator is an integral, and any single value has probability zero [S:62:42]. The rule still works with densities, but the phrase "probability of the hypothesis" no longer names an event. A second edge is the base rate. A test that always returns negative scores about 99% on a population with 1% prevalence and detects nothing [S:5:715]. The gap between accuracy and posterior can be wide: a p-value of 0.05 can sit beside a posterior of 0.30 for the null [S:63:232].

What the core hid. The core treated the denominator as a count of positive tests. That is right for a finite list and one test. It hides that the denominator is a normalising constant, fixed so that the posterior beliefs sum to one over the list. It also hides that the prior may be a density, and that choosing it is a modelling decision rather than a consequence of the rule.

Where the analogy breaks. A sieve keeps the same mesh everywhere. A test does not, because its sensitivity and false-positive rate can change with the population it meets. The analogy also suggests a passive catch. Here you choose the list of hypotheses, and a hypothesis you omitted cannot be recovered by any denominator.

Original sources. Downey derives the rule from the commutativity of conjunction, then applies it to the cookie problem [S:40:21]. The two ways of writing the joint belief carry the whole derivation. The textbook statement with the partition in the denominator is in Murphy [S:62:42]. The medical-test version with whole-person counts is in Mathematics for Computer Science [S:5:715].

An open question. Where the prior comes from. Murphy states the question directly: priors refer to parameters that are not observable [S:62:106]. De Finetti's theorem answers it for infinitely exchangeable sequences, because such a sequence is a mixture of independent draws conditional on a hidden parameter, so a prior follows from exchangeability [S:62:106]. Finite exchangeability alone does not give that representation. Whether your sequence is infinitely exchangeable is an empirical question. Evidence that would settle a dispute about a prior: a test of exchangeability on the data, or a sensitivity analysis showing that the posterior barely moves across the priors that survive the test [S:32:48].

A frontier result. Bayesian inference at scale is the engine of probabilistic artificial intelligence, where a posterior is computed or approximated for models with many parameters [S:60:12]. Calibration is the practical check: among the cases where a model announces 0.8, about 80% should be positive [S:35:316]. Rubin's summary is the working standard: Bayesian in principle and calibrated in practice [S:63:234].

The rule composes. Apply it once per observation, using each posterior as the next prior. The final belief does not depend on the order of the observations, because each order is a different factorisation of the same joint belief [S:40:21]. The likelihood principle separates the readings: Bayesian inference depends on the data only through the likelihood, while frequentist procedures can depend on outcomes that did not occur [S:63:232].

What to read next. Read Downey Chapter 1 for the derivation [S:40:21], Murphy Section 2.1.6 for the partition form [S:62:42], and Gelman and colleagues for the applied workflow and prior sensitivity [S:32:48].