Uncertainty as extended logic

Tomorrow's forecast reads a 30% chance of rain, a 45% chance of wind, and a 15% chance of both. Your bag has room for an umbrella or a gate strap, but not for both. Each number is a degree of belief: a value between zero and one saying how strongly you hold a proposition, which is a claim that is either true or false. The three beliefs are tied together because they describe one table of four cells: the two single-event beliefs and their overlap fix the remaining cells, so you cannot change one without changing at least one other.

Three rules do the work. Beliefs in cases that cannot occur together add. The beliefs in all the cases together total one. And the belief in two propositions together equals the belief in one times the belief in the other given the first. The rest of this section shows why those rules are forced rather than chosen; the first two are enough to fill a table. !Rain occupies 0.30 of the unit interval and wind 0.45. The two intervals overlap by 0.15, so adding the beliefs counts that stretch twice; the union runs to 0.60 and the remainder is 0.40. The position of the intervals is chosen for display, and only the lengths carry the numbers. Status: schematic.

Worked example. Start with the table. Two questions sort tomorrow: rain or no rain, and wind or no wind. The four cells are rain with wind, rain without wind, no rain with wind, and neither.

Rain with wind is given as 0.15. Rain without wind is 0.30 minus 0.15, which equals 0.15. No rain with wind is 0.45 minus 0.15, which equals 0.30. The last cell takes the remainder of 1.00 minus 0.60, which equals 0.40.

The four cells total 1.00, the rain row totals 0.30, and the wind column totals 0.45. Those are the three original numbers, recovered from the table rather than assumed again. The belief in rain or wind is 0.15 + 0.15 + 0.30 = 0.60, and the belief in neither is the remaining 0.40. A two-by-two table is four cells sorted by two yes-or-no questions, and filling it is the sum rule doing the bookkeeping.

Forecasts overlap, so addition alone is insufficient. Rain and wind may arrive together, and a plain sum would count the overlap twice. The correction is one subtraction.

$$P(\text{rain or wind})=P(\text{rain})+P(\text{wind})-P(\text{rain and wind})$$

That line says: add the two beliefs, then subtract the part counted twice. The overlap is the cell where rain meets wind, and it belongs to both totals. [F01]

The consistency requirement also supplies a way to check your arithmetic. Add the three cells that contain rain or wind, then compute 1.00 minus the belief in neither; both routes must return 0.60. When the two answers disagree, the disagreement locates the faulty cell rather than merely announcing an error.

Why should a table be forced at all? Edwin Jaynes, the physicist whose book this section follows, states five conditions in two groups [S:29:32]. The bracketed tags name source pages, and Appendix F resolves each one to a title; a tag such as [F01] beside an equation points to that line's derivation.

Three conditions are structural. Beliefs are real numbers. A stronger premise never lowers the belief in a consequence, which he calls qualitative correspondence with common sense [S:29:33]. And every route to the same conclusion returns the same value, which he calls consistency [S:29:33]. Two are interfaces between the rules and the information: all relevant evidence is used, and equivalent states of knowledge receive equivalent beliefs [S:29:33], which is what makes an assignment depend on the information rather than on the person holding it.

The consistency condition is the strongest of the three. If two routes of reasoning begin from the same evidence, they must end at the same number, and a rule that permitted two answers would be discarded [S:29:33]. The interface condition on equivalent knowledge carries that requirement across reasoners: two people with the same evidence and the same starting beliefs receive the same assignment [S:29:33].

Under the five conditions, continuity in each argument, and additional denseness conditions on the propositions, the rules are unique up to a choice of scale [S:29:34][S:42:114]. Continuity alone does not close the argument; the denseness conditions are a separate hypothesis, not a technicality [S:42:114]. That is why the arithmetic is a consequence rather than a convention.

The number is written P for short, so P(rain) denotes your belief that it rains tomorrow. Zero represents impossibility and one represents certainty [S:29:45]. A probability is the same object with the rules attached, and this chapter treats the two expressions as synonyms. The terminology is not a confession of bias: it names the same number seen with its rules attached, and the rules do not choose between starting beliefs [S:29:58].

The sum rule concerns alternatives. When two events cannot occur together, the belief in either one or the other equals the sum of their separate beliefs [S:5:695], and the rule generalises to any collection of mutually exclusive cases. It also fixes the belief in the negation, since the two beliefs must total one.

Logic is the special case in which every entry equals zero or one. If you are certain that all cats are mammals, your belief that a particular animal is both a cat and not a mammal is zero. The rules then reproduce the truth table of ordinary logic: the table that lists, for each combination of true and false inputs, whether a compound statement is true. Jaynes therefore calls the result probability theory as extended logic [S:29:13]. The same rules carry deduction into cases where the premises leave you short of certainty.

The idea has a price. The rules never ask where 0.30 originated, so a consistent table built from poor inputs stays consistent and stays wrong. Jaynes also treats the real-number representation as an assumption about the reasoner rather than a discovery about minds [S:29:35]. A reasoner who reported a range of values, such as "between 0.3 and 0.5", instead of one number would fall outside Cox's theorem, the uniqueness result above, and the rules themselves cannot tell you which reasoner deserves your trust. That is the honest cost of a calculus of belief: it disciplines the relations among your numbers without ever auditing the numbers themselves.

The formal statement. Take a set of propositions closed under "and" and "not", and fix a background condition $C$. Assign each proposition a real number, its belief given $C$. Jaynes states five conditions in two groups [S:29:32]. The structural conditions: beliefs are real numbers; a greater belief in a premise never lowers the belief in a consequence, which he calls qualitative correspondence with common sense [S:29:33]; and every route to the same conclusion returns the same value, which he calls consistency [S:29:33]. The interface conditions: all relevant evidence is taken into account, and equivalent states of knowledge receive equivalent beliefs [S:29:33]. The last condition is what forces two reasoners with the same information to the same number. He calls the conditions desiderata rather than axioms, because they state goals and not truths [S:29:32].

The theorem says that, under the desiderata, continuity in each argument, and additional denseness conditions on the propositions, the assignment is isomorphic to a probability function [S:42:114]. With the background condition fixed, the product and sum rules read

$$P(A\cap B\mid C)=P(A\mid B\cap C)P(B\mid C)$$

and

$$P(A\cup B\mid C)=P(A\mid C)+P(B\mid C)-P(A\cap B\mid C).$$

The first line says that belief in a conjunction, given $C$, is belief in one part times belief in the other given the first and $C$. The second says that beliefs in two alternatives add, with the overlap subtracted once. The unconditional forms in the previous section are the case where $C$ is everything you are taking for granted.

The derivation. Require that two ways of combining three propositions agree. Jaynes carries out that algebra and reports that the resulting rules are unique [S:29:34]. I omit the solution of the functional equation that appears there. That step needs a continuity or differentiability hypothesis, and it is the one place where the argument is longer than it looks. Continuity alone does not close the derivation: Paris exhibited a loophole in Cox's argument, and the denseness assumptions are what close it [S:42:114]. They are a separate hypothesis, not a technicality.

The edge case. Give a reasoner interval-valued beliefs, a range such as 0.3 to 0.5, instead of single numbers. The structural and interface conditions can still hold, and the two rules do not apply. That reasoner is not incoherent; it sits outside Cox's theorem. Jaynes treats the real-number representation as an assumption about the reasoner rather than a discovery about minds [S:29:35]. A second edge is size: the consistency theorem covers finite sets of propositions, and infinite sets need a limiting argument [S:29:57].

What the core hid. The core said the conditions leave exactly one family of rules, that is, one recipe for combining beliefs apart from the unit you measure them in. They leave one set of rules once you fix the scale, and fixing the scale is a convention. The core also did not name continuity, denseness and finiteness among the hypotheses. It was safe there because every example used two or three propositions with three outcomes.

Where the analogy breaks. "Extended logic" suggests that probability is logic with fractional truth values. In logic, every pair of propositions is comparable and every connective has a truth table. Here, comparability across unrelated propositions is an extra assumption, and a truth table appears only in the zero-one case [S:29:13].

Original sources. Richard Cox produced the derivation, and Jaynes builds his system on it [S:60:11]. Jaynes states the desiderata in Chapter 1 and deduces the rules in Chapter 2 [S:29:32]. Murphy states the modern reading: the Kolmogorov axioms can be shown to follow from a more basic set of assumptions about reasoning under uncertainty, which is Cox's theorem [S:62:41]. The derivation needs more than the desiderata: Paris found a loophole, closed by additional denseness assumptions [S:42:114]. The betting alternative is the Dutch book theorem, which says that beliefs violating the rules lose for sure against a clever opponent [S:63:233].

An open question. The field has not settled how far the argument reaches. Jaynes proves consistency for finite sets of propositions only, and treats infinite sets through limits [S:29:57]. De Finetti keeps finite additivity and rejects some countable extensions, and Jaynes records his disagreement with that treatment [S:29:13]. Which extension is correct is a convention about idealisation, not a theorem inside the calculus. Evidence that would settle it: a representation theorem deriving one extension from assumptions both sides accept, or a betting scheme that separates the two.

A frontier result. Probabilistic artificial intelligence treats probability as the logic of uncertainty and builds inference algorithms on that reading [S:60:11]. The same rules underlie probabilistic programming, where a program denotes a distribution over its outputs. The frontier difficulty is computational rather than foundational: the rules say what to believe, and an algorithm must approximate the answer within a budget [S:60:12].

The two justifications are independent. One asks whether your reasoning is consistent, and the other asks whether a bettor can drain your wallet [S:63:233]. A reader who rejects the continuity or denseness assumptions in the first argument can still accept the second, which is why the Dutch book survives as a separate line of defence.

What to read next. Read Jaynes Chapters 1 and 2 for the desiderata and the deduction [S:29:32], then the axioms in Garrett Thomas for the measure-theoretic form [S:26:37] and in Murphy for the same form with commentary [S:62:41]. For the decision-theoretic justification, read the Dutch book discussion in Murphy [S:63:233].