Sample spaces and counting

Two ordinary dice, rolled once, produce thirty-six equally likely pairs; a total of seven appears in six of those pairs, and a total of twelve appears in one. Nobody needs to roll the dice to determine this, because both counts follow from the dice themselves [S:5:686], and the entire calculation is a list followed by two divisions.

How many outcomes are there? You record every possible result exactly once, count the results inside the case you care about, and divide one count by the other. That procedure answers the section question and works whenever the listed outcomes are equally likely; most of this section is devoted to making the two counts honest. !Each cell is one ordered pair, so the lattice holds 36 outcomes. The six cells on the descending diagonal total seven, and the single cell in the corner totals twelve. The ringed (3,4) and (4,3) are two outcomes, not one, which is why the count uses 36 rather than 21. Status: schematic.

Write each result as an ordered pair, first die then second. The order matters: (3, 4) and (4, 3) are different rolls, because the first die showed a different number. The list runs (1, 1) through (1, 6) in the first row, (2, 1) through (2, 6) in the second, and so on to (6, 1) through (6, 6). That list of thirty-six ordered pairs is the sample space [S:5:694].

One pair is an outcome. "The total is seven" selects six pairs, and any selection of outcomes from the list is an event, which is to say a subset: any part of the list you choose to name [S:5:694]. In this elementary treatment the list is finite and nonempty; the general theory drops both restrictions.

The counting rule needs one more commitment: a finite sample space is uniform when every outcome carries the same belief, and since the thirty-six beliefs total one, each is 1/36. For an event $E$, the belief is the number of outcomes in $E$ divided by 36 [S:5:696]. An event is the union of its own outcomes — the collection of all outcomes it contains — so the sum rule adds the six beliefs of 1/36 that make up "the total is seven" [S:5:695].

$$P(\text{sum is }7)=\frac{6}{36}=\frac{1}{6}$$

That line states that six of the thirty-six pairs total seven, so the belief is six parts in thirty-six. [F01]

Worked example. The sample space contains every ordered pair, from (1, 1) through (6, 6). Write the first number for the first die and the second for the second die. The pairs that total seven are (1, 6), (2, 5), (3, 4), (4, 3), (5, 2) and (6, 1), which is six pairs; the only pair that totals twelve is (6, 6).

The row of counts sums to 36, which is the size of the sample space. The belief in a total of seven is 6/36, which equals 1/6 and approximately 0.167; the belief in a total of twelve is 1/36, which is approximately 0.028. Seven is therefore six times as likely as twelve, and the ratio of the two counts supplies that factor directly.

Events can be described in words and then translated into lists: the event that the total is at least ten contains (4, 6), (5, 5), (5, 6), (6, 4), (6, 5) and (6, 6). It contains six outcomes, so its belief is also 6/36, the same as the belief in a total of seven, although the two events share no outcome at all. The complement of an event is everything outside it, and its belief equals one minus the belief in the event; since only (1, 1) totals two, the belief that the total is not two is 35/36 [S:5:695].

The division also assumes uniformity, so it answers a question about fair dice rather than about any particular pair of dice on a table. The sample space fixes the possibilities, and the uniform assignment fixes their relative weights; the counting rule uses both commitments at once.

The pairs are ordered, and the order matters for the count; a 3 on the first die and a 4 on the second is a different outcome from a 4 followed by a 3, even though both total seven. Collapsing them into one result leaves 21 unordered pairs: the six doubles (1, 1) through (6, 6), plus fifteen pairs of two different numbers, because the thirty non-double ordered pairs pair up. Those 21 are not equally likely, so the counting rule would return an incorrect answer; the sample space has to record outcomes at the finest grain the experiment distinguishes.

A bag with three red and two blue counters makes the same point without dice: the sample space of one draw contains five outcomes, one per counter, and the belief in red is 3/5, which equals 0.6. If you record only the colour, the space contains two outcomes and they are not equally likely, so division by two would return 0.5 and lose the difference. The grain of the record determines which arithmetic is available.

Counting outcomes resembles counting seats in a theatre to find the share held by one section. The resemblance stops when the seats are not fixed: here you select what counts as one outcome, and a careless selection silently alters the denominator.

The idea has two costs. The outcomes must be equally likely: a loaded die, one weighted so that some faces are more likely than others, breaks the division; so does a biased coin, one that does not land heads half the time. Either sends you back to the rules of the previous section. The list must also be complete and correctly grained, and both requirements become difficult as the experiment grows: ten dice produce more than sixty million ordered outcomes, which no hand can enumerate.

The formal statement. A probability space has three parts: a set of outcomes, a collection of events, and a probability measure $P$ [S:26:37]. In the elementary treatment the outcome set is finite and nonempty, every subset is an event, and the measure obeys three axioms: no negative values, total mass one, and additivity on disjoint events [S:5:694][S:62:41]. In the general theory the outcome set may be any set, the events are the members of a $\sigma$-algebra rather than every subset, and the measure is countably additive [S:60:13]. In a finite space where every outcome carries the same weight, normalisation forces that weight to be $1/|S|$, and the axioms give

$$P(\text{event})=\frac{\text{outcomes inside the event}}{\text{outcomes in the space}}.$$

That line says that, in a uniform finite space, belief is a ratio of two counts [S:5:696].

The derivation. Uniformity gives every outcome the same weight. Normalisation forces that weight to be one divided by the number of outcomes, because the weights must sum to one. Additivity then sums the weights inside the event, and the common weight cancels between numerator and denominator. The counting is separate work. The multiplication principle supplies the denominator without listing, and inclusion-exclusion keeps the numerator honest [S:5:695].

The edge case. Let the outcome set be the real interval from zero to one, and let the event be "the number is at most one half". You cannot list the outcomes, and a single point carries weight zero. The ratio of counts is undefined, and probability becomes an integral of a density [S:62:40]. No countably infinite space is uniform in the counting sense either, because equal positive weights cannot sum to one; a uniform assignment exists only on a finite space. On an uncountable space not every subset can be given a consistent probability, so the events are restricted to a $\sigma$-algebra such as the Borel sets [S:62:40]. Countability is what lets the elementary treatment call every subset an event, and it is also what avoids set-theoretic trouble such as the Banach-Tarski paradox [S:5:694]. The instruction to list and divide fails here, and care in counting does not repair it.

What the core hid. The core said the sample space lists every possible outcome once. That holds for the dice and the counters, and fails in general. It was safe because every example had finitely many equally likely outcomes. The formal object is a measure; counting is the special case where the space is finite and the measure is uniform.

Where the analogy breaks. Counting seats in a theatre assumes that the seats are fixed and countable. Here you choose the event collection as part of the model, and an uncountable space contains subsets that cannot be given a consistent size [S:62:40]. The analogy also suggests that every subset has a count. The axioms promise no such thing.

Original sources. Jaynes quotes Laplace's definition: probability is the ratio of the number of favourable cases to the number of all equally possible cases [S:29:57]. He notes that James Bernoulli gave the same definition in 1713 [S:29:57]. The modern form is Kolmogorov's, stated as three axioms on a probability measure [S:62:41] and as a triple of sample space, event space and measure [S:26:37].

An open question. The counting definition needs a uniform assignment on a finite set. The field has not settled how to extend it to infinite spaces without arbitrariness. Jaynes argues that an infinite set carries no probability until you specify the limiting process that generates it from finite sets [S:29:13]. The measure-theoretic tradition instead fixes a $\sigma$-algebra and a countably additive measure [S:62:41]. Both avoid contradiction, and they can disagree on particular cases. Evidence that would settle it: a principled rule for choosing the limit, tested on a case where the two extensions return different numbers.

The counting side is a combinatorial object in its own right. Choosing five cards from a deck gives a sample space whose size a binomial coefficient reports, and the probability of a hand follows from one division [S:5:696]. The restriction to countable spaces in the elementary treatment also avoids set-theoretic trouble such as the Banach-Tarski paradox [S:5:694].

A frontier result. Monte Carlo integration computes an integral by sampling, and it is how large probabilistic models evaluate the sums and integrals that the axioms define [S:66:541]. It depends on the sample space and the measure, and it returns an answer with a random error rather than an exact count [S:66:541].

What to read next. Read Jaynes Chapter 2 for the finite-sets policy [S:29:57], Garrett Thomas Section 5.1 for the axioms in a few lines [S:26:37], and Murphy Chapter 2 for the continuous case and the $\sigma$-algebra [S:62:40].