Why Enumerate a Sampling Distribution?
A statistic is a function of the observed sample. Its sampling distribution describes which values that function can take and the probability of each value before the sample is observed. For a small discrete model, the most direct way to find this distribution is to list every possible ordered sample, calculate the statistic for each one, and combine the probabilities of samples that produce the same result.
Keeping the samples ordered matters whenever the statistic treats sample positions differently. For example, a statistic involving the first observation need not have the same value when the first and second observations are reversed. Even when a statistic does not depend on order, ordered enumeration provides a reliable way to count how many outcomes produce each value.
Set Up the Ordered Sample Space
Suppose the observations are independent and identically distributed (iid), and each can take one of \(k\) equally likely values. For sample size \(n\), there are \(k^n\) possible ordered samples. Independence gives each ordered sample probability \(1/k^n\). Thus, if \(m_t\) ordered samples produce \(T=t\), then
This equal-probability shortcut applies only when the sampling model makes all ordered samples equally likely. In particular, it applies to iid draws with replacement from a finite set of equally likely values. If population values have unequal probabilities, ordered samples still need to be listed, but each sample receives the product of its observation probabilities. The probabilities of samples producing the same statistic value are then added.
The sample space is made of ordered values, so repeated values are allowed in sampling with replacement. For example, \((a,b)\) and \((b,a)\) are two distinct outcomes, and \((a,a)\) is also possible. Do not replace the ordered sample space with combinations unless the sampling design and the statistic specifically justify doing so.
Specify the distribution of each observation, the sample size, and whether observations are independent. Determine whether ordered samples are equally likely.
Include every possible sequence, including repeats when sampling is with replacement.
Apply its stated formula to each ordered sample without rearranging the observations.
Add the probabilities of all ordered samples that produce each value of the statistic.
Confirm that the probabilities are nonnegative and sum to one, and that no possible statistic value has been omitted.
Worked Enumerations
Worked Example: A Statistic That Depends on Sample Order
A hypothetical two-reading device records iid readings \(X_1,X_2\), each equally likely to be \(1\) or \(4\). Define the statistic \(T=|X_1-X_2|+I(X_1=4)\), where \(I(X_1=4)\) is 1 if the first reading is 4 and 0 otherwise. Find the exact distribution of \(T\), and calculate \(P(T\geq 3)\).
Method. Enumerate the ordered pairs. There are \(2^2=4\) pairs, each with probability \(1/4=0.2500\), because the readings are independent and each value has probability \(1/2\).
| Ordered sample \((X_1,X_2)\) | \(|X_1-X_2|\) | \(I(X_1=4)\) | \(T\) |
|---|---|---|---|
| \((1,1)\) | 0 | 0 | 0 |
| \((1,4)\) | 3 | 0 | 3 |
| \((4,1)\) | 3 | 1 | 4 |
| \((4,4)\) | 0 | 1 | 1 |
Each of the four statistic values occurs once, so \(P(T=0)=P(T=1)=P(T=3)=P(T=4)=1/4=0.2500\). The requested probability is \(P(T\geq 3)=P(T=3)+P(T=4)=0.2500+0.2500=0.5000\).
Check. The four probabilities sum to \(4(0.2500)=1.0000\), and the two outcomes meeting the threshold give \(2/4=0.5000\). A common mistake is to treat \((1,4)\) and \((4,1)\) as interchangeable: the indicator uses the first position, so these outcomes give different values, 3 and 4.
Worked Example: A Nonlinear Statistic With Repeated Values
A hypothetical monitoring system produces three iid readings, each equally likely to be 1 or 2. For the ordered readings \(X_1,X_2,X_3\), define \(T=X_1X_2+X_3\). Find the exact sampling distribution and \(P(T\geq 4)\).
Method. There are \(2^3=8\) equally likely ordered triples, each with probability \(1/8=0.1250\). Evaluate the product of the first two readings, then add the third reading.
| Ordered sample | \(X_1X_2\) | \(X_3\) | \(T\) |
|---|---|---|---|
| \((1,1,1)\) | 1 | 1 | 2 |
| \((1,1,2)\) | 1 | 2 | 3 |
| \((1,2,1)\) | 2 | 1 | 3 |
| \((1,2,2)\) | 2 | 2 | 4 |
| \((2,1,1)\) | 2 | 1 | 3 |
| \((2,1,2)\) | 2 | 2 | 4 |
| \((2,2,1)\) | 4 | 1 | 5 |
| \((2,2,2)\) | 4 | 2 | 6 |
The value 2 occurs once, 3 occurs three times, 4 occurs twice, and 5 and 6 each occur once. Therefore, \(P(T=2)=1/8=0.1250\), \(P(T=3)=3/8=0.3750\), \(P(T=4)=2/8=0.2500\), and \(P(T=5)=P(T=6)=1/8=0.1250\).
For the requested tail probability, the outcomes with \(T\geq4\) are \((1,2,2)\), \((2,1,2)\), \((2,2,1)\), and \((2,2,2)\). Hence \(P(T\geq4)=4/8=0.5000\). The probabilities check because \((1+3+2+1+1)/8=8/8=1\). A common mistake is to assign a separate probability to each distinct value of \(T\); values such as 3 and 4 occur more than once, so their probabilities must reflect their frequencies.
Worked Example: Unequal Probabilities in an Ordered Sample
A hypothetical sensor records iid values from \(\{0,1,2\}\), with \(P(X_i=0)=1/2\), \(P(X_i=1)=1/3\), and \(P(X_i=2)=1/6\). For a sample of size two, let \(T=|X_1-X_2|\). Find the exact distribution of \(T\).
Method. The nine ordered pairs are possible, but they are not equally likely. Independence means the probability of \((x_1,x_2)\) is \(P(X_1=x_1)P(X_2=x_2)\). List the pairs and group by absolute difference.
| Ordered pair(s) | Statistic value | Total probability |
|---|---|---|
| \((0,0),(1,1),(2,2)\) | 0 | \((1/2)^2+(1/3)^2+(1/6)^2=14/36=0.3889\) |
| \((0,1),(1,0),(1,2),(2,1)\) | 1 | \(2(1/2)(1/3)+2(1/3)(1/6)=4/9=0.4444\) |
| \((0,2),(2,0)\) | 2 | \(2(1/2)(1/6)=1/6=0.1667\) |
Thus \(P(T=0)=14/36=0.3889\), \(P(T=1)=4/9=0.4444\), and \(P(T=2)=1/6=0.1667\). The exact probabilities sum to \(14/36+16/36+6/36=36/36=1\).
The calculation uses the independence of \(X_1\) and \(X_2\). For example, each of \((0,1)\) and \((1,0)\) has probability \((1/2)(1/3)=1/6\), so together they contribute \(1/3\) to \(P(T=1)\). A common mistake is to count all nine ordered pairs as equally likely and assign \(1/9\) to each. That would contradict the stated marginal probabilities; equal counting is valid only when the ordered outcomes really are equiprobable.
Why Grouping the Outcomes Works
For any possible statistic value \(t\), the event \(T=t\) consists exactly of the sample outcomes that the function \(g\) maps to \(t\). These outcomes are mutually exclusive, so the addition rule for probabilities gives the exact mass at \(t\).
If all \(k^n\) ordered samples have probability \(1/k^n\), each term in this sum is the same. There are \(m_t\) terms, which gives \(P(T=t)=m_t/k^n\). This also explains why ties in the statistic must be grouped: each tied sample is a distinct way for the event \(T=t\) to occur, even though the statistic has the same value for all of them.
Common Traps and a Reliable Check
- Dropping sample order. Keep \((x_1,x_2)\) distinct from \((x_2,x_1)\) until you have evaluated the statistic. They may have different statistic values or probabilities.
- Assuming equal probabilities without checking. Equal counting requires equally likely ordered samples. If observation probabilities differ, multiply the probabilities within each ordered sample and then add.
- Counting statistic values instead of sample outcomes. The probability of a statistic value depends on how many, or how much probability, of the sample space maps to it.
- Leaving out repeats. Under sampling with replacement, a sample can contain the same population value more than once.
- Failing to verify the total. The probabilities across all distinct values of a statistic must sum to 1. A total below or above 1 signals a missed outcome or a counting error.
Exact enumeration is most useful when the sample space is small enough to list completely. It makes the model assumptions visible and provides a direct check on how the statistic behaves, including ties and order effects. For a larger sample space, a formula or a more efficient probability method may be preferable, but the same principle remains: identify the sample outcomes corresponding to each statistic value and add their probabilities.
The ability to construct and check these distributions is aligned with the SOA VEE Mathematical Statistics syllabus. The central discipline is simple but important: define the sample space from the model, evaluate the statistic on each ordered outcome, and use the correct probability for every outcome.
Check Your Understanding
For each problem, use the stated sampling model, preserve the order of observations, and group outcomes that give the same statistic value.
- Let \(X_1,X_2\) be iid and equally likely to be 2 or 5. Find the distribution of \(T=X_1+X_2\). (Answer: \(P(T=4)=0.2500,\ P(T=7)=0.5000,\ P(T=10)=0.2500\).)
- Let \(X_1,X_2\) be iid and equally likely to be 0, 1, or 3. Find the distribution of \(T=|X_1-X_2|\). (Answer: \(P(T=0)=0.3333,\ P(T=1)=0.2222,\ P(T=2)=0.2222,\ P(T=3)=0.2222\).)
- Let \(X_1,X_2,X_3\) be iid and equally likely to be 1 or 2. Find the distribution of \(T=X_1+X_2X_3\). (Answer: \(P(T=2)=0.1250,\ P(T=3)=0.3750,\ P(T=4)=0.2500,\ P(T=5)=0.1250,\ P(T=6)=0.1250\).)
- Let \(X_1,X_2\) be iid with probabilities \(P(X_i=-1)=1/2\), \(P(X_i=0)=1/4\), and \(P(X_i=2)=1/4\). For \(T=I(X_1=X_2)\), find \(P(T=0)\) and \(P(T=1)\). (Answer: \(P(T=0)=0.6250,\ P(T=1)=0.3750\).)