Exact Sampling Distributions from Small Populations

Why Enumerate a Sampling Distribution?

A statistic changes from sample to sample, even when the population stays fixed. For a small population, we can see exactly how this happens: list every possible sample of a chosen size, calculate the statistic for each one, and record how often each value occurs. The resulting probability distribution is the exact sampling distribution for that statistic under the stated sampling plan.

Enumeration is more than a way to get an answer for a tiny example. It makes the probability model visible. In particular, it distinguishes the possible samples from the possible values of a statistic: different samples can give the same mean or proportion, and their probabilities must be combined.

Samples and Their Probabilities

Suppose a finite population has \(N\) distinct units, with a fixed value \(x_i\) recorded for unit \(i\). A simple random sample without replacement (SRSWOR) of size \(n\) selects \(n\) different units so that every subset of \(n\) population units is equally likely. The sample is treated as an unordered set: selecting units A and B is the same sample as selecting B and A.

The statistic values do not generally have equal probabilities. Equal probability applies to the sample subsets. If several subsets yield the same statistic value, that value receives the sum of their probabilities.

This equal-subset model follows from the definition of simple random sampling. If the \(n\) units are drawn one at a time without replacement, each ordered sequence of distinct units has probability \(1/[N(N-1)\cdots(N-n+1)]\). Any particular subset has exactly \(n!\) possible orders. Therefore every subset has the same probability, \(n!/[N(N-1)\cdots(N-n+1)]=1/\binom{N}{n}\).

Exact Distribution of a Sample Mean

For each possible sample subset \(S\), calculate the sample mean by adding its \(n\) recorded values and dividing by \(n\). The population values remain fixed throughout; the randomness is which units enter the sample. When different subsets have equal means, count each subset toward that mean’s probability.

Exact Distribution of a Sample Proportion

A sample proportion is a sample mean of zero-one indicators: assign 1 to a unit with the attribute of interest and 0 to a unit without it. If \(K\) sampled units have the attribute, then \(\hat{p}=K/n\). In a finite population with \(M\) units having the attribute, the population proportion is \(p=M/N\).

Combining Samples with the Same Statistic

A distribution table should have one row for each distinct statistic value, not necessarily one row for each sample. The next example includes repeated population values, which makes it especially important to keep track of distinct units even when their measurements match.

A Reliable Enumeration Procedure

For a small population, a systematic sequence of steps prevents missed samples and probability errors. Keep the population units identifiable, even when some recorded values are equal.

Why the Counting Rule Works

The rule \(P(T=t)=m_t/\binom{N}{n}\) follows directly from the sample space. There are \(\binom{N}{n}\) equally likely subsets, each with probability \(1/\binom{N}{n}\). The event \(T=t\) consists of exactly the \(m_t\) subsets that produce \(t\). Since these subsets are mutually exclusive, their probabilities add:

Every possible subset produces one statistic value, so grouping the subsets by their values partitions the full sample space. In symbols, the counts across all distinct \(t\) values add to \(\binom{N}{n}\); dividing by \(\binom{N}{n}\) shows that all probabilities add to 1. This is both a justification of the formula and a useful check on hand calculations.

Common Mistakes and Why Exact Distributions Matter

• Giving equal probability to distinct statistic values. A mean that occurs for one subset and a mean that occurs for three subsets do not have the same probability. Divide occurrence counts by the total number of subsets.
• Counting values instead of units. Two units with the same measurement are still distinct possible selections. Identify units while enumerating, then combine samples only when their statistic values match.
• Using ordered draws as separate samples. Under the SRSWOR definition used here, a subset is one sample regardless of the order in which its units might have been drawn. Counting orders is possible only if the probabilities are handled consistently.
• Treating sample proportions as population proportions. A sample proportion varies by sample; the population proportion \(p=M/N\) is fixed. The exact distribution gives probabilities for possible \(\hat{p}\) values, not uncertainty about which population value was recorded.
• Assuming the distribution has a familiar smooth shape. Enumeration gives the actual finite set of possible statistic values and their probabilities. It may be discrete, uneven, or concentrated at only a few values.

Exact enumeration is useful because it lets us inspect the sampling behavior without approximating it. It also provides a direct check on claims about possible statistic values and their probabilities. For larger populations, listing every subset may be impractical, but the same probability model remains the reference point for understanding what a sampling distribution describes.