From One Sample to a Distribution of Means
A sample mean is one number calculated from one sample. But before the sample is drawn, the observations can vary, so the mean can vary too. To understand how informative a sample mean may be, ask not only what value it has in one sample, but also what values it could have across repeated samples and how likely each value is.
Imagine repeatedly drawing samples of the same size according to the same sampling plan, calculating the mean each time. The collection of possible means, together with their probabilities, is the sampling distribution of the sample mean. This is a probability model for a statistic; it is not the distribution of individual observations in the population.
The notation separates the random quantity from the result actually observed. Before sampling, \(\bar{X}\) represents the not-yet-known sample mean. After a particular sample has been observed, its calculated mean is written \(\bar{x}\). For example, \(\bar{X}\) might have possible values 2, 4, and 6; if one sample produces a mean of 4, then the observed value is \(\bar{x}=4\).
To specify a sampling distribution, we need a sampling model: a description of how observations are selected and the probabilities of the possible sample outcomes. In the examples below, sampling is with replacement from a stated population distribution. Thus each draw has the same probabilities, and the draws are independent. A different sampling plan can produce a different distribution of sample means, even when it concerns the same population.
How Sample Outcomes Produce Mean Probabilities
A sample mean is calculated from a sample outcome. If several distinct outcomes lead to the same mean, the probability of that mean is found by adding the probabilities of those outcomes. This addition is valid because the outcomes are mutually exclusive: in a single sample, only one of them occurs.
If the draws are independent, the probability of a particular ordered sequence of draws is the product of the probabilities of its individual draws.
Keeping the outcomes ordered is a reliable way to avoid missing or miscounting samples. For instance, drawing 1 and then 5 differs as an ordered outcome from drawing 5 and then 1, even though both samples have the same mean. Their probabilities must both be included when finding the probability of that mean.
Worked Example: Two Equally Likely Population Values
Suppose a hypothetical population consists of values 2 and 6, each with probability \(1/2\). Draw \(n=2\) observations independently with replacement. Find the sampling distribution of \(\bar{X}\).
Model check. The population probabilities are specified, and the problem states that the two draws are independent and made with replacement. The ordered sample outcomes are therefore \((2,2)\), \((2,6)\), \((6,2)\), and \((6,6)\), each with probability \((1/2)(1/2)=1/4\).
Calculate the mean for each outcome:
| Ordered sample | Sample mean | Probability of sample |
|---|---|---|
| (2, 2) | 2 | 1/4 |
| (2, 6) | 4 | 1/4 |
| (6, 2) | 4 | 1/4 |
| (6, 6) | 6 | 1/4 |
The mean 4 occurs for two sample outcomes, so its probability is \(1/4+1/4=1/2\). The means 2 and 6 each occur for one outcome, so each has probability \(1/4\). Thus the sampling distribution is:
| Possible value of \(\bar{X}\) | Probability |
|---|---|
| 2 | 1/4 |
| 4 | 1/2 |
| 6 | 1/4 |
The probabilities sum to 1: \(1/4+1/2+1/4=1\). Although individual population observations can only be 2 or 6, a sample mean can be 4. The sampling distribution therefore has possible values that are not themselves possible individual observations.
Unequal Probabilities and Combining Outcomes
When population values are not equally likely, sample outcomes need not be equally likely either. It is not enough to count how many outcomes produce each mean: their probabilities must be included. Independence still lets us find the probability of an ordered sample by multiplying the probabilities of its draws.
Worked Example: A Rare Population Value
Suppose a hypothetical population has value 0 with probability 0.8 and value 10 with probability 0.2. Draw \(n=2\) observations independently with replacement. Find the possible sample means and their probabilities.
Model check. The two draws follow the same stated probability distribution and are independent. The ordered outcomes and their probabilities are:
| Ordered sample | Sample mean | Probability of sample |
|---|---|---|
| (0, 0) | 0 | \(0.8(0.8)=0.64\) |
| (0, 10) | 5 | \(0.8(0.2)=0.16\) |
| (10, 0) | 5 | \(0.2(0.8)=0.16\) |
| (10, 10) | 10 | \(0.2(0.2)=0.04\) |
The sample mean is 5 for either mixed outcome, so \(P(\bar{X}=5)=0.16+0.16=0.32\). The other probabilities are \(P(\bar{X}=0)=0.64\) and \(P(\bar{X}=10)=0.04\). The distribution is:
| Possible value of \(\bar{X}\) | Probability |
|---|---|
| 0 | 0.64 |
| 5 | 0.32 |
| 10 | 0.04 |
The probabilities sum to \(0.64+0.32+0.04=1\). Notice that counting the three possible means as if they were equally likely would be incorrect. The mean 0 is much more likely than the mean 10 because a sample of two zeros is more likely than a sample of two tens under this population model.
A Full Distribution and a Probability Question
A useful sampling distribution can answer questions about events involving the mean, not just probabilities of exact values. For example, once the possible means and their probabilities are known, the probability that a sample mean is at least a stated threshold is found by adding the probabilities of all qualifying means.
Worked Example: Probability the Mean Exceeds a Threshold
A hypothetical population has values 0, 2, and 4 with probabilities 0.2, 0.5, and 0.3, respectively. Draw \(n=2\) observations independently with replacement. Find the sampling distribution of \(\bar{X}\), then calculate \(P(\bar{X}\geq 3)\).
Model check. The population probabilities sum to 1, and the sample is specified to consist of two independent draws with replacement. For any two values \(a\) and \(b\), the resulting mean is \((a+b)/2\); the probability of the ordered outcome \((a,b)\) is \(P(X=a)P(X=b)\).
The possible means are 0, 1, 2, 3, and 4. Their probabilities are calculated by grouping the ordered outcomes:
- \(P(\bar{X}=0)=P((0,0))=0.2(0.2)=0.04\).
- \(P(\bar{X}=1)=P((0,2))+P((2,0))=0.2(0.5)+0.5(0.2)=0.20\).
- \(P(\bar{X}=2)=P((0,4))+P((2,2))+P((4,0))=0.2(0.3)+0.5(0.5)+0.3(0.2)=0.37\).
- \(P(\bar{X}=3)=P((2,4))+P((4,2))=0.5(0.3)+0.3(0.5)=0.30\).
- \(P(\bar{X}=4)=P((4,4))=0.3(0.3)=0.09\).
The resulting probabilities add to \(0.04+0.20+0.37+0.30+0.09=1\), as required. The event \(\bar{X}\geq 3\) includes means 3 and 4, so:
Under this hypothetical sampling model, 39% of repeated samples of size 2 would have a mean of at least 3. This probability describes repeated sample means, not the chance that an individual population value is at least 3.
The Population Distribution Is Not the Sampling Distribution
The population distribution describes the possible values and probabilities for an individual observation. The sampling distribution describes the possible values and probabilities for a statistic calculated from a whole sample. They answer different questions and generally do not have the same list of values or probabilities.
The distinction is visible in the examples. In the last example, an individual observation could be 0, 2, or 4, with probabilities 0.2, 0.5, and 0.3. The mean of two observations could instead be 0, 1, 2, 3, or 4, with a different set of probabilities. The sampling distribution arises by applying the mean calculation to each possible sample outcome and then combining the probabilities of outcomes that lead to the same mean.
For populations with continuously varying values, the sample mean may also have a continuous distribution. In that setting, a single exact value can have probability zero; probabilities are assigned to intervals of values. The same central idea applies: the sampling model determines possible samples, and the statistic calculated from those samples determines the distribution of \(\bar{X}\).
This idea matters because statistical conclusions based on a sample mean depend on how that mean could vary from sample to sample. A single observed mean does not, by itself, reveal that distribution. The sampling plan and the population model are needed to describe the mean's possible outcomes and probabilities.
Common Mistake
Wrong reasoning: “The population only contains 0 and 10, so the sample mean can only be 0 or 10.” This confuses individual observations with a statistic computed from several observations.
Correction: Calculate the statistic for each possible sample. In the example with population values 0 and 10 and \(n=2\), the samples \((0,10)\) and \((10,0)\) both have mean 5. Therefore 5 is a possible sample mean even though it is not a possible individual population value. Also, do not give each possible mean equal probability unless the sampling model actually implies those probabilities.
Check Your Understanding
Use the distinction between individual observations and sample means to answer these questions.
- A population has values 1 and 7, each with probability \(1/2\). For two independent draws with replacement, what are the possible values of \(\bar{X}\)?
- In the previous question, which ordered sample outcomes produce a mean of 4, and what is \(P(\bar{X}=4)\)?
- A population has values 0 and 5, with probabilities 0.9 and 0.1. Why is it incorrect to assume the possible sample means for \(n=2\) are equally likely?
- Explain in your own words how the population distribution differs from the sampling distribution of \(\bar{X}\).
- If several distinct sample outcomes produce the same mean, how do you find the probability of that mean?