Sampling Distributions and Their Supports

From a Random Sample to a Random Statistic

A population distribution describes the possible values of one observation. A sampling distribution describes the possible values of a quantity calculated from a random sample. Those are different objects: a statistic is a function of several observations, so its possible values depend both on the population and on how the statistic combines the sample values.

The first task is often to find the statistic’s support: the set of values it can take, or, for a continuous statistic, the values around which it can have positive probability. Finding the support is not the same as finding the probabilities at those values. It is the structural step that comes first.

Definition: Let \(X_1,\ldots,X_n\) be a random sample from a population distribution. A statistic is a function \(T=g(X_1,\ldots,X_n)\) that depends only on the observed sample and contains no unknown population parameter. The sampling distribution of \(T\) is the probability distribution induced by the random sample. The support of \(T\) consists of values that are possible under that distribution; for a continuous distribution, support includes boundary points when every neighborhood of the point has positive probability.

The random-sample assumption means that \(X_1,\ldots,X_n\) are independent and have the same population distribution. If the population is discrete, a sample outcome is an ordered \(n\)-tuple of possible observation values. The statistic maps each such tuple to a value of \(T\). If the population is continuous, the joint support of the sample describes the possible regions of sample space, and the function \(g\) maps those regions into values of the statistic.

For a finite discrete sample space, the attainable set is found by evaluating \(g\) at every possible sample outcome and retaining the distinct outputs. The sampling distribution also assigns probabilities to those outputs: outcomes that map to the same statistic value contribute to that value’s probability. For a continuous sample, the image of the sample support gives the attainable range; the distribution’s support is understood with the neighborhood convention in the definition.

Why the Sample Space Determines the Distribution

The probability rule follows directly from what it means for a statistic to take a value in a set. The event that \(T\) belongs to a set \(B\) is exactly the event that the sample outcome belongs to the set of inputs that \(g\) maps into \(B\).

P(T\in B) = P\bigl(g(X_1,\ldots,X_n)\in B\bigr) = P\bigl((X_1,\ldots,X_n)\in g^{-1}(B)\bigr).

Here \(g^{-1}(B)\) means the collection of sample outcomes whose statistic values lie in \(B\). In a finite discrete model, if \(t\) is one possible value, this gives

P(T=t) = \sum_{\boldsymbol{x}:g(\boldsymbol{x})=t} P\bigl((X_1,\ldots,X_n)=\boldsymbol{x}\bigr).

This is a justification of the basic mapping method: the statistic’s probability comes from the sample outcomes that produce it. In this tutorial, the focus is on identifying the possible outputs. Calculating all their probabilities is a separate task.

Worked Example: A Weighted Statistic From Two Bernoulli Observations

Let \(X_1\) and \(X_2\) be independent and identically distributed Bernoulli random variables with parameter \(p=0.3\), so each observation is 1 with probability \(0.3\) and 0 with probability \(0.7\). Define \(T=X_1+2X_2\). Find the support of \(T\), and compare it with the support of one observation.

Method. List the possible ordered sample outcomes, then apply \(g(x_1,x_2)=x_1+2x_2\) to each. Independence does not make the two observations one combined population value; the statistic is calculated from their ordered pair.

Each observation can be 0 or 1, so the sample space is \(\{(0,0),(0,1),(1,0),(1,1)\}\). Evaluate the statistic:

For \((0,0)\), \(T=0+2(0)=0\).
For \((0,1)\), \(T=0+2(1)=2\).
For \((1,0)\), \(T=1+2(0)=1\).
For \((1,1)\), \(T=1+2(1)=3\).

Therefore the support of \(T\) is \(\{0,1,2,3\}\). The support of one observation \(X_1\) is only \(\{0,1\}\). The two supports differ because \(T\) combines two observations, with the second observation receiving weight 2.

Check. All four outputs are distinct, and each sample outcome has positive probability because \(0<p<1\). Thus each listed output is genuinely possible. No other output can occur because the four outcomes exhaust the sample space. The common mistake is to report \(\{0,1\}\) as the support of \(T\), confusing the population support with the statistic’s support.

Finding Supports in Continuous Models

In continuous models, individual points usually have probability zero. That does not mean their values are impossible in the support sense. For example, an endpoint can be in the support if every interval around it contains possible observations with positive probability. To find a statistic’s support, examine the full range of its function over the sample support, and then check whether the candidate boundary values meet the neighborhood condition.

Worked Example: The Difference of Two Uniform Observations

Let \(X_1\) and \(X_2\) be independent and identically distributed Uniform\((2,5)\) random variables, each with density \(1/3\) between 2 and 5. Define \(T=X_1-X_2\). Find the support of \(T\), and explain the status of its endpoints.

Method. Determine the range of \(x_1-x_2\) when both coordinates belong to the sample support \([2,5]\). The smallest difference occurs when \(x_1\) is smallest and \(x_2\) is largest; the largest occurs with the reverse choices.

The minimum is \(2-5=-3\), and the maximum is \(5-2=3\). Every intermediate value is attainable. If \(0\leq t\leq3\), choose \(x_2=2\) and \(x_1=2+t\); then both coordinates lie in \([2,5]\) and \(x_1-x_2=t\). If \(-3\leq t<0\), choose \(x_1=2\) and \(x_2=2-t\); again both coordinates lie in \([2,5]\), and \(x_1-x_2=t\). Therefore the range is \([-3,3]\).

The endpoints are in the support even though \(P(T=-3)=P(T=3)=0\). For instance, values of \(X_1\) close to 2 and \(X_2\) close to 5 produce differences close to \(-3\). Such pairs occupy a region of positive area in the joint sample support, so every neighborhood of \(-3\) has positive probability. The same reasoning applies at 3.

Check. The statistic cannot exceed \(5-2=3\) or fall below \(2-5=-3\), and the construction shows that every value between those bounds is attainable. The support is \([-3,3]\). A common mistake is to exclude the endpoints merely because each exact endpoint has probability zero; support for a continuous distribution is determined by neighborhoods, not by the probability of a single point.

Population Support Is Not Statistic Support

A statistic can have a support that overlaps the population support, is contained in it, or extends beyond it. There is no general rule that the two supports must match. The defining function and the number of observations matter.

Worked Example: Absolute Difference in a Three-Value Population

Suppose \(X_1\) and \(X_2\) are independent and identically distributed, with \(P(X=1)=0.2\), \(P(X=3)=0.5\), and \(P(X=6)=0.3\). Define \(T=|X_1-X_2|\). Find the support of \(T\) and compare it with the population support.

Method. Consider the distinct population values and find the absolute difference for equal and unequal pairs. Since all three population values have positive probability, every pair of values considered is possible under independence.

Equal values produce \(T=|1-1|=0\), \(T=|3-3|=0\), or \(T=|6-6|=0\). Unequal values produce:

\(|1-3|=2\),
\(|1-6|=5\),
\(|3-6|=3\).

Reversing the order of a pair does not create a new absolute difference. Thus the support of \(T\) is \(\{0,2,3,5\}\), while the support of an individual observation is \(\{1,3,6\}\). The statistic’s support is not the same set as the population’s, and it includes 0 even though 0 cannot be an individual observation.

Check. Every listed statistic value comes from at least one possible pair, and there are no other types of pair than equal values or the three unequal pairs listed. Therefore the support is complete. This finds possible values only; it does not calculate the probabilities assigned to those values. The common mistake is to copy the population support as the support of \(T\), or to assume that a statistic must take values that are themselves population values.

A Reliable Support-Finding Procedure

Use the following process before attempting to calculate a sampling distribution’s probabilities. It makes the distinction between the sample, the statistic, and the population explicit.

State the sampling model.
Identify the population distribution, the sample size, and whether the observations are independent. Specify discrete possible values or the continuous support as appropriate.

Write the statistic as a function.
Express it as \(T=g(X_1,\ldots,X_n)\). Keep the input observations distinct from the output \(T\).

Map the sample space.
For a finite sample space, evaluate the function on all possible outcomes. For a continuous sample support, determine the full range of the function.

Check endpoints and completeness.
Verify that each listed value can occur and that no other value can occur. In continuous models, use neighborhoods to decide whether boundary values belong to the support.

Keep probabilities as a separate step.
The support identifies possible values; the sampling distribution additionally assigns probabilities to them. For discrete statistics, probabilities are combined over all sample outcomes producing the same value.

Common Mistakes and Why the Distinction Matters

The sampling distribution is the probability law of a statistic over repeated samples from a specified population. It is not the population distribution of a single observation, nor is it the list of values observed in one particular sample. This distinction matters whenever a statistical procedure is assessed through the behavior of a statistic across samples.

Using the population support directly. A function of several observations can create new values, as the absolute-difference example showed.
Listing observed values instead of possible values. One realized sample gives one realized statistic value. The support considers all sample outcomes allowed by the model.
Confusing range and probability. Knowing that a value is possible does not tell you how likely it is. Multiple sample outcomes can map to the same value, and their probabilities must be combined.
Dropping continuous endpoints because a point has probability zero. The support of a continuous distribution is based on positive probability in neighborhoods, not positive probability at each individual point.
Ignoring the ordered sample structure. When the statistic treats sample positions differently, such as \(X_1+2X_2\), swapping observations can change its value. Do not collapse ordered outcomes unless the statistic and model justify doing so.

Key takeaway: A statistic is a function of a random sample. Its support is found by mapping the sample space through that function; its sampling distribution also records the probabilities induced by the sample. Always distinguish both from the distribution of one population observation.

Check Your Understanding

For each problem, identify the statistic’s support and distinguish it from the population support where requested. Focus on possible values, not on calculating a full probability distribution.

Let \(X_1,X_2\) be independent Bernoulli observations with parameter \(p=0.4\). For \(T=2X_1+X_2\), list the support of \(T\) and the support of one observation. (Answer: support of \(T\) is \(\{0,1,2,3\}\); support of one observation is \(\{0,1\}\).)
Let \(X_1,X_2\) be independent Uniform\((0,4)\) observations and \(T=X_1+X_2\). Find the support of \(T\), including its endpoints. (Answer: \([0,8]\).)
An independent sample of size 2 comes from a population with support \(\{2,5,9\}\), with positive probability at each value. For \(T=\min(X_1,X_2)\), find the support of \(T\). (Answer: \(\{2,5,9\}\).)
Let \(X_1,X_2\) be independent with population support \(\{-2,1,4\}\), each value having positive probability. For \(T=X_1+X_2\), find the support of \(T\). (Answer: \(\{-4,-1,2,5,8\}\).)
Let \(X_1,X_2\) be independent Uniform\((1,3)\) observations and define \(T=X_1/X_2\). Determine the support by finding the minimum and maximum possible ratios. (Answer: \([1/3,3]\).)

What You'll Learn