Why Look at the Sample Given a Statistic?
A statistic compresses a sample into a summary. For example, a sample of Bernoulli observations can be reduced to its total number of successes. The summary is useful when it retains all the sample’s information about the unknown parameter. To express that idea precisely, examine what the full sample can still tell us after the statistic is known.
If the conditional distribution of the sample given the statistic does not depend on the parameter, then knowing the detailed sample beyond the statistic cannot provide further information about that parameter. This conditional-distribution definition gives a direct way to understand sufficiency, without relying on a separate criterion based on the joint probability function.
In a discrete model, the definition is tested by calculating \(P_\theta(X=x\mid T=t)\) for sample outcomes \(x\) that satisfy \(T(x)=t\). The conditioning event must have positive probability. In a continuous model, individual outcomes generally have probability zero, so the corresponding conditional distribution is understood through a conditional density or a regular conditional law. In either setting, the question is whether the resulting conditional law varies with \(\theta\).
A Direct Test in Discrete Models
For discrete random variables, conditional probability gives the basic calculation. If \(T=T(X)\), then for any sample value \(x\) with \(T(x)=t\) and \(P_\theta(T=t)>0\),
The numerator is the probability of one particular sample; the denominator is the total probability of all sample outcomes that produce the statistic value \(t\). If the parameter cancels from this ratio for every possible \(t\) and every compatible sample outcome, then the conditional distribution does not depend on the parameter.
Here is the underlying justification: conditional probability divides the joint probability of the sample event and the conditioning event by the probability of the conditioning event. Since \(T\) is determined by \(X\), the event \(\{X=x\}\) is contained in \(\{T=t\}\) whenever \(T(x)=t\). Thus the joint probability in the numerator is just \(P_\theta(X=x)\). The ratio compares the probability of one compatible sample outcome with the probability of all compatible outcomes combined.
Worked Example: The Number of Successes in a Bernoulli Sample
Let \(X_1,\ldots,X_4\) be independent Bernoulli random variables with unknown success probability \(p\), where \(0<p<1\). Each \(X_i\) equals 1 for a success and 0 for a failure. Consider \(T=X_1+X_2+X_3+X_4\). Find the conditional probability of the ordered sample \((1,0,1,0)\) given \(T=2\), and determine whether the conditional distribution depends on \(p\).
Method. Use the discrete conditional-probability ratio and count all sample outcomes with two successes.
The probability of the specified ordered sample is
There are \(\binom{4}{2}=6\) ordered samples with exactly two successes, and each has probability \(p^2(1-p)^2\). Therefore,
The required conditional probability is
For a numerical check, at \(p=0.2\) the numerator is \(0.2^2(0.8)^2=0.0256\) and the denominator is \(6(0.0256)=0.1536\), giving \(0.0256/0.1536=1/6\). At \(p=0.7\), they are \(0.7^2(0.3)^2=0.0441\) and \(6(0.0441)=0.2646\), again giving \(1/6\). For any \(t\), all \(\binom{4}{t}\) ordered outcomes with \(t\) successes are equally likely conditional on \(T=t\), each with probability \(1/\binom{4}{t}\). This conditional law is independent of \(p\), so \(T\) is sufficient for \(p\).
Check. At \(T=2\), the six conditional probabilities are each \(1/6\), and their sum is \(6(1/6)=1\). This method avoids the common mistake of assuming that a statistic is sufficient just because it is a familiar summary; the conditional distribution must actually be checked.
What the Conditional Distribution Says About Information
Sufficiency does not mean that the statistic is independent of the parameter. In the Bernoulli example, the distribution of \(T\) changes with \(p\): for instance, \(P_p(T=4)=p^4\). That changing distribution is how the statistic carries information about \(p\). Sufficiency says something different: once \(T\) is fixed, the detailed arrangement of successes and failures has a conditional distribution that no longer depends on \(p\).
This distinction is the central interpretation of conditional-distribution sufficiency. The statistic can retain information about the parameter, while the sample details omitted by the statistic contribute no additional parameter information. Conversely, if the conditional distribution still depends on the parameter, the data reduction has discarded some information relevant to it.
Worked Example: Conditioning Two Poisson Counts on Their Total
Suppose \(X_1\) and \(X_2\) are independent Poisson counts with means \(2\lambda\) and \(3\lambda\), respectively, where \(\lambda>0\) is unknown. Let \(T=X_1+X_2\). Given an observed total of 5, calculate \(P_\lambda(X_1=2\mid T=5)\) and use the conditional distribution to assess whether \(T\) is sufficient for \(\lambda\).
Method. First calculate the joint probability of the specified pair and the probability of the total, then simplify their ratio. The joint mass at \((2,3)\) is
The sum of independent Poisson variables is Poisson with mean equal to the sum of the means, so \(T\sim\operatorname{Poisson}(5\lambda)\). Thus
Divide the joint probability by the total probability. Since \(X_2=3\) when \(T=5\) and \(X_1=2\), the result is
More generally, given \(T=t\), \(X_1\) has a binomial distribution with \(t\) trials and success probability \(2/5\); the remaining count is \(X_2=t-X_1\). The conditional law depends on the known mean proportions \(2/5\) and \(3/5\), but not on \(\lambda\). Therefore \(T\) is sufficient for \(\lambda\). As a numerical check on the displayed probability, \(10(0.16)(0.216)=10(0.03456)=0.3456\).
Common mistake avoided. The conditional success probability is \(2/5\), not \(2\lambda\) or \(2\lambda/5\). The two Poisson means share the unknown scale \(\lambda\); conditioning on the total removes that scale and leaves the ratio of the means.
Continuous Samples and Parameter-Dependent Support
The same idea applies when sample values are continuous, although the conditional law is described by a conditional density or another conditional distribution rather than probabilities of individual points. In these models, the range of possible statistic values can depend on the parameter. The definition must therefore be read for values \(t\) that can occur under the parameter value being considered: the conditional distribution must agree across all parameter values for which \(T=t\) is possible.
Worked Example: The Maximum from a Uniform Sample
Let \(X_1,X_2\) be independent observations from the continuous uniform distribution on \((0,\theta)\), where \(\theta>0\) is unknown. Let \(T=\max(X_1,X_2)\). Determine the conditional distribution of the ordered pair given \(T=t\), and assess whether the maximum is sufficient for \(\theta\).
Method. Describe which observation attains the maximum and then the distribution of the other observation, conditional on that choice. Ties have probability zero because the observations have continuous distributions. Given \(T=t\), either \(X_1=t\) or \(X_2=t\); by symmetry each case has conditional probability \(1/2\). If \(X_1=t\), then \(X_2\) is uniform on \((0,t)\); if \(X_2=t\), then \(X_1\) is uniform on \((0,t)\). These conditional laws do not involve \(\theta\).
For a numerical check, take \(t=6\) and ask for \(P(X_1\leq 3\mid T=6)\). If \(X_1\) is the maximum, then \(X_1=6\), so this event does not occur. If \(X_2\) is the maximum, \(X_1\) is uniform on \((0,6)\), and the conditional probability that it is at most 3 is \(3/6=1/2\). Combining the two cases gives
In general, for \(0\leq x<t\), \(P(X_1\leq x\mid T=t)=x/(2t)\): only the case where \(X_2\) is the maximum can contribute, and that case has probability \(1/2\), followed by a uniform probability \(x/t\). At or above \(t\), the conditional CDF is 1. The entire conditional law is determined by \(t\), not by \(\theta\). For any \(t\) that is possible under the model, namely \(0<t<\theta\), the maximum is therefore sufficient for \(\theta\).
Check. At \(t=6\) and \(x=3\), the general expression gives \(3/(2\cdot6)=1/4\), matching the two-case calculation. A common mistake is to treat \(X_1\) and \(X_2\) as still independent uniform observations after conditioning on their maximum. They are not: the condition forces one observation to equal the maximum and constrains the other to be no greater than it.
A Practical Checklist and Common Traps
When testing sufficiency by conditional distributions, specify the model and parameter first, then condition on the proposed statistic. The following sequence keeps the logic clear.
Record the distribution, parameter, and any independence assumptions. A conditional calculation is only as sound as the model that supports it.
List or characterize the sample outcomes that produce that value. In a continuous model, describe the conditional law rather than assigning positive probability to a point.
Use the conditional-probability ratio in a discrete model, or derive the corresponding conditional law in a continuous model.
If the conditional law is the same for every allowed parameter value, the statistic is sufficient. If it changes with the parameter, the statistic does not meet this definition.
- Do not confuse the statistic’s distribution with the conditional distribution. The distribution of a sufficient statistic will generally depend on the parameter. The test concerns the sample given the statistic.
- Do not check only one convenient value. A statistic must have a parameter-free conditional law for every possible statistic value, not merely one selected value.
- Do not ignore support. When a model’s support depends on the parameter, state for which parameter values the conditioning event is possible and compare the conditional laws on that domain.
- Do not assume a summary is sufficient because it seems natural. A statistic’s name or familiarity does not establish sufficiency; the conditional distribution does.
Check Your Understanding
For each problem, identify the conditional distribution or probability and decide what it implies about sufficiency where requested.
- Let \(X_1,\ldots,X_5\) be independent Bernoulli observations with unknown success probability \(p\), and let \(T=\sum X_i\). Given \(T=3\), find the conditional probability of the ordered sample \((1,0,1,0,1)\). (Answer: \(1/\binom{5}{3}=1/10\).)
- Let \(X_1\) and \(X_2\) be independent Poisson variables with means \(4\lambda\) and \(\lambda\), where \(\lambda>0\). Given \(T=X_1+X_2=4\), find \(P(X_1=3\mid T=4)\). (Answer: \(\binom{4}{3}(4/5)^3(1/5)=256/625=0.4096\).)
- In the preceding Poisson model, state the conditional distribution of \(X_1\) given \(T=t\), and say whether the conditional-distribution definition shows \(T\) is sufficient for \(\lambda\). (Answer: \(X_1\mid T=t\sim\operatorname{Binomial}(t,4/5)\); yes, this law does not depend on \(\lambda\).)
- Let \(X_1,X_2\) be independent uniform observations on \((0,\theta)\), and set \(T=\max(X_1,X_2)\). For \(0\leq x<t\), find \(P(X_1\leq x\mid T=t)\). (Answer: \(x/(2t)\).)
- Suppose a proposed statistic \(U\) in a discrete model has two sample outcomes with \(U=2\), whose conditional probabilities are \(\theta/(1+\theta)\) and \(1/(1+\theta)\), for \(\theta>0\). Does the conditional-distribution definition establish that \(U\) is sufficient? (Answer: No; the conditional probabilities depend on \(\theta\).)