Why Factorization Identifies Sufficiency
A statistic is useful for estimation when it retains all the information in the sample about an unknown parameter. Sufficiency gives this idea a precise meaning: once the statistic is known, the remaining details of the sample have a conditional distribution that does not depend on the parameter. The factorization criterion turns that definition into a practical test using a likelihood.
The test applies to discrete samples through probability mass functions and to continuous samples through probability densities. Its key discipline is to separate the likelihood into a part that may depend on the parameter only through the statistic and a part that does not depend on the parameter at all.
To use the criterion, write the joint mass function or density of the sample as \(f_\theta(x)\), where \(x=(x_1,\ldots,x_n)\) is a possible sample. The likelihood is this same function viewed as a function of \(\theta\) for the observed \(x\). A factor that depends on \(x\) but not on \(\theta\) is allowed to retain sample details; those details do not affect inference about \(\theta\) once the statistic is known.
Why the Criterion Holds
The factorization is equivalent to saying that conditional probabilities or densities within a fixed value of \(T\) do not change with the parameter. The following argument gives the direct calculation for discrete data and explains the density version.
Suppose \(f_\theta(x)=g_\theta(T(x))h(x)\). For a discrete sample and a value \(t\), sum over all sample points \(y\) with \(T(y)=t\). Then \(P_\theta(T=t)=g_\theta(t)\sum_{y:T(y)=t}h(y)\). For any \(x\) with \(T(x)=t\), division gives \(P_\theta(X=x\mid T=t)=h(x)/\sum_{y:T(y)=t}h(y)\). The parameter-dependent factor cancels, so the conditional distribution is parameter-free.
For continuous samples, condition on \(T=t\) using a regular conditional distribution. In the disintegration of the dominating measure along the sets \(T(x)=t\), the conditional density is proportional to \(h(x)\); the factor \(g_\theta(t)\) is constant within each such set and cancels in normalization. Thus the conditional law of \(X\) given \(T\) does not depend on \(\theta\).
Let \(\nu\) be a common dominating measure and disintegrate it along \(T\), writing its marginal measure for \(T\) as \(\nu_T\) and its conditional measures as \(\nu_t\). If \(T\) is sufficient, let \(K(dx\mid t)\) be the parameter-free conditional law of \(X\) given \(T=t\). The distribution of \(T\) under \(\theta\) has a density \(g_\theta(t)=dP_\theta^T/d\nu_T(t)\). The conditional law \(K\) has a density \(h(x)\) relative to \(\nu_{T(x)}\), independent of \(\theta\). Combining the marginal density and the conditional density gives \(f_\theta(x)=g_\theta(T(x))h(x)\), almost everywhere. This is the required factorization.
In the discrete proof, the denominator is an ordinary finite or countable sum. In a continuous model, the corresponding step is conditional normalization under the disintegration of the dominating measure; one should not interpret it as dividing by a probability of the event \(T=t\), which is usually zero. The theorem therefore applies to both types of samples, with sums replaced by the appropriate conditional integration.
Worked Examples
Worked Example: A Sufficient Count in a Bernoulli Sample
A hypothetical inspection records whether each of five independently selected components passes a test. Let \(X_i=1\) for a pass and \(X_i=0\) otherwise, with \(X_i\sim\operatorname{Bernoulli}(p)\), where \(0<p<1\). For the observed sequence \((1,0,1,1,0)\), determine whether the number of passes \(T=\sum_{i=1}^5X_i\) is sufficient for \(p\). Also calculate the conditional probability of this particular ordered sequence given \(T=3\).
Method. Write the joint mass function and factor it through \(T\). For any possible binary sequence \(x\), \(f_p(x)=\prod_{i=1}^5p^{x_i}(1-p)^{1-x_i}=p^{\sum x_i}(1-p)^{5-\sum x_i}\). Thus \(f_p(x)=g_p(T(x))h(x)\), with \(g_p(t)=p^t(1-p)^{5-t}\) and \(h(x)=1\). By the factorization criterion, \(T\) is sufficient for \(p\).
For the observed data, the sum is \(T=1+0+1+1+0=3\). There are \(\binom{5}{3}=10\) ordered binary sequences with three passes. Each has mass \(p^3(1-p)^2\), so \(P_p(T=3)=10p^3(1-p)^2\). The conditional probability of the specified sequence is therefore \(p^3(1-p)^2/[10p^3(1-p)^2]=1/10=0.1\).
Check. The ten possible sequences each have conditional probability \(0.1\), and their probabilities sum to \(10(0.1)=1\). The calculation avoids treating the order of the five results as additional information about \(p\): conditional on the total number of passes, every ordering has the same probability.
Worked Example: A Sufficient Statistic in a Normal Sample
Suppose three independent observations follow \(X_i\sim N(\mu,4)\), where the variance is known to be \(4\) and the mean \(\mu\) is unknown. The observed sample is \((1,2,4)\). Use the likelihood factorization to identify a sufficient statistic for \(\mu\), then compare its likelihood with that of the sample \((0,3,4)\).
Method. For one observation, the density is \(f_\mu(x_i)=(1/\sqrt{8\pi})\exp(-(x_i-\mu)^2/8)\). Independence gives the joint density \(f_\mu(x)=(1/\sqrt{8\pi})^3\exp(-\sum_{i=1}^3(x_i-\mu)^2/8)\). Expand the sum: \(\sum_{i=1}^3(x_i-\mu)^2=\sum x_i^2-2\mu\sum x_i+3\mu^2\). Therefore \(f_\mu(x)=(1/\sqrt{8\pi})^3\exp(-\sum x_i^2/8)\exp(\mu\sum x_i/4-3\mu^2/8)\). The first exponential depends on the data but not on \(\mu\); the second depends on the data only through \(T=\sum X_i\). The factorization criterion shows that \(T\) is sufficient for \(\mu\).
For \((1,2,4)\), \(T=1+2+4=7\) and \(\sum x_i^2=1+4+16=21\). For \((0,3,4)\), \(T=0+3+4=7\) and \(\sum x_i^2=0+9+16=25\). Their likelihood ratio is \(f_\mu(1,2,4)/f_\mu(0,3,4)=\exp(-(21-25)/8)=\exp(1/2)\), which does not depend on \(\mu\). Thus, although these two samples have different likelihood values, their relative likelihood is parameter-free when their sums agree.
Check. The ratio is positive and greater than one because the first sample has a smaller sum of squared deviations from its own sample mean. The statistic is the sum, not the sum of squares: the latter appears only in the parameter-free factor. The model assumptions used here are independence, a common normal mean, and known variance \(4\).
Worked Example: Handling Parameter-Dependent Support
A hypothetical device has a lifetime \(X_i\) with density \(f_\theta(x)=1/\theta\) for \(0<x<\theta\), and zero otherwise, where \(\theta>0\). Four independent device lifetimes are observed as \((1.2,2.0,0.7,1.5)\), in years. Use factorization to identify a sufficient statistic for \(\theta\), explicitly accounting for the support.
Method. Independence gives the joint density \(f_\theta(x)=\theta^{-4}\prod_{i=1}^4\mathbf{1}_{\{0<x_i<\theta\}}\). For positive observations, the product of indicators equals \(\mathbf{1}_{\{x_{(4)}<\theta\}}\), where \(x_{(4)}=\max(x_1,x_2,x_3,x_4)\). Hence \(f_\theta(x)=\bigl[\theta^{-4}\mathbf{1}_{\{T(x)<\theta\}}\bigr]\bigl[\prod_{i=1}^4\mathbf{1}_{\{x_i>0\}}\bigr]\), where \(T(x)=\max_i x_i\). The second factor is parameter-free, and the first depends on the observations only through their maximum. Therefore the sample maximum is sufficient for \(\theta\).
For the observed sample, \(T=\max(1.2,2.0,0.7,1.5)=2.0\) years. If \(\theta=3\) years, the density at these observations is \(3^{-4}=1/81\) per year to the fourth power. If \(\theta=1.8\) years, the density is zero because \(2.0\not<1.8\). The support indicator is essential: it records whether the observed sample is possible under the proposed parameter.
Check. The statistic is the maximum because the entire sample is in the support exactly when every observation is below \(\theta\), equivalently when the maximum is below \(\theta\). The model assumes independent lifetimes with the stated uniform density; the endpoint has probability zero, so the strict inequality at the boundary does not change probabilities.
Common Mistakes and Exam Traps
A frequent error is to omit a parameter-dependent support condition. In the lifetime example, an incorrect calculation might write \(f_\theta(x)=\theta^{-4}\) for every positive sample and then claim that any statistic, or no statistic at all, is needed. That expression is not the joint density: it assigns positive density even when some observed lifetime exceeds \(\theta\). The correct expression includes \(\mathbf{1}_{\{T(x)<\theta\}}\), allowing the support restriction to be represented through the maximum.
Another trap is to let the purported factor \(h(x)\) depend on \(\theta\). For example, a decomposition \(f_\theta(x)=g_\theta(T(x))h_\theta(x)\) does not establish sufficiency merely because it looks separated. The data-only factor must be parameter-free, and the parameter-dependent factor may use the data only through \(T(x)\).
- Do not confuse the likelihood with a probability of the parameter. The likelihood is the joint mass or density as a function of \(\theta\) for fixed data.
- Do not discard support information. Include any parameter-dependent indicator in the parameter-and-statistic factor.
- Do not claim a statistic is sufficient because it appears in the likelihood. Check that all remaining sample dependence is in a parameter-free factor.
- For continuous data, do not divide by \(P(T=t)\). Use conditional distributions or densities; a single exact value of a continuous statistic generally has probability zero.
Check Your Understanding
For each model, identify the proposed sufficient statistic if the factorization supports it, and keep parameter-dependent support conditions in the likelihood.
- Four independent trials have success probability \(p\). The observed outcomes are \((0,1,1,0)\). Factor the likelihood through the number of successes and find the probability of this ordered outcome conditional on exactly two successes. (Answer: \(T=\sum X_i\) is sufficient; the conditional probability is \(1/6\).)
- Two independent observations have densities \(f_\theta(x)=\theta e^{-\theta x}\) for \(x>0\), with \(\theta>0\). Write the joint density and identify a statistic through which its parameter dependence factors. (Answer: \(T=X_1+X_2\) is sufficient; \(f_\theta(x_1,x_2)=\theta^2e^{-\theta T}\mathbf{1}_{\{x_1>0,x_2>0\}}\).)
- Three independent observations follow \(N(\mu,1)\) with unknown \(\mu\). Their values are \(2,0,1\). Expand the likelihood and identify a sufficient statistic. (Answer: \(T=\sum X_i=3\) is sufficient; the likelihood is proportional to \(\exp(3\mu-3\mu^2/2)\), with a parameter-free factor.)
- Two independent observations have density \(1/\theta\) on \(0<x<\theta\), zero otherwise, with \(\theta>0\). The observed values are \(0.8\) and \(1.4\). State the sufficient statistic and evaluate whether the likelihood is zero at \(\theta=1.2\). (Answer: \(T=\max(X_1,X_2)=1.4\) is sufficient; the likelihood is zero at \(\theta=1.2\) because \(1.4\not<1.2\).)