Sufficient Statistics as Likelihood-Ratio Reductions

Why Reduce a Test to a Statistic?

A Neyman–Pearson test can be described using the complete observed sample: calculate its likelihood ratio and reject the null hypothesis in a specified region. But a sample may contain many details that do not affect that ratio. If the ratio depends only on a statistic, the test can be carried out using that statistic alone. The reduction is useful only if it preserves the test's behavior under both hypotheses.

The central question is not whether a statistic summarizes the data in some broad sense. It is whether the statistic retains the information needed for the particular likelihood-ratio comparison. This is a precise condition that can be checked algebraically.

Definition: Consider two simple hypotheses $H_0:\theta=\theta_0$ and $H_1:\theta=\theta_1$. Suppose the sample $X$ has probability mass functions or densities $f_0(x)$ and $f_1(x)$ under the two hypotheses, with respect to a common measure. On points where $f_0(x)>0$, the likelihood ratio in favor of $H_1$ is $$ \Lambda(x)=\frac{f_1(x)}{f_0(x)}. $$ A statistic $T=T(X)$ captures the likelihood-ratio information for this pair if there is a function $g$ such that $\Lambda(x)=g(T(x))$ on the relevant sample space, up to sets having probability zero under both hypotheses.

The likelihood ratio may be zero or infinite when one hypothesis assigns zero probability or density where the other does not. In that case, use the extended likelihood ratio: it is zero where $f_1=0<f_0$, and infinite where $f_0=0<f_1$. The reduction must retain these cases as well. If the two models have different supports, it is not safe to cancel factors or reason only on their common positive-density region.

This pair-specific condition is closely connected to statistical sufficiency, but the terms should not be conflated. A statistic is sufficient for a whole parameter family if the conditional distribution of the sample given the statistic does not depend on the parameter. A statistic that captures the likelihood ratio for just $\theta_0$ versus $\theta_1$ is enough for that testing problem; it need not be sufficient for every parameter value in a larger model.

How the Reduction Preserves a Neyman–Pearson Test

For two simple hypotheses, the Neyman–Pearson lemma says that a most powerful test of a specified size rejects for sufficiently large values of $\Lambda(X)$, with possible randomization at the boundary. If $\Lambda(X)=g(T(X))$, the rejection decision can therefore be made from $T$. This does not change which samples are rejected or the probability of rejection under either hypothesis.

Write the likelihood ratio.
Use the probability mass functions or densities for the two simple hypotheses, and account for their supports.

Find the statistic that determines it.
Rearrange the ratio to identify a statistic $T(X)$ such that $\Lambda(X)=g(T(X))$.

Translate the critical rule.
Write the rejection condition in terms of $T$, preserving the original inequality and any boundary randomization.

Check size and power.
Because the reduced rule makes the same decision for every sample, its rejection probabilities under $H_0$ and $H_1$ are unchanged.

Theorem: If a test function $\phi(x)$, taking values in $[0,1]$, can be written as $\psi(T(x))$, then for $j=0,1$, $$ E_{\theta_j}[\phi(X)] =E_{\theta_j}[\psi(T(X))]. $$ In particular, the size $E_{\theta_0}[\phi(X)]$ and the power $E_{\theta_1}[\phi(X)]$ are preserved by replacing the observation-level rule with the equivalent rule based on $T$.

Here a test function value between zero and one represents the probability of rejecting $H_0$ at that observation. Thus the equality also covers randomized tests. If the likelihood-ratio rule rejects when $\Lambda(x)>c$, accepts when $\Lambda(x)<c$, and rejects with probability $\gamma$ when $\Lambda(x)=c$, then all three decisions can be expressed using $T$ whenever $\Lambda=g(T)$.

Worked Examples

Worked Example: Reducing a Bernoulli Sample to Its Success Count

Suppose $X_1,\ldots,X_4$ are independent Bernoulli observations, where $X_i=1$ denotes a recorded event. Test $H_0:p=1/4$ against $H_1:p=1/2$. Consider the likelihood-ratio rule that rejects when $\Lambda(X)\ge 16/3$. Show that the test reduces to the success count, and calculate its size and power.

Method. Under each hypothesis, the sample has the Bernoulli product model with the stated success probability. Let $K=\sum_{i=1}^4 X_i$. A particular ordered sample with $k$ successes has likelihood $$ L_0(x)=\left(\frac14\right)^k\left(\frac34\right)^{4-k}, \qquad L_1(x)=\left(\frac12\right)^k\left(\frac12\right)^{4-k}. $$ Therefore, $$ \Lambda(x)=\frac{L_1(x)}{L_0(x)} =2^k\left(\frac23\right)^{4-k}, $$ which depends on the data only through $K$.

For $k=0,1,2,3,4$, respectively, the ratios are $$ \frac{16}{81},\quad \frac{16}{27},\quad \frac{16}{9},\quad \frac{16}{3},\quad 16. $$ The rule $\Lambda\ge 16/3$ is thus exactly the rule $K\ge 3$. The observation-level test and the count-based test reject the same samples.

Under $H_0$, $K$ has a binomial distribution with $n=4$ and $p=1/4$. Its size is $$ P_{1/4}(K\ge3) ={4\choose3}\left(\frac14\right)^3\left(\frac34\right) +\left(\frac14\right)^4 =\frac{12}{256}+\frac{1}{256} =\frac{13}{256} =0.0508. $$ Under $H_1$, $K$ has a binomial distribution with $n=4$ and $p=1/2$. Its power is $$ P_{1/2}(K\ge3) ={4\choose3}\left(\frac12\right)^4 +\left(\frac12\right)^4 =\frac{5}{16} =0.3125. $$ Probabilities are rounded to four decimals. As a check, both calculations sum the probabilities of the same rejected samples, grouped by their count $K$. The common mistake is to use an incorrect factor for failures in the ratio; the failure contribution is $(3/4)/(1/2)=3/2$ in $L_0/L_1$, or $2/3$ in $L_1/L_0$, as used above.

Worked Example: Reducing Categorical Outcomes to Counts

Two independent devices each produce one label, $A$, $B$, or $C$. Under $H_0$, the label probabilities are $(0.5,0.3,0.2)$; under $H_1$, they are $(0.2,0.3,0.5)$, in the order $A,B,C$. A test rejects when the likelihood ratio is greater than $2.5$. Find a statistic that captures the ratio and calculate the size and power.

Method. Let $N_A,N_B,N_C$ be the counts of each label in the two observations. The observations are independent, so the likelihood ratio is the product of the individual probability ratios. The ratios for one observation of $A,B,C$ are, respectively, $$ \frac{0.2}{0.5}=0.4,\qquad \frac{0.3}{0.3}=1,\qquad \frac{0.5}{0.2}=2.5. $$ Consequently, $$ \Lambda(X)=0.4^{N_A}1^{N_B}2.5^{N_C}. $$ The count vector $T=(N_A,N_B,N_C)$ captures the likelihood ratio, even though it does not record the order of the two labels.

For the possible unordered pairs $AA,AB,AC,BB,BC,CC$, the corresponding ratios are $0.16,0.4,1,1,2.5,6.25$. Since the rule is $\Lambda>2.5$, it rejects only for the pair $CC$. The strict inequality matters: a pair $BC$ or $CB$ has ratio exactly $2.5$ and is not rejected.

Under $H_0$, the probability of two $C$ labels is $0.2^2=0.04$, so the size is $0.0400$. Under $H_1$, it is $0.5^2=0.25$, so the power is $0.2500$. Both probabilities are exact to four decimals. The check is direct: the reduced rule $N_C=2$ is precisely the original observation-level event $CC$. A common mistake is to reject at equality even when the stated rule uses $>$; doing so would also reject $BC$ and $CB$, changing both size and power.

Worked Example: Reducing Exponential Observations to Their Sum

Let $X_1,X_2,X_3$ be independent exponential observations with rate $\lambda$, so each density is $f(x\mid\lambda)=\lambda e^{-\lambda x}$ for $x\ge0$, and zero otherwise. Test $H_0:\lambda=2$ against $H_1:\lambda=1$ using the rule that rejects when $S=X_1+X_2+X_3>2$. Show why the rule uses a likelihood-ratio statistic and calculate its size and power.

Method. Under either hypothesis the support is $x_i\ge0$, and independence gives the joint density $\lambda^3e^{-\lambda S}$. Thus $$ \Lambda(X)=\frac{1^3e^{-S}}{2^3e^{-2S}} =\frac{e^S}{8}. $$ The likelihood ratio is a function of $S$ alone. Moreover, $S>2$ is equivalent to $\Lambda>e^2/8$, so the specified observation-level rule is exactly a rule based on the statistic $S$.

For three independent exponential observations with common rate $\lambda$, $S$ has a gamma distribution with shape $3$ and rate $\lambda$. Its upper-tail probability at $s>0$ is $$ P_\lambda(S>s)=e^{-\lambda s}\left(1+\lambda s+\frac{(\lambda s)^2}{2}\right). $$ This follows by integrating the gamma density $\lambda^3s^2e^{-\lambda s}/2$ from the threshold to infinity (or by two integrations by parts).

At $s=2$, under $H_0$ the product $\lambda s$ is $4$, so the size is $$ P_2(S>2)=e^{-4}\left(1+4+\frac{4^2}{2}\right)=13e^{-4}=0.2381. $$ Under $H_1$, $\lambda s=2$, so the power is $$ P_1(S>2)=e^{-2}\left(1+2+\frac{2^2}{2}\right)=5e^{-2}=0.6767. $$ Probabilities are rounded to four decimals. The check is that the same event $S>2$ was evaluated under each model, not a condition on any single observation. The common mistake is to use a one-observation exponential tail $e^{-\lambda s}$; the sum of three observations has the stated gamma tail instead.

Why the Formula Works

The preservation result can be justified directly. Let $T=T(X)$, and suppose the observation-level test is $\phi(x)=\psi(T(x))$. Under hypothesis $H_j$, the expectation of the test function is $$ E_{\theta_j}[\phi(X)] =\int \phi(x)f_j(x)\,d\nu(x) =\int \psi(T(x))f_j(x)\,d\nu(x) =E_{\theta_j}[\psi(T(X))], $$ where $\nu$ is the common dominating measure. For a discrete sample, the same argument replaces the integral with a sum. This equality applies separately under $H_0$ and $H_1$; hence it preserves both size and power, not merely one of them.

To connect this identity to the likelihood-ratio rule, suppose $\Lambda(x)=g(T(x))$. A rejection condition $\Lambda(x)>c$ becomes $g(T(x))>c$, which is a condition on $T(x)$. The same substitution works for acceptance and boundary randomization. Therefore no sample information relevant to that likelihood-ratio decision is lost.

In a common-support two-point model, this factorization also shows why the statistic is sufficient for the pair of hypotheses. Taking $h(x)=f_0(x)$, the two densities can be written as $f_0(x)=h(x)$ and $f_1(x)=h(x)g(T(x))$. The parameter-dependent factor is determined by $T$. This does not establish sufficiency for parameter values outside the pair.

Common Mistakes and Exam Traps

Claiming a reduction from a visual summary. A statistic is useful for this test only after the likelihood ratio has been shown to be a function of it.
Mixing up the direction of the ratio. If $\Lambda=f_1/f_0$, then large values favor $H_1$. Reversing the ratio reverses the direction of the critical inequality.
Dropping parameter-dependent factors. A factor may be canceled from a likelihood ratio only if it is the same under both hypotheses. The Bernoulli example's failure contribution is essential.
Ignoring support differences or ties. A zero density under one hypothesis can produce a zero or infinite ratio, and a strict boundary rule is different from a non-strict one. Randomization at equality must also be retained.
Calling every likelihood-ratio reduction globally sufficient. Capturing the ratio for two specified parameter values is a pair-specific statement; sufficiency for a whole model is a stronger claim.

Key takeaway: First establish $\Lambda(X)=g(T(X))$; then express the complete rejection rule, including any boundary treatment, in terms of $T$. Because the reduced rule makes the same decision for each sample, its size and power are exactly unchanged.

Check Your Understanding

For each problem, identify the relevant reduction and work through the requested likelihood-ratio or test calculation. Answers are included for checking.

Let $X_1,X_2,X_3$ be independent Bernoulli observations. Test $p_0=0.2$ against $p_1=0.4$. Find the statistic that determines the likelihood ratio and calculate the ratio for a sample with two successes. (Answer: $T=\sum X_i$, and $\Lambda=3.0000$.)
Two independent categorical observations have probabilities $(0.4,0.4,0.2)$ under $H_0$ and $(0.2,0.4,0.4)$ under $H_1$, for labels $A,B,C$. Express the likelihood ratio using the label counts and find its value for counts $(N_A,N_B,N_C)=(1,0,1)$. (Answer: The count vector captures the ratio, and $\Lambda=1.0000$.)
A single Poisson count has mean $\lambda$. For $H_0:\lambda=2$ versus $H_1:\lambda=5$, identify the likelihood-ratio statistic and calculate the ratio when $X=3$. (Answer: $T=X$, and $\Lambda=e^{-3}(5/2)^3=0.7779$.)
Let $X_1,X_2$ be independent exponential observations with rate $\lambda$, and test $H_0:\lambda=3$ against $H_1:\lambda=1$. Express the likelihood ratio in terms of $S=X_1+X_2$, then evaluate it at $S=1$. (Answer: $T=S$, $\Lambda=e^{2S}/9$, and at $S=1$, $\Lambda=0.8210$.)
A test rejects with probability one when $\Lambda(X)>c$, with probability $\gamma$ when $\Lambda(X)=c$, and with probability zero otherwise. If $\Lambda(X)=g(T(X))$, explain whether replacing the observation by $T$ changes size or power. (Answer: No; express all three decisions using $g(T)$, and both size and power remain exactly the same.)

What You'll Learn