Complete Sufficient Statistics in Exponential Families

Why Completeness Matters in Exponential Families

A sufficient statistic retains all the sample information about a parameter that is relevant to inference. Completeness is a further property: it rules out nonzero functions of that statistic whose expected value is zero for every parameter value. For exam problems, the exponential-family form often makes it possible to establish both properties systematically, rather than attempting to check completeness from its definition for every possible function.

The key condition is that the family has a full-rank canonical representation and its natural parameter space contains an open set in the appropriate Euclidean space. The openness condition matters: varying the natural parameter through a neighborhood supplies enough distinct exponential weightings to force a function with zero expectation everywhere to vanish almost surely.

Definition: A statistic $T$ is complete for a family of distributions if, for every measurable function $g$ for which $E_\eta[|g(T)|]$ is finite for every parameter value $\eta$, the condition $$ E_\eta[g(T)]=0\quad\text{for every }\eta $$ implies $P_\eta(g(T)=0)=1$ for every $\eta$. It is sufficient if the conditional distribution of the sample given $T$ does not depend on the parameter.

For a full-rank $k$-parameter exponential family, a standard canonical representation is

f_\eta(x)=h(x)\exp\{\eta^{\mathsf T}T(x)-A(\eta)\},\qquad \eta\in\Theta.

Here the support of $x$ is fixed across parameter values, $T(x)$ is a $k$-component statistic, $\eta$ is the natural parameter, and $A(\eta)$ normalizes the density or probability mass function. The natural parameter space is the set of $\eta$ for which the normalizing integral or sum is finite. Full rank means that no nonzero linear combination of the components of $T(x)$ is constant almost everywhere on the support. In this tutorial, the condition for the completeness result is that this full natural parameter space is open in $\mathbb R^k$.

Theorem: In a full-rank exponential family with fixed support and an open natural parameter space $\Theta\subseteq\mathbb R^k$, the statistic $T$ in the canonical representation is sufficient and complete. The theorem applies to the stated representation and conditions; it should not be used merely because a likelihood contains an exponential term. Check the support, rank, and openness conditions.

Why an Open Natural Parameter Space Gives Completeness

Sufficiency follows directly from the factorization: the sample enters the parameter-dependent part of the joint density or mass function only through $T(x)$. For completeness, consider a function $g$ such that $E_\eta[g(T)]=0$ for every $\eta\in\Theta$. Define $\nu$ as the measure on statistic values induced by $h(x)$ and the underlying sample-space measure. The expectation condition becomes

0=e^{A(\eta)}E_\eta[g(T)] =\int g(t)e^{\eta^{\mathsf T}t}\,d\nu(t), \qquad \eta\in\Theta.

Thus the Laplace transform of the signed measure with density $g(t)$ relative to $\nu$ vanishes throughout an open set. Integrability under every parameter value ensures that this transform is finite in a neighborhood of each point in $\Theta$. Uniqueness of the Laplace transform then implies that the signed measure is zero. Consequently, $g(t)=0$ for $\nu$-almost every $t$, which means $g(T)=0$ almost surely under every member of the family. This is the reason the open-set condition is useful: it gives a neighborhood of parameter values, not just a single exponential weighting.

Write the joint model in canonical form.
Identify $h(x)$, $T(x)$, $\eta$, and $A(\eta)$, and verify that the support does not depend on the parameter.

Check rank and the natural parameter space.
Confirm that the statistic components are not linearly redundant and that the allowed natural parameters form an open set in $\mathbb R^k$.

Apply the theorem to $T$.
Under these conditions, $T$ is both sufficient and complete. Keep the full vector $T$; a component alone need not be sufficient.

Poisson Samples

Suppose $X_1,\ldots,X_n$ are independent and identically distributed with $X_i\sim\operatorname{Poisson}(\lambda)$, where $\lambda>0$. Their joint mass function is

\begin{aligned} P_\lambda(X_1=x_1,\ldots,X_n=x_n) &=\prod_{i=1}^n\frac{e^{-\lambda}\lambda^{x_i}}{x_i!}\\ &=\left(\prod_{i=1}^n\frac{1}{x_i!}\right) \exp\left\{(\log\lambda)\sum_{i=1}^n x_i-n\lambda\right\}. \end{aligned}

The natural parameter is $\eta=\log\lambda$, which ranges over all of $\mathbb R$, an open set. The statistic is $T=\sum_{i=1}^n X_i$, and $A(\eta)=ne^\eta$. The family is full rank, so the theorem says that the sum is complete and sufficient for $\lambda$.

Worked Example: A Poisson Count Total

A hypothetical monitoring system records independent daily event counts $X_1,\ldots,X_5$, each with distribution $\operatorname{Poisson}(\lambda)$, where the unknown mean event count $\lambda>0$ is constant across days. The observations are $2,0,3,1,4$. Identify the complete sufficient statistic and verify the conditions.

Step 1: Calculate the statistic. The sum is $T=2+0+3+1+4=10$. There are $n=5$ observations, so the joint likelihood is $$ \begin{aligned} L(\lambda) &=\prod_{i=1}^5\frac{e^{-\lambda}\lambda^{x_i}}{x_i!}\\ &=\frac{e^{-5\lambda}\lambda^{10}}{2!\,0!\,3!\,1!\,4!} =\frac{e^{-5\lambda}\lambda^{10}}{2\cdot1\cdot6\cdot1\cdot24} =\frac{e^{-5\lambda}\lambda^{10}}{288}. \end{aligned} $$

Step 2: Check the canonical conditions. The natural parameter is $\eta=\log\lambda$, so $\eta\in\mathbb R$, which is open. The statistic is $T=\sum X_i$; the parameter-dependent part of the likelihood is $e^{-5\lambda}\lambda^T$. The support consists of nonnegative integer count vectors and does not depend on $\lambda$.

Conclusion. The full-rank exponential-family theorem applies, so $T=10$ is the observed value of a complete sufficient statistic for $\lambda$. The common mistake to avoid is calling the sample average a different sufficient statistic without explaining the equivalence: here $\bar X=T/5$, a one-to-one rescaling of $T$, also retains exactly the same information.

Normal Samples with Known Variance

Now suppose $X_1,\ldots,X_n$ are independent with $X_i\sim N(\mu,\sigma^2)$, where $\sigma^2$ is known and positive and $\mu\in\mathbb R$ is unknown. Expanding the normal likelihood gives

\prod_{i=1}^n f(x_i) =(2\pi\sigma^2)^{-n/2} \exp\left\{ \frac{\mu}{\sigma^2}\sum_{i=1}^n x_i -\frac{1}{2\sigma^2}\sum_{i=1}^n x_i^2 -\frac{n\mu^2}{2\sigma^2} \right\}.

Since the variance is fixed, the term involving $\sum x_i^2$ can be absorbed into $h(x)$. The natural parameter is $\eta=\mu/\sigma^2$, with $\eta\in\mathbb R$, and the statistic is $T=\sum X_i$. Therefore $T$ is complete and sufficient for $\mu$. Equivalently, $\bar X=T/n$ is complete and sufficient because it is a one-to-one transformation of $T$.

Worked Example: Known-Variance Normal Data

A hypothetical laboratory measures a quantity on five independent specimens. Assume each measurement has distribution $N(\mu,4)$, where the variance is known to be $4$ and the mean $\mu$ is unknown. The recorded values are $8.1, 7.4, 9.0, 8.5,$ and $7.0$. Find the observed complete sufficient statistic and check the natural parameter condition.

Step 1: Add the observations. The sum is $8.1+7.4=15.5$, $15.5+9.0=24.5$, $24.5+8.5=33.0$, and $33.0+7.0=40.0$. Thus $T=\sum X_i=40.0$, and $\bar X=40.0/5=8.0$.

Step 2: Express the natural parameter. With $\sigma^2=4$, the natural parameter is $\eta=\mu/4$. Since $\mu$ can be any real number, $\eta$ ranges over $\mathbb R$, which is open. The joint support is $\mathbb R^5$, regardless of $\mu$.

Conclusion. The statistic $T=40.0$ is the observed value of a complete sufficient statistic for $\mu$; the equivalent sample mean is $8.0$. The method uses the known-variance model. The common mistake is to use the two-parameter normal statistic in a model where the variance is known: the squared-observation term can be put into the parameter-free factor here, so a second statistic is not required.

Normal Samples with Both Parameters Unknown

If both the mean and variance are unknown, the squared-observation term is no longer parameter-free. For independent $X_i\sim N(\mu,\sigma^2)$, with $\mu\in\mathbb R$ and $\sigma^2>0$, the likelihood can be written

\begin{aligned} \prod_{i=1}^n f(x_i) &=(2\pi)^{-n/2} \exp\left\{ \frac{\mu}{\sigma^2}\sum_{i=1}^n x_i -\frac{1}{2\sigma^2}\sum_{i=1}^n x_i^2 -\frac{n\mu^2}{2\sigma^2} -\frac n2\log\sigma^2 \right\}. \end{aligned}

The natural statistic is $T=(\sum X_i,\sum X_i^2)$, with natural parameters $\eta_1=\mu/\sigma^2$ and $\eta_2=-1/(2\sigma^2)$. As $\mu$ ranges over $\mathbb R$ and $\sigma^2>0$, the natural parameter space is

\Theta=\{(\eta_1,\eta_2):\eta_1\in\mathbb R,\ \eta_2<0\}.

This is an open subset of $\mathbb R^2$. The two statistic components are full rank, so $T$ is complete and sufficient for $(\mu,\sigma^2)$. For $n\geq2$, the pair is equivalent to $(\bar X,S^2)$, where $S^2=\sum_{i=1}^n(X_i-\bar X)^2/(n-1)$: knowing $\bar X$ and $S^2$ determines both sums, and conversely.

Worked Example: Both Normal Parameters Unknown

A hypothetical calibration study produces four independent readings $1,3,4,8$, modeled as $N(\mu,\sigma^2)$ with both $\mu\in\mathbb R$ and $\sigma^2>0$ unknown. Find the observed natural statistic and its equivalent mean-and-variance representation, then check whether the completeness theorem applies.

Step 1: Compute both components. The sum is $1+3+4+8=16$. The sum of squares is $1^2+3^2+4^2+8^2=1+9+16+64=90$. Thus the observed statistic is $T=(16,90)$, and $\bar X=16/4=4$.

Step 2: Calculate the sample variance. The sum of squared deviations is $$ \sum_{i=1}^4(x_i-\bar x)^2 =(1-4)^2+(3-4)^2+(4-4)^2+(8-4)^2 =9+1+0+16=26. $$ Therefore $S^2=26/(4-1)=26/3$.

Step 3: Verify the family conditions. The natural parameter values are $(\eta_1,\eta_2)=(\mu/\sigma^2,-1/(2\sigma^2))$, giving $\eta_1\in\mathbb R$ and $\eta_2<0$. This half-plane is open. The joint support is all of $\mathbb R^4$, independent of the parameter, and the sum and sum of squares are not linearly redundant.

Conclusion. The observed complete sufficient statistic is $(16,90)$, equivalently $(4,26/3)$. The common mistake is to retain only $S^2$: with both parameters unknown, the likelihood depends on the data through both the sum and the sum of squares. Omitting the sum loses information about $\mu$.

Common Traps and Practical Checks

Completeness is a property of a statistic across the whole parameter family, not a property inferred from one observed sample or from a statistic's name. Apply the theorem only after identifying the model correctly.

Checking only sufficiency. Factorization establishes sufficiency, but completeness here comes from the additional full-rank and open-natural-parameter-space conditions.
Using the wrong parameter space. Check openness in natural-parameter coordinates, not just in the original parameters. For the unknown-variance normal model, the natural parameter set is $\eta_2<0$, not all of $\mathbb R^2$.
Dropping a statistic component. With unknown normal mean and variance, both $\sum X_i$ and $\sum X_i^2$ appear with parameter-dependent coefficients. A single component generally cannot replace the pair.
Assuming every exponential-looking model qualifies. A restricted parameter space may fail the open-set condition, or the chosen statistic may be redundant. Verify the theorem's assumptions before applying its conclusion.
Confusing completeness with zero variability. Completeness does not mean that $T$ is constant. It means that no nonzero integrable function of $T$ has expectation zero for every parameter value.

Key takeaway: Write the full sample model in canonical form, identify the natural statistic and natural parameter space, and check fixed support, full rank, and openness. When those conditions hold, the natural statistic is complete and sufficient.

Check Your Understanding

For each model, identify the relevant statistic and use the stated conditions to determine whether the completeness theorem applies.

Four independent observations have distribution $\operatorname{Poisson}(\lambda)$, $\lambda>0$, and their values are $1,2,0,5$. Give the observed natural statistic and natural parameter space. (Answer: $T=8$; $\eta=\log\lambda\in\mathbb R$; the sum is complete and sufficient.)
Six independent observations have distribution $N(\mu,9)$, with known variance $9$, and values $7,8,9,10,11,9$. Compute the observed sample mean and state a complete sufficient statistic. (Answer: $\bar X=54/6=9$; $T=\sum X_i=54$, or equivalently $\bar X=9$, is complete and sufficient.)
Four independent observations have distribution $N(\mu,\sigma^2)$ with both parameters unknown; the values are $-1,0,2,5$. Compute $\sum X_i$, $\sum X_i^2$, and $S^2$, and state whether the natural parameter space is open. (Answer: $(\sum X_i,\sum X_i^2)=(6,30)$; $\bar X=1.5$, $S^2=(30-36/4)/3=7$; the natural parameter space $\{(\eta_1,\eta_2):\eta_1\in\mathbb R,\eta_2<0\}$ is open, so the pair is complete and sufficient.)
Independent counts satisfy $X_i\sim\operatorname{Poisson}(c_i\theta)$, where $(c_1,c_2,c_3)=(1,2,3)$ are known and $\theta>0$ is unknown. The observations are $2,1,4$. Write the natural parameter and statistic, and check whether the theorem applies. (Answer: $\eta=\log\theta\in\mathbb R$, $T=\sum X_i=7$, and $A(\eta)=6e^\eta$; the support is fixed and the statistic is full rank, so $T$ is complete and sufficient.)
For a normal sample with unknown mean and variance, a proposed statistic is only $\sum_{i=1}^n X_i^2$. Explain whether the full-rank exponential-family theorem establishes this statistic as complete sufficient for the pair of parameters. (Answer: No. The likelihood also depends on $\sum X_i$, so the required natural statistic is $(\sum X_i,\sum X_i^2)$; the proposed one-component statistic is not sufficient for the pair.)

What You'll Learn