Why a Plausible Formula May Not Be a Canonical Representation
A density or mass function can often be rearranged into an expression that resembles an exponential family. That resemblance is not enough. A proposed representation must use a carrier that does not change with the parameter, must account for the support correctly, and must normalize to a probability distribution. Even when these conditions hold, the listed statistics may contain redundant components, so the displayed natural parameters need not be identifiable.
The audit begins by fixing a common dominating measure: typically counting measure for a discrete law or Lebesgue measure for a continuous law. Support statements below are understood up to sets of measure zero under that measure. This matters because changing a density at a single point does not change its distribution.
The formula is only a candidate until these requirements are checked. A factor depending on \(\theta\) alone may be placed in \(A(\theta)\), but an \(x\)-dependent factor that varies with \(\theta\) cannot simply be called the carrier. Also, a representation can be valid without being minimal: redundant statistics can make different natural-parameter vectors describe the same distribution.
Three Basic Checks
First, identify the set on which the carrier is positive. Since an exponential of a finite real number is strictly positive, a representation with fixed \(h\) assigns positive density or mass precisely where \(h(x)>0\), except on null sets. Thus the family must have a common support in this measure-theoretic sense.
Second, calculate the normalizer rather than guessing it. For a proposed \(h\), \(T\), and natural parameter \(\eta\), define the partition integral or sum \(Z(\eta)=\int h(x)\exp\{\eta^\mathsf{T}T(x)\}\,d\mu(x)\). A valid representation requires \(0<Z(\eta)<\infty\) for every allowed \(\eta\), and its log-normalizer must be \(A(\eta)=\log Z(\eta)\). A finite-looking formula for \(A\) does not establish that it is correct.
Third, inspect the statistics for redundancy. If a nonzero vector \(c\) and a constant \(d\) satisfy \(c^\mathsf{T}T(x)=d\) almost everywhere on the carrier support, the components of \(T\) are affinely dependent. The representation may still be valid, but it is not minimal in those coordinates.
Worked Example: A Uniform Family With Moving Support
Let \(X\sim\operatorname{Uniform}(0,\theta)\), where \(\theta>0\). Its density with respect to Lebesgue measure is \(f_\theta(x)=\theta^{-1}\mathbf{1}_{(0,\theta)}(x)\). Can this be represented using a fixed carrier and finite-valued fixed statistics in an ordinary exponential-family form?
For \(\theta_1=1\), the density is positive on \((0,1)\) and zero outside that interval. For \(\theta_2=2\), it is positive on \((0,2)\) and zero outside. In particular, the interval \((1,2)\) has positive density under the second distribution and zero density under the first. The supports differ on a set of positive Lebesgue measure.
In a proposed canonical form with a fixed carrier \(h\), finite \(\eta(\theta)\), finite \(T(x)\), and finite \(A(\theta)\), the exponential factor is strictly positive wherever the statistics are defined. Therefore the density is positive exactly where \(h(x)>0\), for every \(\theta\). That would give the same support at \(\theta=1\) and \(\theta=2\), contradicting the densities just compared.
So this family cannot have that fixed-support representation. Writing the indicator \(\mathbf{1}_{(0,\theta)}(x)\) into a supposed carrier does not fix the problem: it makes the carrier depend on \(\theta\). The issue is the changing support, not a missing algebraic rearrangement.
Worked Example: Auditing the Carrier and Normalizer for an Exponential Law
Let \(X\) have an exponential distribution with rate \(\lambda>0\), with density \(f_\lambda(x)=\lambda e^{-\lambda x}\) for \(x>0\), and zero otherwise. A proposed form uses \(h_\lambda(x)=\lambda\), \(T(x)=x\), \(\eta(\lambda)=-\lambda\), and \(A(\lambda)=0\). Is this a valid canonical representation?
It is not in the stated form, because the proposed carrier \(h_\lambda\) depends on the parameter. Moreover, if we replace it with the fixed carrier \(h(x)=\mathbf{1}_{(0,\infty)}(x)\) but keep \(A(\lambda)=0\), the resulting function is \(e^{-\lambda x}\) on \(x>0\). Its integral is \[ \int_0^\infty e^{-\lambda x}\,dx =\left[-\frac{1}{\lambda}e^{-\lambda x}\right]_0^\infty =\frac{1}{\lambda}, \] which is not generally one.
The correct fixed-carrier representation takes \(h(x)=\mathbf{1}_{(0,\infty)}(x)\), \(T(x)=x\), and \(\eta=-\lambda\). For \(\eta<0\), \[ Z(\eta)=\int_0^\infty e^{\eta x}\,dx =\left[\frac{1}{\eta}e^{\eta x}\right]_0^\infty =-\frac{1}{\eta}. \] This is finite and positive, and therefore \(A(\eta)=\log(-1/\eta)=-\log(-\eta)\). Substitution gives \[ h(x)e^{\eta x-A(\eta)} =\mathbf{1}_{(0,\infty)}(x)e^{-\lambda x+\log\lambda} =\lambda e^{-\lambda x}\mathbf{1}_{(0,\infty)}(x), \] as required.
The factor \(\lambda\) in the original density is parameter-only, so it belongs in the normalizing term, not in a parameter-dependent carrier. The integral check confirms exactly which normalizer is needed.
Support and Normalization: Two Results for the Audit
Proof. Fix any allowed \(\theta\). If \(h(x)=0\), the displayed density is zero. If \(h(x)>0\) and the exponent is finite, then \(\exp\{\eta(\theta)^\mathsf{T}T(x)-A(\theta)\}>0\), so \(f_\theta(x)>0\). Thus the set on which \(f_\theta\) is positive agrees, up to null sets, with \(\{x:h(x)>0\}\). Since this set does not depend on \(\theta\), the supports agree across the family. \(\square\)
This argument gives a quick impossibility test: if two members have supports that differ on a set of positive measure, no representation satisfying these fixed-carrier and finite-exponent conditions can describe both. It also explains why moving a parameter-dependent indicator into \(h\) violates the definition rather than solving the problem.
Derivation. Integrating the proposed density over the sample space gives \[ 1=\int h(x)\exp\{\eta^\mathsf{T}T(x)-A(\eta)\}\,d\mu(x) =e^{-A(\eta)}\int h(x)\exp\{\eta^\mathsf{T}T(x)\}\,d\mu(x). \] Write the integral on the right as \(Z(\eta)\). If \(Z(\eta)\) is positive and finite, then \(e^{A(\eta)}=Z(\eta)\), and taking logarithms gives \(A(\eta)=\log Z(\eta)\). If \(Z(\eta)\) is zero or infinite, no finite \(A(\eta)\) can make the expression a normalized density. \(\square\)
Redundant Statistics: A Valid Form That Is Not Identifiable
Worked Example: A Redundant Bernoulli Representation
Let \(X\sim\operatorname{Bernoulli}(p)\), with \(0<p<1\). Since \(x\) is either 0 or 1, its mass function can be written as \(f_p(x)=p^x(1-p)^{1-x}\). Set \(T(x)=(x,1-x)^\mathsf{T}\), \(\eta(p)=(\log p,\log(1-p))^\mathsf{T}\), and \(h(x)=1\) on \(\{0,1\}\). With \(A(p)=0\), the proposed expression is exactly the Bernoulli mass function.
It is normalized: the value at \(x=1\) is \(p\), and the value at \(x=0\) is \(1-p\), so their sum is \(p+(1-p)=1\). Equivalently, the partition sum at a general \(\eta=(\eta_1,\eta_2)^\mathsf{T}\) is \(Z(\eta)=e^{\eta_1}+e^{\eta_2}\), and at the proposed \(\eta(p)\) this is \(p+(1-p)=1\).
But \(T_1(x)+T_2(x)=1\) for both possible values of \(x\). Thus the two statistics are affinely dependent. The general partition function is \(A(\eta)=\log(e^{\eta_1}+e^{\eta_2})\), and the probability of \(X=1\) is \(e^{\eta_1}/(e^{\eta_1}+e^{\eta_2})\). Adding the same constant \(c\) to both coordinates leaves that probability unchanged: \[ \frac{e^{\eta_1+c}}{e^{\eta_1+c}+e^{\eta_2+c}} =\frac{e^{\eta_1}}{e^{\eta_1}+e^{\eta_2}}. \] Therefore the natural-parameter vector is not identifiable in this redundant two-statistic form.
A minimal representation uses \(T(x)=x\), \(\eta=\log\{p/(1-p)\}\), and \(A(\eta)=\log(1+e^\eta)\). This removes the redundant statistic while preserving the Bernoulli law.
How to Diagnose Redundancy
The Bernoulli example illustrates a general test. Suppose \(c\neq 0\) and \(c^\mathsf{T}T(x)=d\) almost everywhere on the carrier support, for some constant \(d\). For any two natural-parameter vectors \(\eta\) and \(\eta+sc\) in the natural-parameter domain, the unnormalized kernel changes by the constant factor \(e^{sd}\): \[ h(x)e^{(\eta+sc)^\mathsf{T}T(x)} =e^{sd}h(x)e^{\eta^\mathsf{T}T(x)}. \] Consequently, its partition function changes by that same factor, \(Z(\eta+sc)=e^{sd}Z(\eta)\). After normalization, the factor cancels, so both parameter vectors give the same distribution. The statistics are redundant, even though the representation may be perfectly normalized.
Do not confuse this with a changing support or an incorrect normalizer. Those defects invalidate the proposed representation. Redundancy instead means the representation is valid but carries more statistic coordinates than are needed, and natural-parameter coordinates may fail to identify distinct distributions.
A Practical Audit
State whether the density is with respect to Lebesgue measure, counting measure, or another common measure; compare supports up to null sets.
Check that the proposed \(h(x)\) is independent of \(\theta\). Put parameter-only multiplicative factors into the normalizer, not into \(h\).
Verify that the density or mass is positive on the same set for every allowed parameter. A varying cutoff is a warning that a fixed-support form is impossible.
Check that it is positive and finite, then set \(A\) equal to its logarithm. Substitute back to verify normalization.
Look for a nontrivial linear combination of \(T(x)\) that is constant almost everywhere. If one exists, distinguish validity from minimality and identifiability.
Common Mistakes
- Putting a parameter-dependent indicator in the carrier. An indicator such as \(\mathbf{1}_{(0,\theta)}(x)\) changes with \(\theta\); it cannot serve as a fixed carrier in this form.
- Assuming an algebraic factorization is normalized. The normalizer must be derived by integrating or summing the kernel. A guessed \(A\) can leave total mass different from one.
- Treating redundancy as invalidity. The two-statistic Bernoulli form is normalized and represents the right family, but its statistics are redundant. Validity and minimality are separate checks.
- Confusing parameter-only and data-dependent factors. A factor depending only on \(\theta\) can be absorbed into \(A(\theta)\). A factor that depends jointly on \(x\) and \(\theta\) may change the carrier or support and requires further examination.
Check Your Understanding
For each question, separate representation validity from minimality.
- Why does a fixed carrier with a finite exponent imply a common support up to null sets?
- For the exponential density with rate \(\lambda\), what integral determines the log-normalizer when \(T(x)=x\) and \(\eta=-\lambda\)?
- Why can a parameter-only factor be absorbed into the normalizer, while a parameter-dependent support indicator cannot be placed in a fixed carrier?
- In the two-statistic Bernoulli representation, state the affine relation between the statistics and explain its consequence for natural-parameter identifiability.
- A proposed kernel has partition integral zero or infinity for an allowed parameter value. Can a finite log-normalizer make it a probability density? Explain.