Parameter-Dependent Support and Other Factorization Traps

Why Factorizations Fail

Writing a density in exponential-family form is not just a matter of rearranging symbols until an exponential appears. The representation has structural requirements: the reference measure and carrier term must be fixed with respect to the parameter being modeled, and all parameter dependence must be accounted for in the natural parameter and log-partition terms. A density can look close to the desired form and still fail because its support changes with the parameter or because a parameter has been hidden in the carrier.

These distinctions matter when deciding whether a distribution belongs to an ordinary exponential family. They also prevent a common algebraic error: treating any term that does not multiply the observed response as if it were part of a parameter-free carrier.

The Fixed-Carrier Requirement

Definition: A one-parameter density or mass function is in fixed-carrier exponential-family form, with respect to a fixed measure $\mu$, if it can be written $$ f_\theta(y)=h(y)\exp\{\eta(\theta)T(y)-A(\theta)\}, $$ where $h$ and $T$ do not depend on $\theta$, $\eta(\theta)$ and $A(\theta)$ are finite for each allowed $\theta$, and the expression is a density or mass function with respect to $\mu$. The measure $\mu$ is also fixed with respect to $\theta$. For a vector statistic, replace $\eta(\theta)T(y)$ by the inner product $\eta(\theta)^\mathsf{T}T(y)$.

The carrier $h(y)$ may vanish, but the exponential factor is strictly positive wherever its arguments are finite. Therefore the set on which the density is positive, up to $\mu$-null sets, is determined by $h$, not by $\theta$. In a dispersion-scaled representation, a carrier may depend on a separate dispersion parameter; the requirement is that it not depend on the parameter whose natural-parameter factorization is being asserted.

Theorem: In a fixed-carrier exponential-family representation with finite natural and log-partition parameters, all distributions in the family have the same null sets. In particular, their densities are positive on the same set, up to null sets of the fixed reference measure.

Proof. Let $H=\{y:h(y)>0\}$. For any allowed $\theta$, the factor $\exp\{\eta(\theta)T(y)-A(\theta)\}$ is positive and finite wherever $T(y)$ is defined and finite. Thus $f_\theta(y)>0$ precisely on $H$, apart from any $\mu$-null exceptional set. Consequently, a measurable set has probability zero under one member exactly when its intersection with $H$ has $\mu$-measure zero; that condition does not depend on $\theta$. Hence all members have the same null sets. $\square$

This is an immediate diagnostic for changing support. It rules out an ordinary fixed-carrier representation when two parameter values assign probability zero and positive probability, respectively, to the same measurable set.

Worked Example: Why the Uniform Support Cannot Be Hidden

For $\theta>0$, consider $X\sim\operatorname{Uniform}(0,\theta)$, whose density with respect to Lebesgue measure is $$ f_\theta(y)=\frac{1}{\theta}\mathbf{1}_{\{0<y<\theta\}}. $$

Choose $\theta_1=1$ and $\theta_2=2$, and let $B=(1,2)$. Under $\theta_1$, $P_{\theta_1}(X\in B)=0$, because the support ends at 1. Under $\theta_2$, $$ P_{\theta_2}(X\in B)=\int_1^2 \frac{1}{2}\,dy=\frac{1}{2}. $$

The two distributions therefore have different null sets. By the theorem, they cannot both have the stated fixed-carrier form with a common reference measure and finite natural and log-partition parameters. In the displayed density, the indicator $\mathbf{1}_{\{0<y<\theta\}}$ is the obstruction: it depends jointly on the observation and the parameter, so it cannot be put into a parameter-free carrier $h(y)$.

One might try to absorb the indicator into $h$, but then $h$ would have to vanish for $y\geq\theta$, making it depend on $\theta$. One might instead put it into the exponential, but a zero indicator cannot be produced by the exponential of a finite real number. This family can be studied as a statistical model, but not in this ordinary fixed-carrier form.

Parameter Terms Must Be Put in the Right Place

For a valid representation, the carrier is a function of the observation alone (or, in a dispersion-scaled form, may also involve the designated dispersion parameter). Terms depending only on the model parameter belong in the log-partition term, while terms involving both the parameter and the observation must be represented through the natural parameter multiplying a fixed statistic, or through a specified vector of such statistics.

A useful necessary check comes from taking the ratio of two densities in a one-statistic family. Where both densities are positive, $$ \log\frac{f_{\theta_1}(y)}{f_{\theta_0}(y)} = \{\eta(\theta_1)-\eta(\theta_0)\}T(y) -\{A(\theta_1)-A(\theta_0)\}. $$ Thus the log density ratio must be an affine function of the same fixed statistic $T(y)$. This check can expose a parameter mistakenly placed in the carrier, or a proposed statistic that changes with the parameter.

Result: If $f_\theta(y)=h(y)\exp\{\eta(\theta)T(y)-A(\theta)\}$, then every pair $\theta_0,\theta_1$ has a log density ratio affine in $T(y)$. This is a necessary diagnostic, not by itself a complete proof that a proposed family has a valid exponential-family representation.

Derivation. On the common positive set, take logarithms at each parameter value: $$ \log f_{\theta_j}(y)=\log h(y)+\eta(\theta_j)T(y)-A(\theta_j),\qquad j=0,1. $$ Subtract the equation for $\theta_0$ from the equation for $\theta_1$. The $\log h(y)$ terms cancel because the carrier is the same at both parameter values, leaving the displayed ratio identity. $\square$

Worked Example: Correctly Sorting Terms in a Normal Density

Let $X\sim N(\mu,\sigma^2)$, where $\sigma^2$ is known and $\mu$ varies. Start with the density and expand the square: $$ f_\mu(y)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(y-\mu)^2}{2\sigma^2}\right\} =\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{y^2}{2\sigma^2}+\frac{\mu y}{\sigma^2}-\frac{\mu^2}{2\sigma^2}\right\}. $$

The term involving both $y$ and $\mu$ is $\mu y/\sigma^2$, so take $T(y)=y$ and $\eta(\mu)=\mu/\sigma^2$. The parameter-only term is $-\mu^2/(2\sigma^2)$, so $A(\mu)=\mu^2/(2\sigma^2)$. The remaining terms give $$ h(y)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{y^2}{2\sigma^2}\right\}. $$ This $h$ is independent of $\mu$, and the factorization is valid.

The ratio check confirms the sorting. For any $\mu_0,\mu_1$, $$ \log\frac{f_{\mu_1}(y)}{f_{\mu_0}(y)} =\frac{\mu_1-\mu_0}{\sigma^2}y -\frac{\mu_1^2-\mu_0^2}{2\sigma^2}, $$ which is affine in the fixed statistic $y$. If instead the factor $\exp\{-\mu^2/(2\sigma^2)\}$ were put into $h$, the proposed carrier would depend on $\mu$. It would no longer be a valid fixed carrier, even though the numerical density would be unchanged.

A Valid Parameter-Only Term: Exponential Density

Worked Example: The Exponential Rate Belongs in the Log-Partition

For a rate parameter $\lambda>0$, the exponential density is $$ f_\lambda(y)=\lambda e^{-\lambda y}\mathbf{1}_{\{y>0\}}. $$ The support $\{y>0\}$ does not depend on $\lambda$, so it can be included in a fixed carrier. Set $h(y)=\mathbf{1}_{\{y>0\}}$, $T(y)=y$, and $\eta(\lambda)=-\lambda$. With $A(\eta)=-\log(-\eta)$, the exponential-family expression becomes $$ h(y)\exp\{\eta y-A(\eta)\} =\mathbf{1}_{\{y>0\}}\exp\{-\lambda y+\log\lambda\} =\lambda e^{-\lambda y}\mathbf{1}_{\{y>0\}}. $$

To check the normalizing term, $\eta=-\lambda<0$, and $$ \int_0^\infty e^{\eta y}\,dy=-\frac{1}{\eta}=\exp\{-\log(-\eta)\}=\exp\{A(\eta)\}. $$ Thus $A$ supplies precisely the normalization needed to make the density integrate to one. The factor $\lambda$ is parameter-dependent, so it cannot be assigned to $h$; it is correctly accounted for by $-A(\eta)=\log\lambda$.

Common Factorization Traps

Several quick checks help separate a genuine factorization from an algebraic rearrangement that only looks convincing.

Putting changing support in the carrier. A carrier cannot include an indicator whose boundary depends on the modeled parameter. Compare the distributions’ null sets instead.
Calling every parameter-only term a carrier. A parameter-only term may be essential for normalization. It belongs in the log-partition term, as the rate factor does for the exponential density.
Letting the statistic change with the parameter. The statistic $T(y)$ must be fixed across the family. Otherwise the log-ratio diagnostic is not a comparison using one common statistic.
Assuming algebra alone establishes the representation. Check that the carrier and reference measure are parameter-free in the relevant sense, the support is compatible with them, and the expression is normalized for every allowed parameter.

The main practical lesson is to inspect the support before collecting terms, then sort the remaining terms by what they depend on. A common support makes a fixed carrier possible; a valid representation still requires every parameter-dependent term to be accounted for in the natural-parameter and log-partition components. The density-ratio check provides an additional necessary test when the proposed form uses a single statistic.

Check Your Understanding

Use the fixed-carrier requirement and the density-ratio diagnostic to answer these questions.

For $\operatorname{Uniform}(0,\theta)$, identify a measurable set whose probability is zero for one parameter value and positive for another.
In the normal example, why is the factor involving $\mu^2$ not part of the carrier?
For a one-statistic exponential family, what functional form must the log ratio of two densities have as a function of the observation?
In the exponential-rate example, identify the natural parameter and log-partition function, and verify the allowed domain of the natural parameter.
Does a common support alone prove that a proposed factorization is valid? State at least one other condition that must be checked.