Parameter-Dependent Support and Nonregular Families

Why Support Matters

The usual exponential-family form separates dependence on the observation from dependence on the parameter. That separation is more than an algebraic convenience: with a fixed carrier measure and finite canonical parameters, it forces all family members to have the same support, up to sets of carrier measure zero. This makes the Uniform(0, θ) family a useful diagnostic example. Its density is simple and normalized for every θ > 0, yet its support changes with θ.

The point is not that every family with changing support is impossible to write in some exponential-looking notation. One can always display a parameter-dependent indicator. The issue is whether the family has the usual fixed-carrier exponential-family representation, with a carrier and base factor that do not change with the parameter. We will also distinguish that structural issue from two separate questions: whether a density can be normalized, and whether it is differentiable in its parameter.

Fixed-Carrier Exponential Families

Definition: A family of distributions has a usual fixed-carrier exponential-family representation if it has densities, with respect to one parameter-independent σ-finite measure μ, of the form

p_\theta(x)=h(x)\exp\{\eta(\theta)^{\mathsf T}T(x)-A(\theta)\}.

Here \(h\) and the statistic \(T\) are parameter-independent measurable functions, \(h\geq 0\), and the entries of \(\eta(\theta)\), \(T(x)\), and \(A(\theta)\) are finite wherever the density is positive. The density is understood up to μ-null sets.

The exponential factor is strictly positive wherever it is finite. Consequently, the only places where the density can vanish are determined by the fixed function \(h\), not by θ. A parameter-dependent indicator such as \(1\{0\leq x\leq\theta\}\) cannot be hidden inside this form while keeping \(h\), \(T\), and finite \(\eta(\theta)\) parameter-independent in the required way.

Theorem: In a usual fixed-carrier exponential family, any two distributions are mutually absolutely continuous. That is, if \(P_{\theta_1}(B)=0\), then \(P_{\theta_2}(B)=0\), and conversely, for every measurable set \(B\).

Proof. Let \(S=\{x:h(x)>0\}\), ignoring any μ-null set on which the functions in the representation are not finite. For every θ, \(p_\theta(x)>0\) on \(S\) and \(p_\theta(x)=0\) off \(S\), up to μ-null sets. If \(P_{\theta_1}(B)=\int_B p_{\theta_1}\,d\mu=0\), nonnegativity and strict positivity of \(p_{\theta_1}\) on \(S\) imply \(\mu(B\cap S)=0\). Since \(p_{\theta_2}=0\) off \(S\) and is finite μ-almost everywhere on \(S\), this gives \(\int_B p_{\theta_2}\,d\mu=0\). Interchanging \(\theta_1\) and \(\theta_2\) proves the reverse implication. \(\square\)

Mutual absolute continuity is a precise way to express the common-support consequence without relying on a particular choice of topological support. It is also unchanged if the common carrier is replaced by an equivalent carrier measure. A representation that allows an indicator depending on θ, or parameter values that make the exponential factor vanish through infinite exponents, is outside the usual fixed-carrier form just defined.

Worked Examples

Worked Example: Uniform(0, θ) and the Common-Support Test

For θ > 0, take the density \(f_\theta(x)=\theta^{-1}1_{[0,\theta]}(x)\) with respect to Lebesgue measure. The endpoint convention does not affect the distribution. Each density integrates to one because

\int_{\mathbb R} f_\theta(x)\,dx =\int_0^\theta \frac{1}{\theta}\,dx =\frac{\theta}{\theta}=1.

Choose \(0<\theta_1<\theta_2\) and the interval \(B=(\theta_1,\theta_2)\). Under \(P_{\theta_1}\), this interval lies outside the support, so \(P_{\theta_1}(B)=0\). Under \(P_{\theta_2}\),

P_{\theta_2}(B) =\int_{\theta_1}^{\theta_2}\frac{1}{\theta_2}\,dx =\frac{\theta_2-\theta_1}{\theta_2}>0.

Thus \(P_{\theta_2}\) is not absolutely continuous with respect to \(P_{\theta_1}\). The theorem shows that Uniform(0, θ) cannot have a usual fixed-carrier exponential-family representation. In its direct density, the parameter-dependent indicator makes the obstruction visible; the probability comparison proves that no alternative fixed-carrier representation can remove it.

Worked Example: The Likelihood for a Uniform Sample

Let \(X_1,\ldots,X_n\) be independent observations from Uniform(0, θ), with θ > 0, and suppose the observed values are \(x_1,\ldots,x_n\), all nonnegative. Their joint density, viewed as a function of θ, is

L(\theta) =\prod_{i=1}^n \frac{1}{\theta}1_{[0,\theta]}(x_i) =\theta^{-n}1\{\theta\geq M\}, \qquad M=\max_{1\leq i\leq n}x_i.

For θ below \(M\), at least one observation is outside the allowed interval and the likelihood is zero. For θ at least \(M\), the likelihood is \(\theta^{-n}\), which decreases as θ increases: its derivative is \(-n\theta^{-n-1}<0\). Therefore the maximum occurs at the smallest allowed value, \(\hat{\theta}=M\), when \(M>0\). The parameter constraint in the likelihood depends on the observed sample through its maximum. This is the likelihood-level signature of the changing support; it is not an ordinary fixed-support exponential-family factorization in which all observations remain admissible throughout the parameter space.

Support Dependence Is Not an Integrability Failure

A family can have fixed support but be normalizable only for a restricted set of parameter values. This is the kind of issue addressed by integrability and the natural parameter space: the support stays put, while some parameter choices make the normalizing integral infinite. Conversely, the Uniform(0, θ) density integrates to one for every θ > 0, but its support changes. These are logically distinct properties.

Worked Example: Fixed Support With a Restricted Integrability Range

Consider the kernel \(g_\eta(x)=e^{\eta x}1\{x>0\}\), with respect to Lebesgue measure. Its support \((0,\infty)\) does not depend on η. For η < 0, its integral is

\int_0^\infty e^{\eta x}\,dx =\left[\frac{e^{\eta x}}{\eta}\right]_{0}^{\infty} =-\frac{1}{\eta}<\infty.

For η = 0 the integral is \(\int_0^\infty 1\,dx=\infty\). For η > 0, \(e^{\eta x}\geq 1\) for \(x\geq0\), so the integral is also infinite. Thus only η < 0 gives a normalizable density, namely \((-\eta)e^{\eta x}1\{x>0\}\). The support is fixed throughout; the restriction comes solely from integrability. This contrasts directly with Uniform(0, θ), where normalization succeeds but the support varies.

Moving Support and Differentiation

Support dependence can also invalidate a familiar differentiation argument. In a regular fixed-support model, under suitable domination conditions, one may differentiate a density under the integral sign to obtain a score with mean zero. The conclusion depends on those conditions. For a moving boundary, differentiating only the density at fixed interior points misses a boundary contribution.

Result: For Uniform(0, θ), the pointwise score at an interior observation \(0<x<\theta\) is \(\partial_\theta\log f_\theta(x)=-1/\theta\), so its expectation is \(-1/\theta\), not zero. The usual zero-mean score identity does not apply because the support moves with θ.

Derivation. For fixed \(x\) strictly between 0 and θ, \(\log f_\theta(x)=-\log\theta\), and hence \(\partial_\theta\log f_\theta(x)=-1/\theta\). Since this value holds with probability one under Uniform(0, θ), its expectation is also \(-1/\theta\). To see where the usual argument fails, differentiate the normalization integral with its moving upper limit:

\frac{d}{d\theta}\int_0^\theta \frac{1}{\theta}\,dx =\int_0^\theta -\frac{1}{\theta^2}\,dx+\frac{1}{\theta} =-\frac{1}{\theta}+\frac{1}{\theta} =0.

The last term is the boundary contribution from differentiating the upper limit. If one differentiates the density at each fixed interior \(x\) and then integrates that derivative, one gets only \(\int_0^\theta -\theta^{-2}\,dx=-1/\theta\); the missing boundary term explains the disagreement. Thus normalization itself is not failing. Rather, the regularity conditions that justify interchanging differentiation and integration without a moving-boundary term are not satisfied.

Worked Example: Fixed Support Does Not Guarantee Differentiability

To separate differentiability from support dependence, define a density on \(0<x<1\) by

f_\theta(x)=1+|\theta|(2x-1), \qquad -1<\theta<1.

The support is fixed. Since \(-1<2x-1<1\), the density is bounded below by \(1-|\theta|>0\). Its integral is

\int_0^1 f_\theta(x)\,dx =1+|\theta|\int_0^1(2x-1)\,dx =1+|\theta|(1-1)=1.

For a fixed \(x\neq 1/2\), the difference quotient at θ = 0 is

\frac{f_h(x)-f_0(x)}{h} =\frac{|h|}{h}(2x-1).

Its limit from positive \(h\) is \(2x-1\), while its limit from negative \(h\) is \(-(2x-1)\); these differ. The density is therefore not differentiable in θ at 0 for almost every \(x\), despite having fixed support and being normalized for every parameter value. Support dependence, integrability, and differentiability must be checked separately.

Common Mistake and Practical Diagnosis

A common mistake is to see \(f_\theta(x)=\theta^{-1}1_{[0,\theta]}(x)\) and rewrite it as an exponential, treating the indicator as though it were part of the fixed base measure. For example, the algebraic expression \(\exp\{-\log\theta\}\,1_{[0,\theta]}(x)\) is correct as a density, but it is not a fixed-carrier exponential-family representation: the indicator still depends on θ. The correction is to test the distributions themselves, not just rearrange the density. The interval argument above shows that two family members are not mutually absolutely continuous, contradicting a necessary property of the usual form.

The word nonregular is used for several related failures of standard regularity assumptions, and precise usage depends on the context. Here, the relevant structural feature is parameter-dependent support. It explains why the fixed-carrier representation fails and why a standard score calculation can break down. It does not mean that the density fails to integrate, nor does it by itself settle every possible differentiability or inferential question.

Check the support.
Ask whether sets of probability zero under one parameter value can have positive probability under another. If so, the distributions are not mutually absolutely continuous.

Check normalization separately.
For each parameter value, evaluate the integral of the density or kernel. An infinite integral with fixed support is an integrability restriction, not changing support.

Check parameter differentiation separately.
Even with fixed support and a finite normalizing integral, a density may fail to be differentiable. With moving support, include any boundary term before applying score identities.

Key takeaway: A usual fixed-carrier exponential family has a common support up to carrier-null sets. Uniform(0, θ) violates this condition because its distributions are not mutually absolutely continuous. That support obstruction is distinct from whether a kernel is integrable or a density is differentiable in its parameter.

Check Your Understanding

Try these questions using the distinction between support, integrability, and differentiability.

For \(0<\theta_1<\theta_2\), give a measurable set that has probability zero under Uniform(0, \(\theta_1\)) but positive probability under Uniform(0, \(\theta_2\)).
Why does mutual absolute continuity follow from the usual fixed-carrier exponential-family form?
For the kernel \(e^{\eta x}1\{x>0\}\), which parameter values yield a finite normalizing integral, and does its support depend on η?
What boundary term appears when differentiating the Uniform(0, θ) normalization integral?
Can a density have fixed support and still fail to be differentiable in its parameter? Explain using the density \(1+|\theta|(2x-1)\) on \((0,1)\).