Checking Whether A Distribution Is Exponential Family

Why Check the Form Carefully?

A probability model may look unfamiliar and still have exponential-family structure. Conversely, a formula that contains an exponential may fail to be an exponential family in the standard sense. The useful test is structural: identify which parts depend on the observation, which depend on the parameter, and whether the set of possible observations changes with the parameter.

This distinction matters in generalized linear models because exponential-family structure organizes the response distribution through a statistic, a natural parameter, and a normalizing function. The task here is to recognize that structure, not to choose a link or fit a model.

Definition: A density or probability mass function has a one-parameter exponential-family representation if it can be written as $$ f(y;\eta)=\exp\{\eta T(y)-A(\eta)+B(y)\}, $$ with a support that does not depend on $\eta$. Here $T(y)$ is a statistic, $\eta$ is the natural parameter, $A(\eta)$ is the log-normalizing function, and $B(y)$ collects terms depending only on the observation. If the model is originally indexed by $\theta$, the natural parameter may be a function $\eta(\theta)$.

For a density, the expression is normalized by integration with respect to the underlying measure; for a mass function, it is normalized by summation. A fixed support means that the same possible observation values are available at every permitted parameter value. Values outside that support have probability or density zero for every parameter value.

A family can still have this form when its natural parameter is restricted to a proper subset of the real line. The restriction must be kept in view, however. A representation on an open interval is regular in the usual one-parameter sense; a representation restricted to a closed interval or an isolated set may still be exponential-form, but is not regular under that convention.

A Practical Test

Use the following sequence before deciding whether a stated distribution is in exponential-family form. Do not decide based only on whether the formula contains a logarithm or exponential.

Record the support.
Write the set of possible observations and check whether it changes with the parameter. A parameter-dependent indicator or endpoint is an immediate warning for the standard form.

Take logarithms where useful.
Separate the log density or log mass into terms involving both the observation and parameter, terms involving only the parameter, and terms involving only the observation.

Identify a statistic and natural parameter.
Try to express the joint observation-parameter term as $\eta(\theta)T(y)$. The statistic must be a function of the observation alone; the natural parameter must be a function of the model parameter alone.

Find and check the normalizer.
Choose $A$ so that the mass sums to one or the density integrates to one over the fixed support. Then state the allowed range of $\eta$.

The separation is substantive. It is not valid to place a parameter inside $T(y)$, or the observation inside $\eta$, merely to make a formula appear to fit. Likewise, a parameter-dependent support cannot be hidden in $B(y)$, because $B$ is fixed as the parameter varies.

Normalization result: For a fixed support $\mathcal Y$, a proposed form $\exp\{\eta T(y)-A(\eta)+B(y)\}$ is normalized when $$ A(\eta)=\log\left(\int_{\mathcal Y}\exp\{\eta T(y)+B(y)\}\,d\nu(y)\right), $$ where the integral is replaced by a sum for a discrete distribution. The quantity inside the logarithm must be finite and positive for every allowed $\eta$.

Why the Normalizing Function Has This Form

The formula follows directly from requiring total probability to equal one. This also gives a useful check on a proposed rewriting: the parameter-only term must exactly compensate for the total unnormalized mass.

Start with the proposed representation.
For a density on fixed support $\mathcal Y$, write $f(y;\eta)=\exp\{\eta T(y)-A(\eta)+B(y)\}$.

Integrate over the support.
Factor out the term that does not depend on $y$:
$\int_{\mathcal Y} f(y;\eta)\,d\nu(y)=e^{-A(\eta)}\int_{\mathcal Y}e^{\eta T(y)+B(y)}\,d\nu(y)$.

Impose total probability one.
Set the preceding product equal to 1. Taking logarithms gives $A(\eta)=\log\int_{\mathcal Y}e^{\eta T(y)+B(y)}\,d\nu(y)$. The same argument uses a sum for a mass function.

The derivation also explains why the parameter range matters: if the integral or sum is infinite for a proposed $\eta$, no finite $A(\eta)$ can normalize that expression at that value.

Worked Examples

Worked Example: A Density with a Restricted Parameter Range

Let $X$ have density $f(x;\theta)=\theta x^{\theta-1}$ for $0<x<1$, and density zero otherwise. The parameter is restricted to $2<\theta<5$. Determine whether this is an exponential family, identify its components, and check the density at $\theta=3$ and $x=1/2$.

Method. Check the support first, then take the logarithm and separate the terms. The common mistake is to assume a parameter restriction prevents an exponential-family representation.

Support. For every allowed $\theta$, the support is $0<x<1$. It does not depend on $\theta$.

Rewrite. On the support,

f(x;\theta)=\theta x^{\theta-1} =\exp\{\theta\log x+\log\theta-\log x\}.

Thus $T(x)=\log x$, $\eta(\theta)=\theta$, $A(\eta)=-\log\eta$, and $B(x)=-\log x$. The natural parameter is restricted to $2<\eta<5$, which is an open interval. The restriction does not prevent the representation.

Normalization check. For every allowed $\theta>0$,

\int_0^1\theta x^{\theta-1}\,dx =\left[x^\theta\right]_0^1 =1-0 =1.

Requested density value. At $\theta=3$ and $x=1/2$,

f(1/2;3)=3(1/2)^{3-1} =3(1/4) =3/4 =0.75.

Check. The computed value is nonnegative, the integral is one, and the support is fixed. The density is in exponential-family form with a restricted natural parameter range.

Worked Example: A Discrete Exponential Tilt on a Restricted Interval

A random variable $Y$ takes values in $\{0,1,2\}$, with probabilities $$ P(Y=y\mid\theta)=\frac{c_y e^{\theta y}}{1+2e^\theta+e^{2\theta}}, \qquad (c_0,c_1,c_2)=(1,2,1), $$ where $-\log 4<\theta<0$. Show the exponential-family representation and calculate all three probabilities at $\theta=-\log 2$.

Method. Treat the denominator as the normalizing sum and identify the terms depending on $y$. The trap is to mistake a restricted parameter interval for a parameter-dependent support.

Support and form. The support is the fixed set $\{0,1,2\}$. The denominator factors as

1+2e^\theta+e^{2\theta}=(1+e^\theta)^2.

Therefore

P(Y=y\mid\theta) =\exp\{y\theta-2\log(1+e^\theta)+\log c_y\}.

The components are $T(y)=y$, $\eta=\theta$, $A(\eta)=2\log(1+e^\eta)$, and $B(y)=\log c_y$. The natural parameter is restricted to $-\log 4<\eta<0$, an open interval.

Calculate at $\theta=-\log 2$. Since $e^\theta=1/2$, the denominator is

$$ 1+2(1/2)+(1/2)^2=1+1+1/4=9/4. $$

For $y=0$, the numerator is $1$, so $P(Y=0)=1/(9/4)=4/9$. For $y=1$, the numerator is $2(1/2)=1$, so $P(Y=1)=4/9$. For $y=2$, the numerator is $(1/2)^2=1/4$, so $P(Y=2)=(1/4)/(9/4)=1/9$.

Check. The probabilities sum to $4/9+4/9+1/9=1$. Also, $-\log4<-\log2<0$, so the requested parameter value is permitted.

Worked Example: A Moving Support That Fails the Standard Form

Let $X$ be uniformly distributed on $(\theta,\theta+1)$, where $0<\theta<1$. Its density is 1 on that interval and 0 elsewhere. Does it have the standard exponential-family form with fixed support?

Method. Compare the supports at two permitted parameter values before attempting algebra. The common mistake is to focus on the constant density value and overlook the indicator that changes with $\theta$.

At $\theta=0.1$, the support is $(0.1,1.1)$; at $\theta=0.5$, it is $(0.5,1.5)$. In particular, $x=0.25$ is inside the first support but outside the second:

f(0.25;0.1)=1,\qquad f(0.25;0.5)=0.

In a standard representation with fixed support, a given $x$ is either outside the support for every parameter value, or its exponential expression is positive for every parameter value for which the parameters are finite. Here the density at $x=0.25$ changes from positive to zero as $\theta$ changes. That cannot be represented by a fixed-support exponential-family form.

Normalization check. Each stated density still integrates to one: the interval has length $(\theta+1)-\theta=1$, and the density is 1 throughout it. The failure is not a normalization error; it is the parameter-dependent support.

Conclusion. This location-shifted uniform model does not have the standard fixed-support exponential-family representation. Broader treatments may discuss families with parameter-dependent support separately, but that is not the form being tested here.

Restricted Parameters, Support, and Common Traps

A parameter restriction limits which distributions in a family are allowed; it does not necessarily change the support of any one distribution. For instance, the first example permits only $2<\theta<5$, while every permitted density remains supported on $(0,1)$. The second example restricts the natural parameter to an interval while retaining the same three possible outcomes.

This differs fundamentally from the moving-support example. There, changing $\theta$ changes whether particular $x$-values are possible. A support indicator such as $\mathbf{1}\{\theta<x<\theta+1\}$ cannot be treated as a parameter-free $B(x)$.

Feature	What to check	Implication
Restricted parameter range	Does the same support apply at every permitted parameter?	May still have exponential-family form; state the restricted natural-parameter range.
Parameter-dependent support	Can an observation have positive density or mass for one parameter and zero for another?	Fails the standard fixed-support form.
Normalization	Does the proposed mass sum or density integral equal one throughout the allowed range?	If not, the proposed $A(\eta)$ or denominator is incorrect.
Regularity	Does the natural parameter range contain an open interval?	A closed or isolated restriction can preserve the algebraic form but fail the usual regularity condition.

A frequent exam error is to call any model with a restricted parameter space “not an exponential family.” The correction is to test the representation and support first, then report the restriction. The opposite error is also possible: a neat algebraic rewrite does not override a moving support.

Key takeaway: A reliable classification needs three checks: the support is fixed, the log mass or log density separates into a parameter times a statistic plus parameter-only and observation-only terms, and the normalizer is valid throughout the stated parameter range. A restricted range for the natural parameter is compatible with exponential-family form.

Check Your Understanding

For each problem, check the support and normalization before deciding whether the stated model has exponential-family form.

A density is $f(x;\theta)=\theta x^{\theta-1}$ for $0<x<1$, with $\theta\in(1,4)$. State the support, $T(x)$, $\eta$, $A(\eta)$, and whether the restriction prevents exponential-family form. (Answer: Support $(0,1)$; $T(x)=\log x$, $\eta=\theta$, $A(\eta)=-\log\eta$, $B(x)=-\log x$; it is exponential-form and the open restricted range does not prevent it.)
A mass function on $\{0,1,2\}$ is proportional to $c_y e^{\eta y}$, where $(c_0,c_1,c_2)=(1,3,2)$ and $-1<\eta<1$. Give its normalizer and identify the statistic and observation-only term. (Answer: $Z(\eta)=1+3e^\eta+2e^{2\eta}$; $T(y)=y$, $B(y)=\log c_y$, and $A(\eta)=\log Z(\eta)$.)
A density is uniform on $(0,\theta)$, with $0<\theta<2$. Does it satisfy the standard fixed-support form? Explain using two parameter values. (Answer: No; for example, $x=0.5$ is inside the support at $\theta=1$ but outside it at $\theta=0.25$, so the support depends on the parameter.)
For $x>0$, let $f(x;\theta)=\theta^{-1}e^{-x/\theta}$, with $1<\theta<3$. Write the natural parameter and log-normalizing function, and state whether the natural-parameter range is open. (Answer: $T(x)=x$, $\eta=-1/\theta$, $A(\eta)=-\log(-\eta)$, $B(x)=0$; $-1<\eta<-1/3$, an open interval.)
A mass function on $\{0,1,2\}$ is $p(y;\eta)=e^{\eta y}/(1+e^\eta+e^{2\eta})$, with $0\leq\eta\leq1$. Is it exponential-form, and is it regular under the open-interval convention? (Answer: It is exponential-form with $T(y)=y$, $A(\eta)=\log(1+e^\eta+e^{2\eta})$, and $B(y)=0$; the closed natural-parameter range is not regular under that convention.)

What You'll Learn