Why Support and Parameter Constraints Matter
An algebraic expression can look like an exponential-family model without defining a probability distribution for every value of its apparent parameter. The response must have the correct support, and the normalizing constant must be finite. These conditions determine which parameter values are valid. They also help distinguish the distribution’s original parameter, such as a probability or rate, from the natural parameter used in an exponential-family representation.
For example, a Poisson response can take only nonnegative integer values, while an exponential response is continuous and nonnegative. Their natural parameters also have different domains: the Poisson natural parameter can be any real number, whereas the exponential natural parameter must be negative. The support and the parameter domain are related through normalization, but they are not the same thing.
For a discrete response, normalization requires summing over its support. For a continuous response, it requires integrating over the support. In either case, the carrier term \(h(y)\) must be nonnegative on the support, and the model assigns zero probability or density outside it. In the usual regular exponential-family representation, the support and carrier term do not depend on \(\theta\). The parameter changes the relative weights of the allowed response values, not which response values are allowed.
The examples below use \(T(y)=y\). A more general sufficient statistic \(T(y)\) is allowed, but the same normalization test applies. When a dispersion parameter is present, the test is performed with that parameter held fixed; it does not remove the requirement that the distribution be normalizable.
Finding the Natural-Parameter Domain
The domain can be found directly from the normalizing expression. Define \(Z(\theta)=\sum_y h(y)e^{\theta T(y)}\) for a discrete response, or \(Z(\theta)=\int h(y)e^{\theta T(y)}\,dy\) for a continuous response, with the sum or integral taken over the support. The distribution is normalized by setting \(A(\theta)=\log Z(\theta)\), provided \(0<Z(\theta)<\infty\). If \(Z(\theta)\) is infinite, no finite value of \(A(\theta)\) can make the expression a probability distribution.
For a continuous response, replace the sum in this expression with an integral. The set of parameter values that pass this test is the natural-parameter domain. In a regular one-parameter exponential family, this domain is an open interval. A distribution may sometimes be defined at a boundary value where the normalizer is finite, but that boundary need not belong to the regular domain. Whether the boundary is included must be checked rather than assumed.
Worked Examples: Discrete and Continuous Supports
Worked Example: Bernoulli Response and an Unrestricted Natural Parameter
Let \(Y\) indicate whether a hypothetical device passes a quality check. Assume \(Y\sim\operatorname{Bernoulli}(p)\), with \(P(Y=1)=p\), \(P(Y=0)=1-p\), and \(0<p<1\). Write the distribution in exponential form for \(p=0.20\), identify the natural-parameter domain, and verify the two probabilities.
Method. The response support is \(\{0,1\}\). Its probability mass function is \(p^y(1-p)^{1-y}\). Taking logarithms of the kernel gives \(y\log(p/(1-p))+\log(1-p)\). Thus the natural parameter and log-partition function are \(\theta=\log(p/(1-p))\) and \(A(\theta)=\log(1+e^\theta)\), with \(h(y)=1\) on \(\{0,1\}\).
Step 1: Calculate the natural parameter. With \(p=0.20\), \(\theta=\log(0.20/0.80)=\log(0.25)\approx-1.3863\).
Step 2: Calculate the log-partition function. Since \(e^\theta=0.25\), \(A(\theta)=\log(1+0.25)=\log(1.25)\approx0.2231\).
Step 3: Verify both probabilities. For \(y=1\), \(P(Y=1)=e^{\theta-A(\theta)}=0.25/1.25=0.20\). For \(y=0\), \(P(Y=0)=e^{-A(\theta)}=1/1.25=0.80\). They sum to \(1.00\), as required.
Step 4: Check the natural-parameter domain. For every real \(\theta\), the normalizing sum is \(1+e^\theta\), which is positive and finite. Therefore the natural-parameter domain is all real numbers. This differs from the original probability parameter, which must satisfy \(0<p<1\).
Common mistake avoided. Restricting \(\theta\) to \((0,1)\) because \(p\) is a probability is incorrect. The probability parameter and natural parameter are related by a transformation; \(\theta\) ranges over all real values as \(p\) ranges over \((0,1)\).
Worked Example: Poisson Support and a Finite Normalizer
Suppose the number \(Y\) of minor equipment faults in a hypothetical inspection period follows a Poisson distribution with rate \(\lambda=3.2\). Find its exponential-family representation, establish the natural-parameter domain, and calculate \(P(Y=2)\).
Method. The response support is \(\{0,1,2,\ldots\}\), and the mass function is \(e^{-\lambda}\lambda^y/y!\). Set \(\theta=\log\lambda\). Then the mass function has the form \(h(y)e^{\theta y-A(\theta)}\), where \(h(y)=1/y!\) and \(A(\theta)=e^\theta\).
Step 1: Map the rate to the natural parameter. \(\theta=\log(3.2)\approx1.1632\), so \(e^\theta=3.2\) and \(A(\theta)=3.2\).
Step 2: Verify the normalizer and domain. The normalizing sum is \(\sum_{y=0}^{\infty}e^{\theta y}/y!=\exp(e^\theta)\). This is finite for every real \(\theta\), so the natural-parameter domain is \((-\infty,\infty)\). The support remains nonnegative integers for every such value.
Step 3: Calculate the requested probability. \[ P(Y=2)=\frac{e^{-3.2}3.2^2}{2!} =\frac{e^{-3.2}(10.24)}{2} =5.12e^{-3.2} \approx5.12(0.0407622)=0.2087. \]
Check. The value is between \(0\) and \(1\); the calculation uses the factorial \(2!=2\), and the exponential-family form reproduces the original Poisson mass function.
Common mistake avoided. Taking \(\theta=\lambda\) is incorrect. The coefficient of \(y\) in the exponent is \(\log\lambda\), not \(\lambda\); the rate is \(e^\theta\).
Worked Example: Exponential Response and a Restricted Domain
Let \(Y\) be a hypothetical waiting time with an exponential distribution of rate \(\lambda=2\) per time unit. Write its density in exponential-family form, determine the full natural-parameter domain, and calculate \(P(Y>0.5)\).
Method. The response has continuous support \([0,\infty)\), and its density is \(\lambda e^{-\lambda y}\). Set \(\theta=-\lambda\). For \(\theta<0\), define \(A(\theta)=-\log(-\theta)\); then \(e^{\theta y-A(\theta)}=(-\theta)e^{\theta y}\), which is the required density when \(\lambda=-\theta\).
Step 1: Identify the natural parameter. Since \(\lambda=2\), \(\theta=-2\). The domain must be tested, not inferred merely from this example.
Step 2: Test the normalizing integral. For \(T(y)=y\) and \(h(y)=1\), the integral is \(\int_0^\infty e^{\theta y}\,dy\). If \(\theta<0\), it equals \(1/(-\theta)\), which is positive and finite. If \(\theta=0\), it is infinite; if \(\theta>0\), it also diverges. Therefore the natural-parameter domain is \((-\infty,0)\).
Step 3: Verify the density. At \(\theta=-2\), \(A(-2)=-\log 2\), so \(e^{-2y-A(-2)}=e^{-2y+\log 2}=2e^{-2y}\), the exponential density with rate \(2\).
Step 4: Calculate the tail probability. For an exponential distribution with rate \(2\), \(P(Y>0.5)=e^{-2(0.5)}=e^{-1}\approx0.3679\).
Check. The rate is positive, the natural parameter is negative, and the density integrates to \(1\) because \(\int_0^\infty 2e^{-2y}\,dy=1\).
Common mistake avoided. Allowing \(\theta=0\) because it is a valid real number fails the normalization test: the corresponding integral over the nonnegative support is infinite.
Support Must Be Checked Separately
The normalizer determines the admissible natural parameters, but it does not by itself establish that the response support has been represented correctly. A proposed formula must assign mass or density only to possible response values and must reproduce the original distribution on that support. For instance, the Poisson formula is summed over nonnegative integers, not over all real numbers; treating its response as continuous changes the model.
In the standard regular form, the support is fixed as the parameter varies. A distribution whose support moves with its parameter raises a different issue. Consider a uniform distribution on \((0,\tau)\), where \(\tau>0\). Its density is \(1/\tau\) for \(0<y<\tau\) and zero otherwise. The endpoint \(\tau\) determines which response values are possible. A fixed carrier function \(h(y)\) cannot, by itself, encode the changing cutoff while leaving the parameter only in \(\theta T(y)-A(\theta)\). The parameter-dependent indicator \(I(0<y<\tau)\) is essential.
This does not mean that every distribution with parameter-dependent support is impossible to express in any exponential-like notation. It means that it does not meet the usual fixed-support condition for a regular exponential-family representation in the form used here. The support condition and normalizer condition are separate checks.
A Practical Validity Check
When a distribution is presented in exponential-family form, use the following sequence. Each step addresses a different possible failure.
List the response values with positive mass or density, and note whether that set changes with the distribution parameter.
Rewrite the exponent to isolate the coefficient of the sufficient statistic \(T(y)\). Do not assume it is the original probability, rate, or mean.
Use the stated support and carrier term. The expression must be positive and finite.
Include boundary checks, then confirm that the resulting mass function or density matches the original model on its support.
This order prevents a common exam error: finding a plausible algebraic coefficient and declaring the representation valid without checking whether the corresponding normalizer exists. For finite response support, such as Bernoulli or fixed-trial binomial outcomes, the normalizing sum is finite for every finite natural parameter. For unbounded support, convergence can impose a restriction, as it does for the exponential response. The result depends on the support and carrier, not on a general rule that all natural parameters must have the same domain.
Check Your Understanding
For each problem, state the relevant support and check parameter validity rather than relying only on the original parameter’s range.
- A Bernoulli response has \(p=0.65\). Find \(\theta=\log(p/(1-p))\), to four decimals, and state whether it is in the natural-parameter domain. (Answer: \(\theta=\log(13/7)\approx0.6190\); yes, since the domain is all real numbers.)
- A Poisson response has natural parameter \(\theta=0\). Find its rate and state whether the natural parameter is valid. (Answer: \(\lambda=e^0=1\); valid, with support \(\{0,1,2,\ldots\}\).)
- A response has density \(f(y)=\lambda e^{-\lambda y}\) on \([0,\infty)\), with \(\lambda=0.75\) per time unit. Give its natural parameter and calculate \(P(Y>2)\), to four decimals. (Answer: \(\theta=-0.75\), in the domain \((-\infty,0)\); \(P(Y>2)=e^{-1.5}\approx0.2231\).)
- For a proposed continuous representation with support \([0,\infty)\), \(h(y)=1\), and \(T(y)=y\), determine whether \(\theta=0.4\) is admissible. (Answer: No; \(\int_0^\infty e^{0.4y}\,dy\) diverges, so the normalizer is not finite.)
- A uniform response has density \(1/\tau\) on \((0,\tau)\), zero elsewhere, with \(\tau>0\). What feature prevents this from being a regular fixed-support representation of the standard form? (Answer: Its support depends on \(\tau\); the parameter-dependent cutoff cannot be absorbed into a fixed carrier term.)