Dispersion-Scaled Exponential-Family Form

Why Scale the Exponential-Family Form?

A response model can often be written so that its log density separates into a part involving the observed value and a part that normalizes the distribution. The dispersion-scaled exponential-family form makes that separation especially useful: one function describes how the canonical parameter affects the response, while a separate scale function records the role of dispersion. Keeping those roles distinct makes it easier to interpret a model and to derive its moments.

The form is not a universal rewriting of every probability distribution. It describes a family with a particular structure: after scaling by a positive dispersion function, the terms involving the response and canonical parameter share a log-partition function that does not itself depend on dispersion. That restriction is part of the model, not a consequence of notation alone.

Definition: A dispersion-scaled exponential-family density or mass function has the form $$ f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\right\}, $$ with respect to a specified common measure on the response space. Here $a(\phi)>0$; $\theta$ is the canonical parameter; $b(\theta)$ is the log-partition function in the scaled convention; and $c(y,\phi)$ is the carrier term. The support and reference measure are taken not to depend on $\theta$. For each allowed $(\theta,\phi)$, the expression must integrate or sum to one.

The normalization condition is not optional. In particular, the functions must satisfy

\int \exp\left\{\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\right\}\,d\nu(y)=1,

where $\nu$ is the reference measure; a sum replaces the integral for a discrete response. This condition determines how $b$ and $c$ fit together. It also explains why a proposed choice of components must be checked against the actual distribution.

What Each Component Does

The canonical parameter $\theta$ is the parameter that multiplies the response $y$ in the exponent, after the dispersion scaling is taken into account. It is distinct from the dispersion parameter $\phi$. In some models, a parameter of direct scientific interest is not itself canonical; the canonical parameter is the one that gives the specified exponential form.

The dispersion function $a(\phi)$ is positive and scales the parameter-dependent term. It is commonly a variance-like quantity, but it should not automatically be identified with the variance of $Y$. The variance also depends on the curvature of $b$, as shown below.

The log-partition function $b(\theta)$ supplies the parameter-dependent normalizing contribution. Its name reflects the fact that it is tied to the logarithm of a normalizing integral. In this scaled convention, its contribution to the exponent is $-b(\theta)/a(\phi)$.

The carrier term $c(y,\phi)$ contains the remaining response-dependent contribution and any factors needed for normalization, such as a density scale. It may depend on both $y$ and $\phi$, but it must not depend on $\theta$. The carrier can also encode which response values are allowed; outside the support, the density or mass is zero.

Component	Role in the exponent	Key restriction
$\theta$	Controls the coefficient of $y$	Appears in the parameter-dependent term
$a(\phi)$	Scales that term	Must be positive
$b(\theta)$	Provides the scaled normalizing contribution	Does not depend on $\phi$ in this form
$c(y,\phi)$	Supplies the remaining response and dispersion terms	Does not depend on $\theta$

How the Scaled Form Arises

At a fixed value of $\phi$, a one-parameter exponential-family representation can be written as

f(y;\eta,\phi)=\exp\{y\eta-A_\phi(\eta)+c(y,\phi)\},

where $\eta$ is the natural parameter and $A_\phi(\eta)$ is the log normalizer at that value of $\phi$. Set $\theta=a(\phi)\eta$. Then the response term becomes $y\theta/a(\phi)$. For the scaled form to result, the log normalizer must satisfy the additional structural condition

A_\phi\left(\frac{\theta}{a(\phi)}\right)=\frac{b(\theta)}{a(\phi)}

for a single function $b$ that does not depend on $\phi$. Under that condition, substitution gives the stated form. Without it, rewriting the natural parameter as $\theta/a(\phi)$ does not by itself establish a dispersion-scaled representation with a common $b$.

Worked Example: Identifying the Components in a Normal Model

Let $Y\sim N(\mu,\phi)$, where $\phi>0$ is the variance. Its density is $$ f(y;\mu,\phi)=\frac{1}{\sqrt{2\pi\phi}}\exp\left\{-\frac{(y-\mu)^2}{2\phi}\right\}. $$ Expand the square: $$ -\frac{(y-\mu)^2}{2\phi}=-\frac{y^2}{2\phi}+\frac{y\mu}{\phi}-\frac{\mu^2}{2\phi}. $$ Thus the log density is $$ \frac{y\mu-\mu^2/2}{\phi}-\frac{y^2}{2\phi}-\frac12\log(2\pi\phi). $$ Comparing terms gives $\theta=\mu$, $a(\phi)=\phi$, $b(\theta)=\theta^2/2$, and $c(y,\phi)=-y^2/(2\phi)-\tfrac12\log(2\pi\phi)$.

For $y=3$, $\mu=2$, and $\phi=4$, the exponent in the scaled representation is $$ \frac{3(2)-2^2/2}{4}-\frac{3^2}{2(4)}-\frac12\log(2\pi(4)) =1-\frac98-\frac12\log(8\pi) =-\frac18-\frac12\log(8\pi). $$ The direct normal log density gives $$ -\frac{(3-2)^2}{2(4)}-\frac12\log(2\pi(4)) =-\frac18-\frac12\log(8\pi), $$ so the two calculations agree.

Moment Identities from the Log-Partition Function

The log-partition function does more than normalize the density: its derivatives determine the first two moments. The precise result requires conditions that justify differentiating the normalizing integral.

Theorem: Fix $\phi$, let $a(\phi)>0$, and suppose the model is normalized on a support and reference measure that do not depend on $\theta$. Suppose $\theta$ ranges over an open interval, $b$ is twice differentiable there, and differentiation of the normalizing integral twice is valid. A sufficient condition is that, in a neighborhood of each $\theta$, the carrier-weighted exponential integrals at slightly larger and smaller parameter values are finite; this provides integrable bounds for the first and second derivatives. Then $$ E[Y]=b'(\theta),\qquad \operatorname{Var}(Y)=a(\phi)b''(\theta). $$

Proof. Fix $\phi$ and write $a=a(\phi)$. Define $$ M(\theta)=\int \exp\left\{\frac{y\theta}{a}+c(y,\phi)\right\}\,d\nu(y). $$ Normalization of the model implies $M(\theta)=\exp\{b(\theta)/a\}$. The regularity assumptions permit differentiation under the integral, so $$ M'(\theta)=\int \frac{y}{a}\exp\left\{\frac{y\theta}{a}+c(y,\phi)\right\}\,d\nu(y). $$ Dividing by $M(\theta)$, the integrand is weighted by the normalized density, giving $$ \frac{M'(\theta)}{M(\theta)}=\frac{E[Y]}{a}. $$ On the other hand, differentiating $\log M(\theta)=b(\theta)/a$ gives $(\log M)'=b'(\theta)/a$. Equating the expressions yields $E[Y]=b'(\theta)$.

For the second derivative, differentiating again gives $$ M''(\theta)=\int \frac{y^2}{a^2}\exp\left\{\frac{y\theta}{a}+c(y,\phi)\right\}\,d\nu(y), $$ so $M''/M=E[Y^2]/a^2$. Therefore $$ (\log M)''=\frac{M''}{M}-\left(\frac{M'}{M}\right)^2 =\frac{\operatorname{Var}(Y)}{a^2}. $$ But $(\log M)''=b''(\theta)/a$. Equating these expressions proves $\operatorname{Var}(Y)=a(\phi)b''(\theta)$. $\square$

Worked Example: Using Curvature to Find the Variance

In the normal representation above, $b(\theta)=\theta^2/2$, so $b'(\theta)=\theta$ and $b''(\theta)=1$. The theorem gives $E[Y]=\theta=\mu$ and $\operatorname{Var}(Y)=\phi$. For $\mu=2$ and $\phi=4$, these are $2$ and $4$. The dispersion function contributes the factor $4$, while the log-partition curvature contributes the factor $1$.

The score with respect to $\theta$, for one observed $y$, follows directly by differentiating the log density: $$ \frac{\partial}{\partial\theta}\log f(y;\theta,\phi)=\frac{y-b'(\theta)}{a(\phi)}. $$ At $y=3,\theta=2,\phi=4$, this is $(3-2)/4=1/4$. Its expectation is zero because $E[Y]=b'(\theta)$.

Worked Checks and Common Pitfalls

Worked Example: A Sample Mean Has a Different Dispersion Scale

Let $Y_1,\ldots,Y_m$ be independent $N(\mu,\phi)$ observations, and let $\bar Y$ be their mean. Then $\bar Y\sim N(\mu,\phi/m)$. Substituting variance $\phi/m$ into the normal density and expanding gives $$ f(\bar y;\mu,\phi)=\exp\left\{\frac{m(\bar y\mu-\mu^2/2)}{\phi} -\frac{m\bar y^2}{2\phi}-\frac12\log\left(\frac{2\pi\phi}{m}\right)\right\}. $$ This has $\theta=\mu$, $b(\theta)=\theta^2/2$, $a(\phi)=\phi/m$, and $$ c(\bar y,\phi)=-\frac{m\bar y^2}{2\phi}-\frac12\log\left(\frac{2\pi\phi}{m}\right). $$ For $m=4,\phi=2,\mu=1,\bar y=1.5$, the scaled parameter term is $$ \frac{1.5(1)-1^2/2}{2/4}=\frac{1}{0.5}=2. $$ The carrier is $-4(1.5)^2/(2(2))-\tfrac12\log(\pi)=-2.25-\tfrac12\log(\pi)$, so the log density is $-0.25-\tfrac12\log(\pi)$. Directly, a normal density with mean $1$ and variance $0.5$ gives $$ -\frac{(1.5-1)^2}{2(0.5)}-\frac12\log(2\pi(0.5)) =-0.25-\frac12\log(\pi), $$ as required.

Worked Example: Checking a Proposed Log-Partition Function

Return to $Y\sim N(\mu,\phi)$, and keep $a(\phi)=\phi$, $\theta=\mu$, and the normal carrier term. Suppose someone proposes $b(\theta)=\theta^2$, omitting the factor $1/2$. At $y=3,\theta=2,\phi=4$, the proposed log density is $$ \frac{3(2)-2^2}{4}-\frac{9}{8}-\frac12\log(8\pi) =\frac12-\frac98-\frac12\log(8\pi) =-\frac58-\frac12\log(8\pi). $$ The actual normal log density is $-\tfrac18-\tfrac12\log(8\pi)$, as calculated above. The proposed expression differs by $-1/2$, so it does not represent this normal model. The error is not a harmless change in notation: it changes the density.

A frequent mistake is to call $a(\phi)$ the variance in every model. The theorem shows why this is generally wrong: the variance is $a(\phi)b''(\theta)$, and only when $b''(\theta)=1$ does it equal $a(\phi)$. Another mistake is to let the carrier depend on $\theta$. That can obscure the canonical parameter and invalidate the stated moment derivation. Finally, a familiar-looking exponential expression is not enough: check both the parameter restrictions and normalization.

Key takeaway: In the dispersion-scaled form, $a(\phi)$ scales the canonical parameter contribution, $b(\theta)$ supplies the log normalizer, and $c(y,\phi)$ carries the remaining response and dispersion terms. When regularity conditions hold, $b'(\theta)$ is the mean and $a(\phi)b''(\theta)$ is the variance.

Check Your Understanding

Use the definitions and results above to answer these questions.

Which component is required to be positive, and why is it not necessarily the response variance?
What condition must a fixed-dispersion exponential-family representation satisfy to yield a single $b(\theta)$ shared across dispersion values?
State the mean and variance in terms of $b$ and $a(\phi)$, including the regularity requirement in words.
In the normal model, identify the carrier term and explain why it may depend on $y$ and $\phi$ but not $\theta$.
If $b''(\theta)=2$, what does the theorem imply about the variance relative to $a(\phi)$?

Component	Role in the exponent	Key restriction
\(\theta\)	Controls the coefficient of \(y\)	Appears in the parameter-dependent term
\(a(\phi)\)	Scales that term	Must be positive
\(b(\theta)\)	Provides the scaled normalizing contribution	Does not depend on \(\phi\) in this form
\(c(y,\phi)\)	Supplies the remaining response and dispersion terms	Does not depend on \(\theta\)