Deriving The Exponential Dispersion Mean-Variance Law

From the Model Form to a Mean-Variance Law

An exponential dispersion model has a particularly useful feature: one function, $b$, determines both its mean and its variance. The result is not an extra assumption about how the variance changes with the mean. It follows from the model’s normalization, provided the normalizing integral can be differentiated appropriately.

This connection is useful because it separates two roles. The cumulant function $b$ describes the relationship between the natural parameter and the moments; the positive dispersion parameter $\phi$ scales the response variance. When the mean can be used instead of the natural parameter, the same relationship can be written as a variance function.

Definition: An exponential dispersion model, at fixed dispersion $\phi>0$, has density or mass function with respect to a fixed measure $\nu$ of the form $$ f(y;\theta,\phi)=a(y,\phi)\exp\left\{\frac{y\theta-b(\theta)}{\phi}\right\}. $$ Here $a(y,\phi)$ is nonnegative and does not depend on $\theta$, and $\theta$ lies in an open interval $\Theta$. The support and measure $\nu$ do not depend on $\theta$. The function $b$ is the cumulant function for this representation.

The fixed-$\phi$ condition matters in the derivation: derivatives below are taken with respect to $\theta$ while $\phi$ is held constant. We also need conditions that allow differentiation under the integral. One sufficient set of conditions is that, for each $\theta\in\Theta$, there is a $\delta>0$ such that $[\theta-\delta,\theta+\delta]\subset\Theta$ and $$ \int |y|^2 a(y,\phi)\exp\left\{\frac{ty}{\phi}\right\}\nu(dy)<\infty \quad\text{for }t=\theta-\delta,\ \theta+\delta. $$ This endpoint condition supplies an integrable bound for the first two differentiated integrands throughout the interval: for $y\geq 0$, use the upper endpoint, and for $y<0$, use the lower endpoint. In particular, it justifies differentiating the normalizing integral twice near $\theta$.

Deriving the Mean and Variance

For fixed $\phi$, define the integral that contains the $\theta$-dependent part of the density:

H(\theta)=\int a(y,\phi)\exp\left\{\frac{y\theta}{\phi}\right\}\nu(dy).

Since $f$ is normalized, integrating it over the support gives $e^{-b(\theta)/\phi}H(\theta)=1$. Thus $H(\theta)=e^{b(\theta)/\phi}$. Differentiating this identity and relating the derivatives of $H$ to moments gives the desired result.

Theorem: Under the stated fixed-support and differentiation conditions, an exponential dispersion model satisfies $$ E[Y]=b'(\theta),\qquad \operatorname{Var}(Y)=\phi b''(\theta). $$

Differentiate the normalizing integral.
Dominated differentiation gives $H'(\theta)=\int (y/\phi)a(y,\phi)e^{y\theta/\phi}\nu(dy)$ and $H''(\theta)=\int (y^2/\phi^2)a(y,\phi)e^{y\theta/\phi}\nu(dy)$.

Identify normalized moments.
Multiplying the integrands by $e^{-b(\theta)/\phi}$ turns them into the model density, so $H'/H=E[Y]/\phi$ and $H''/H=E[Y^2]/\phi^2$.

Use the log of the normalizer.
The identity $H=e^{b/\phi}$ implies $(\log H)'=b'/\phi$ and $(\log H)''=b''/\phi$. The first derivative yields the mean; the second yields the variance.

Here is the full calculation. First, from the integral definition and the model density,

\frac{H'(\theta)}{H(\theta)} = \frac{\int (y/\phi)a(y,\phi)e^{y\theta/\phi}\nu(dy)} {\int a(y,\phi)e^{y\theta/\phi}\nu(dy)} = \frac{E[Y]}{\phi}.

On the other hand, $H=e^{b/\phi}$ implies $(\log H)'=b'/\phi$. Equating the two expressions proves $E[Y]=b'(\theta)$.

For the second derivative, the quotient rule and the moment expressions above give

(\log H)'' =\frac{H''}{H}-\left(\frac{H'}{H}\right)^2 =\frac{E[Y^2]}{\phi^2}-\frac{E[Y]^2}{\phi^2} =\frac{\operatorname{Var}(Y)}{\phi^2}.

But differentiating $\log H=b(\theta)/\phi$ twice gives $(\log H)''=b''(\theta)/\phi$. Equating these expressions and multiplying by $\phi^2$ proves $\operatorname{Var}(Y)=\phi b''(\theta)$. $\square$

Writing the Variance as a Function of the Mean

The theorem gives the moments in terms of $\theta$. To use the mean as the parameter, consider the set of attainable means $M=\{b'(\theta):\theta\in\Theta\}$. If $b'$ is one-to-one on $\Theta$, each $\mu\in M$ identifies a unique natural parameter $\theta(\mu)$.

Definition: When $b'$ is one-to-one on the natural-parameter domain, the variance function is $$ V(\mu)=b''(\theta(\mu)),\qquad \mu\in M, $$ where $b'(\theta(\mu))=\mu$. Consequently, the response variance is $\operatorname{Var}(Y)=\phi V(\mu)$.

This definition is restricted to the attainable mean domain $M$, not every real number. In a nondegenerate model, $b''(\theta)>0$ wherever the variance is finite, so $b'$ is strictly increasing on an interval $\Theta$ and can be inverted on its image. If the model is degenerate or the mean cannot identify $\theta$, this particular mean-parameterized definition requires additional care.

Result: If $b$ is twice differentiable, $b''(\theta)>0$ on the interval $\Theta$, and $b'(\Theta)=M$, then $b'$ has an inverse on $M$ and $$ \operatorname{Var}(Y)=\phi V(\mu),\qquad \mu=b'(\theta)\in M. $$

Proof. For any $\theta_1<\theta_2$ in $\Theta$, the mean value theorem gives a $\xi\in(\theta_1,\theta_2)$ such that $$ b'(\theta_2)-b'(\theta_1)=b''(\xi)(\theta_2-\theta_1)>0. $$ Therefore $b'$ is strictly increasing and one-to-one, so its inverse $\theta(\mu)$ is defined for $\mu\in M$. By the theorem, $\operatorname{Var}(Y)=\phi b''(\theta)$. Substituting $\theta=\theta(\mu)$ and applying the definition of $V$ yields $\operatorname{Var}(Y)=\phi V(\mu)$. $\square$

Worked Examples

Worked Example: Normal Response

Suppose $Y\sim N(\mu,\phi)$, where the second argument is the variance and $\phi>0$. Write its density in exponential dispersion form and verify the mean-variance law.

The log density is $$ -\frac{1}{2}\log(2\pi\phi)-\frac{(y-\mu)^2}{2\phi} = -\frac{1}{2}\log(2\pi\phi)-\frac{y^2}{2\phi} +\frac{y\mu-\mu^2/2}{\phi}. $$

Thus $\theta=\mu$, $b(\theta)=\theta^2/2$, and $a(y,\phi)=(2\pi\phi)^{-1/2}\exp\{-y^2/(2\phi)\}$. Differentiating gives $b'(\theta)=\theta$ and $b''(\theta)=1$. Therefore $E[Y]=\theta=\mu$ and $\operatorname{Var}(Y)=\phi$. Since $\theta(\mu)=\mu$, the variance function is $V(\mu)=1$, so $\phi V(\mu)=\phi$, as required.

Worked Example: Gamma Response at a Specified Mean

Consider the gamma exponential dispersion model with shape $1/\phi$ and mean $\mu>0$. Its canonical parameter is $\theta=-1/\mu<0$, and $b(\theta)=-\log(-\theta)$. Find the mean and variance at $\mu=2$ and $\phi=1/2$.

Differentiate the cumulant function: $$ b'(\theta)=-\frac{1}{\theta},\qquad b''(\theta)=\frac{1}{\theta^2}. $$ At $\theta=-1/2$, $b'(\theta)=2$, confirming the specified mean, while $b''(\theta)=4$. The theorem therefore gives $$ E[Y]=2,\qquad \operatorname{Var}(Y)=\frac{1}{2}\cdot 4=2. $$

Equivalently, $\theta=-1/\mu$ implies $b''(\theta)=\mu^2$, so $V(\mu)=\mu^2$ and $\phi V(\mu)=(1/2)(2^2)=2$. The corresponding gamma distribution has shape $1/\phi=2$ and scale $\phi\mu=1$; its mean is $2\cdot1=2$ and its variance is $2\cdot1^2=2$, checking the result directly.

Worked Example: Inverse-Gaussian Response

For an inverse-Gaussian exponential dispersion model, take $\theta=-1/(2\mu^2)<0$ and $b(\theta)=-\sqrt{-2\theta}$, with $\mu>0$. Find the variance function and evaluate the variance when $\mu=2$ and $\phi=1/4$.

Differentiation gives $$ b'(\theta)=\frac{1}{\sqrt{-2\theta}},\qquad b''(\theta)=(-2\theta)^{-3/2}. $$ Since $1/\sqrt{-2\theta}=\mu$, the mean is $b'(\theta)=\mu$. Also, $(-2\theta)^{-3/2}=\mu^3$, so $V(\mu)=\mu^3$. At $\mu=2$, $\theta=-1/8$, $b''(\theta)=8$, and the variance is $$ \operatorname{Var}(Y)=\frac14\cdot 8=2. $$

As a check, the inverse-Gaussian shape parameter in this dispersion convention is $1/\phi=4$; its variance is $\mu^3/(1/\phi)=\phi\mu^3=(1/4)(8)=2$. The mean and variance agree with the derivatives of $b$.

Why the Distinction Matters

The formula $\operatorname{Var}(Y)=\phi V(\mu)$ separates the response family’s mean-dependent variance pattern from the scale of dispersion. It is often the variance function $V$ that makes the relationship easy to compare across mean values, while $\phi$ multiplies that function to give the actual response variance. A variance function is therefore not itself the response variance unless $\phi=1$.

The derivation also explains why the log-partition function is central. Normalization forces the integral $H(\theta)$ to equal $e^{b(\theta)/\phi}$. Its first log derivative is a normalized first moment, and its second log derivative is a normalized variance. The mean-variance law follows from those two facts, not from a family-specific guess.

Common Mistake

Dropping the dispersion factor. It is incorrect to conclude that $\operatorname{Var}(Y)=b''(\theta)$ from the second derivative. The second derivative of $\log H$ is $b''(\theta)/\phi$, while it is also $\operatorname{Var}(Y)/\phi^2$. Equating them gives $\operatorname{Var}(Y)=\phi b''(\theta)$.
Calling $b''(\theta)$ the variance function without changing parameters. The variance function is expressed in terms of the mean: $V(\mu)=b''(\theta(\mu))$, when the inverse exists. The quantity $b''(\theta)$ is written in terms of the natural parameter.
Ignoring the mean domain. The inverse $\theta(\mu)$ is defined only for $\mu\in b'(\Theta)$. Extending a variance-function formula outside that range does not follow from the model.

Key takeaway: For a regular exponential dispersion model at fixed $\phi$, normalization yields $E[Y]=b'(\theta)$ and $\operatorname{Var}(Y)=\phi b''(\theta)$. If the mean identifies $\theta$, this becomes $\operatorname{Var}(Y)=\phi V(\mu)$, with $V(\mu)=b''(\theta(\mu))$.

Check Your Understanding

Use the model form and parameter domains carefully when answering.

Why must the support and reference measure be independent of $\theta$ in the differentiation argument?
In the normal example, identify $b'(\theta)$, $b''(\theta)$, and $V(\mu)$, and explain which quantity equals the response variance.
For the gamma example, use $\theta=-1/\mu$ and $b(\theta)=-\log(-\theta)$ to derive $V(\mu)$.
What regularity purpose does the local integrability condition on $|y|^2a(y,\phi)e^{ty/\phi}$ serve?
Explain why a formula for $V(\mu)$ is only established on the attainable mean domain.