The Normal Distribution With Known Dispersion

Why the Normal Distribution Fits the Exponential-Family Framework

The Normal model is often introduced through its mean and variance. In generalized linear model notation, it is also useful to express its density in exponential-family form. Doing so makes the connections among the natural parameter, the mean, and the variance explicit. Here the focus is the case in which the variance is known and fixed, rather than estimated from the observations.

We use the dispersion-form convention $f(y;\theta,\phi)=\exp\{[y\theta-b(\theta)]/a(\phi)+c(y,\phi)\}$. In this form, the dispersion appears in the denominator of the part involving the natural parameter. The notation matters: the natural parameter identified using this convention is not the same as a parameter obtained by absorbing dispersion into it.

Definition: Suppose $Y$ has a Normal distribution with mean $\mu$ and known variance $\sigma^2>0$. Its exponential-family representation is $$ f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\right\}, $$ where $\theta=\mu$, $\phi=\sigma^2$, $a(\phi)=\phi$, $b(\theta)=\theta^2/2$, and $c(y,\phi)=-y^2/(2\phi)-\tfrac12\log(2\pi\phi)$. The natural parameter $\theta$ ranges over all real numbers, and $\phi$ is fixed and positive.

Deriving the Exponential-Family Form

The Normal density with mean $\mu$ and variance $\sigma^2$ is

f(y;\mu,\sigma^2)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{(y-\mu)^2}{2\sigma^2}\right\}.

Expand the square in the exponent. This separates the term involving both the observation and the mean from terms that can be assigned to the log-partition function and the carrier term.

-\frac{(y-\mu)^2}{2\sigma^2} =-\frac{y^2}{2\sigma^2}+\frac{y\mu}{\sigma^2}-\frac{\mu^2}{2\sigma^2} =\frac{y\mu-\mu^2/2}{\sigma^2}-\frac{y^2}{2\sigma^2}.

The normalizing factor contributes $-\tfrac12\log(2\pi\sigma^2)$ to the exponent. Comparing the resulting expression with the stated dispersion-form convention gives $\theta=\mu$, $b(\theta)=\theta^2/2$, $a(\phi)=\phi=\sigma^2$, and $c(y,\phi)=-y^2/(2\phi)-\tfrac12\log(2\pi\phi)$. The carrier term contains the remaining dependence on $y$ and $\phi$, but not on $\theta$.

The support is the full real line, which does not depend on $\theta$. The restriction $\phi>0$ ensures a valid variance and density. In the known-dispersion setting, $\phi$ is specified as fixed; it is not another unknown parameter to be inferred from these observations.

Worked Example: Matching a Known-Variance Density to the Form

Suppose $Y$ has a Normal distribution with mean $12$ and known variance $9$, so its standard deviation is $3$. Identify the exponential-family components and verify the log density at $y=15$.

From the density, $\theta=\mu=12$, $\phi=\sigma^2=9$, $a(\phi)=9$, and $b(\theta)=12^2/2=72$. The carrier term at $y=15$ is $c(15,9)=-15^2/(2\cdot9)-\tfrac12\log(2\pi\cdot9)$.

The exponential-family log density is therefore $$ \frac{15(12)-72}{9}-\frac{15^2}{18}-\frac12\log(18\pi) =12-12.5-\frac12\log(18\pi) \approx -2.518. $$

Directly from the Normal density, the log density is $$ -\frac{(15-12)^2}{2(9)}-\frac12\log(2\pi\cdot9) =-0.5-\frac12\log(18\pi)\approx -2.518. $$

The two expressions agree. The variance used in the dispersion term is $9$, not the standard deviation $3$.

Mean, Variance, and the Log-Partition Function

For a density in this dispersion form, the mean is $b'(\theta)$ and the variance is $a(\phi)b''(\theta)$, provided the derivatives exist and the density is properly normalized. In this case $b(\theta)=\theta^2/2$, so $b'(\theta)=\theta$ and $b''(\theta)=1$. Consequently, the mean is $\theta=\mu$ and the variance is $\phi=\sigma^2$. The variance function, defined by $\operatorname{Var}(Y)=\phi V(\mu)$, is $V(\mu)=1$.

This constant variance function is a defining feature of this Normal family in the dispersion representation. Changing the mean does not change the variance when the dispersion is fixed.

Worked Example: Recovering the Mean and Variance by Differentiation

Let $Y$ have the known-variance Normal model with $\phi=4$ and natural parameter $\theta=-2.5$. Derive its mean and variance from the log-partition function, then state the standard deviation.

The log-partition function is $b(\theta)=\theta^2/2$. Its first derivative is $b'(\theta)=\theta$, so $$ E(Y)=b'(-2.5)=-2.5. $$

Its second derivative is $b''(\theta)=1$, giving $$ \operatorname{Var}(Y)=\phi b''(\theta)=4(1)=4. $$

Thus $Y$ has mean $-2.5$, variance $4$, and standard deviation $\sqrt{4}=2$, in the response’s measurement units. The mean follows from the first derivative; the variance uses the second derivative multiplied by the dispersion.

Key distinction: In the dispersion-form representation used here, the natural parameter is $\theta=\mu$. If dispersion is absorbed into the coefficient of $y$, the exponent can instead be written with coefficient $\eta=\mu/\sigma^2$ on $y$. That alternative coefficient is not $\theta$ under the convention used in this tutorial.

Independent Observations and the Sufficient Statistic

If $Y_1,\ldots,Y_n$ are independent Normal observations with a common unknown mean $\mu$ and a common known variance $\phi$, their joint density is the product of their individual densities. Collecting the terms involving $\mu$ gives

f(y_1,\ldots,y_n;\mu,\phi) =\exp\left\{\frac{\mu\sum_{i=1}^n y_i-n\mu^2/2}{\phi} -\sum_{i=1}^n\frac{y_i^2}{2\phi} -\frac{n}{2}\log(2\pi\phi)\right\}.

For fixed known $\phi$, the data enter the part involving $\mu$ through $\sum_{i=1}^n y_i$. Thus the sum is a sufficient statistic for $\mu$ in this model; equivalently, the sample mean contains the same information about $\mu$ because $\bar{Y}=(\sum_iY_i)/n$. This is a direct application of the factorization of the joint density, not a claim that every summary of the data is sufficient.

Worked Example: A Joint Density with Known Variance

Three independent hypothetical observations are $8$, $10$, and $11$, each modeled with a common mean $\mu$ and known variance $\phi=4$. Write the log likelihood as a function of $\mu$, and evaluate it at $\mu=9$.

The sum is $8+10+11=29$, and the sum of squares is $8^2+10^2+11^2=285$. Substituting these into the joint form gives $$ \ell(\mu)=\frac{29\mu-3\mu^2/2}{4}-\frac{285}{8}-\frac{3}{2}\log(8\pi). $$

At $\mu=9$, the first term is $(29(9)-3(9^2)/2)/4=(261-121.5)/4=34.875$. Therefore, $$ \ell(9)=34.875-35.625-\frac32\log(8\pi)\approx -5.586. $$

As a check, the sum of squared deviations from $9$ is $(8-9)^2+(10-9)^2+(11-9)^2=6$. The direct Normal log likelihood is $-6/(2\cdot4)-\tfrac32\log(2\pi\cdot4)=-0.75-\tfrac32\log(8\pi)$, also approximately $-5.586$. The dependence on $\mu$ in the joint density is through the sum $29$.

Assumptions, Checks, and Alternatives

The representation describes a continuous response on the real line with a Normal distribution, a specified mean, and a positive known variance. For a collection of observations, the product likelihood above additionally assumes independence and a common variance. In applications, the mean may vary across observations; the exponential-family form still applies observation by observation when each conditional response is Normal with its own mean and the stipulated known dispersion.

The known-variance assumption is substantive. It is appropriate when the variance is fixed by the model or supplied by the measurement design, rather than treated as an unknown quantity learned from the same response data. If the variance changes across observations, the common-dispersion expression does not describe that situation without modification. If the response is discrete or constrained to be nonnegative, a Normal density on the whole real line may likewise be inappropriate.

Useful checks include examining whether residuals around the specified means are reasonably compatible with a Normal shape and whether their spread is consistent with the stipulated variance. Unusually heavy tails, strong skewness, outliers, or a spread that changes systematically with the mean are warnings that the assumed response distribution or dispersion may be wrong. These checks do not prove the model correct; they identify discrepancies worth investigating.

Modeling choice	Feature relevant here	When it may be unsuitable
Normal with known dispersion	Continuous real-valued response; variance is fixed and does not depend on the mean.	The variance is unknown or changes across observations, or the response shape is strongly non-Normal.
Bernoulli or binomial	Response records a binary outcome or a count of successes from a known number of trials.	The observed response is a continuous measurement rather than a binary outcome or trial count.
Poisson	Response is a nonnegative integer count with variance tied to its mean in the Poisson model.	The response is continuous, or the count model’s mean-variance structure is unsuitable.

These alternatives are not interchangeable versions of the same density. Their supports and variance structures differ. Choosing a Normal response solely because it is familiar can produce invalid probabilities or misleading uncertainty when the response is a count, strongly skewed, or has a variance inconsistent with the fixed-dispersion assumption.

Common Mistakes

Using the standard deviation as the dispersion. If the standard deviation is $3$, the variance and dispersion are $9$, not $3$.
Calling $\mu/\phi$ the natural parameter without stating a different convention. Under the dispersion-form convention here, $\theta=\mu$; $\mu/\phi$ is the coefficient that appears if dispersion is absorbed into the coefficient of $y$.
Forgetting the dispersion multiplier in the variance formula. The variance is $a(\phi)b''(\theta)=\phi$, not just $b''(\theta)=1$.
Treating known variance as an estimated quantity. In this setting $\phi$ is fixed and specified; fitting an unknown variance is a different modeling problem.
Ignoring support and shape. The Normal response has support on all real numbers. A model that assigns meaningful probability to impossible response values may be a poor choice.

Key takeaway: For a Normal response with known variance $\sigma^2$, the dispersion-form exponential-family components are $\theta=\mu$, $a(\phi)=\phi=\sigma^2$, and $b(\theta)=\theta^2/2$. Differentiating $b$ gives mean $\mu$ and variance $\phi$, so the variance function is constant at one.

Check Your Understanding

Answer each question, then compare your work with the solutions below.

For a Normal response with mean $5$ and known variance $16$, identify $\theta$, $\phi$, and $b(\theta)$.
For $b(\theta)=\theta^2/2$, derive the mean and variance when $\theta=3$ and $\phi=2$.
In this tutorial’s convention, is $\mu/\phi$ the natural parameter? Explain the distinction.
For independent observations with common mean and known common variance, which data summary contains the dependence of the likelihood on the mean?

Solutions. (1) $\theta=\mu=5$, $\phi=16$, and $b(\theta)=\theta^2/2=25/2=12.5$. (2) Since $b'(\theta)=\theta$, the mean is $3$. Since $b''(\theta)=1$, the variance is $\phi b''(\theta)=2(1)=2$. (3) No. Under the dispersion-form convention, $\theta=\mu$; $\mu/\phi$ is the coefficient of $y$ in a representation that absorbs dispersion into that coefficient. (4) The likelihood depends on the common mean through $\sum_iY_i$, or equivalently through the sample mean.