Gaussian Location with Known Variance

Why the Gaussian Location Form Matters

A normal model with a fixed variance and an unknown mean is a basic continuous-response model. Its density is familiar, but writing it in exponential-family form makes explicit how the unknown parameter enters: through a single coefficient multiplying the observed response. The distinction between that coefficient, the normalizing term, and the parameter-free carrier is important; the same algebra also shows exactly which quantities must be held fixed for this to be a one-parameter representation.

Throughout, let \(Y\) be a real-valued response with distribution \(N(\mu,\sigma^2)\), where \(\sigma^2>0\) is known and fixed while \(\mu\in\mathbb{R}\) is unknown. Densities are with respect to Lebesgue measure on \(\mathbb{R}\). The support does not depend on \(\mu\).

Definition: A one-parameter canonical exponential-family density has the form \(f(y;\theta)=h(y)\exp\{\theta T(y)-A(\theta)\}\), with respect to a stated base measure. Here \(\theta\) is the natural parameter, \(T(y)\) is the statistic multiplied by it, \(A(\theta)\) is the log-partition function, and \(h(y)\) is the carrier term. In this tutorial, \(\sigma^2\) is fixed, not an unknown parameter.

Deriving the Canonical Representation

The normal density is \(f(y;\mu)=(2\pi\sigma^2)^{-1/2}\exp\{-(y-\mu)^2/(2\sigma^2)\}\). Expand the square in its exponent:

-\frac{(y-\mu)^2}{2\sigma^2} = -\frac{y^2}{2\sigma^2} +\frac{\mu}{\sigma^2}y -\frac{\mu^2}{2\sigma^2}.

The term involving both the unknown mean and the observation is \((\mu/\sigma^2)y\). This identifies the natural parameter as \(\eta=\mu/\sigma^2\), with \(T(y)=y\). Substituting \(\mu=\sigma^2\eta\) in the parameter-only term gives \(\mu^2/(2\sigma^2)=\sigma^2\eta^2/2\). Thus the full density is

f(y;\eta) = \underbrace{\frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(-\frac{y^2}{2\sigma^2}\right)}_{h(y)} \exp\!\left\{ \eta y-\underbrace{\frac{\sigma^2\eta^2}{2}}_{A(\eta)} \right\}.

Result: For fixed known \(\sigma^2>0\), the normal location model has canonical components \(T(y)=y\), \(\eta=\mu/\sigma^2\), \(A(\eta)=\sigma^2\eta^2/2\), and \(h(y)=(2\pi\sigma^2)^{-1/2}\exp\{-y^2/(2\sigma^2)\}\). The natural-parameter space is \(\mathbb{R}\).

The carrier includes both the normal constant and the part of the exponent involving \(y^2\). It does not contain \(\mu\) or \(\eta\). The log-partition function contains the parameter-only term needed to normalize the density. The variance appears in both the carrier and the log-partition function, but it is a fixed known constant in this representation.

Worked Example: Variance Four and Mean Three

Suppose \(Y\sim N(3,4)\). Identify each canonical component and check the exponent for an observed value \(y=1\).

Here \(\sigma^2=4\) and \(\mu=3\), so \(\eta=\mu/\sigma^2=3/4\). The log-partition function at this natural parameter is \(A(3/4)=4(3/4)^2/2=9/8\). The carrier is \(h(y)=(8\pi)^{-1/2}\exp(-y^2/8)\).

At \(y=1\), the canonical exponent, including the carrier's exponential part, is \(-1/8+(3/4)(1)-9/8=-1/2\). The original normal exponent gives \(-(1-3)^2/(2\cdot4)=-4/8=-1/2\), as required. The common multiplicative factor is \((8\pi)^{-1/2}\) in both expressions.

Normalization and the Natural-Parameter Space

The algebraic expansion identifies a candidate \(A(\eta)\). A separate check confirms that it is exactly the log normalizer. For any \(\eta\in\mathbb{R}\), complete the square in the integral of the carrier multiplied by \(\exp(\eta y)\):

-\frac{y^2}{2\sigma^2}+\eta y = -\frac{(y-\sigma^2\eta)^2}{2\sigma^2} +\frac{\sigma^2\eta^2}{2}.

Theorem: For fixed \(\sigma^2>0\), the integral \(\int_{\mathbb R}h(y)\exp(\eta y)\,dy\) is finite for every real \(\eta\), and equals \(\exp(\sigma^2\eta^2/2)\). Consequently \(A(\eta)=\sigma^2\eta^2/2\) is the log-partition function and the natural-parameter space is all of \(\mathbb R\).

Proof. Using the completed square and the Gaussian integral \(\int_{\mathbb R}\exp\{-(y-a)^2/(2v)\}\,dy=\sqrt{2\pi v}\) for \(v>0\), we obtain

\begin{aligned} \int_{\mathbb R}h(y)\exp(\eta y)\,dy &= \frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(\frac{\sigma^2\eta^2}{2}\right) \int_{\mathbb R} \exp\!\left\{-\frac{(y-\sigma^2\eta)^2}{2\sigma^2}\right\}\,dy\\ &= \frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(\frac{\sigma^2\eta^2}{2}\right) \sqrt{2\pi\sigma^2}\\ &=\exp\!\left(\frac{\sigma^2\eta^2}{2}\right). \end{aligned}

The integral is finite for every real \(\eta\), so every real value is permitted in the natural-parameter space. Taking its logarithm gives \(A(\eta)=\sigma^2\eta^2/2\). Dividing \(h(y)\exp(\eta y)\) by this integral gives a density integrating to one, exactly the displayed canonical form. \(\square\)

There is also a direct parameter correspondence. With \(\sigma^2\) fixed and positive, \(\eta=\mu/\sigma^2\) maps \(\mu\in\mathbb{R}\) to \(\eta\in\mathbb{R}\), and the inverse is \(\mu=\sigma^2\eta\). Thus no possible real mean is lost or represented by more than one natural parameter.

Worked Example: Variance Nine and a Negative Mean

Let \(Y\sim N(-6,9)\). Find the natural parameter and log-partition value, then check the density exponent at \(y=-3\).

The natural parameter is \(\eta=-6/9=-2/3\), and \(A(\eta)=9(-2/3)^2/2=2\). The carrier's exponent at \(y=-3\) is \(-(-3)^2/(2\cdot9)=-1/2\). The full canonical exponent is therefore \(-1/2+(-2/3)(-3)-2=-1/2+2-2=-1/2\).

In the original normal form, the exponent is \(-(-3-(-6))^2/(2\cdot9)=-3^2/18=-1/2\). The two calculations agree. This check also illustrates that a negative mean simply produces a negative natural parameter; the natural-parameter space is not restricted to positive values.

Using the Form for Independent Observations

The same factorization applies observation by observation. If \(Y_1,\ldots,Y_n\) are independent and each has distribution \(N(\mu,\sigma^2)\) with common known variance, multiplying their densities yields a likelihood whose data-dependent parameter term uses the sum of the observations. This is a direct consequence of independence and the one-observation representation; it does not require a new density formula.

Worked Example: A Three-Observation Likelihood

Suppose \(Y_1,Y_2,Y_3\) are independent \(N(1,4)\) observations, and the observed values are \(1,2,4\). Write the likelihood in canonical form and verify its parameter-dependent exponent.

For each observation, \(\eta=\mu/\sigma^2=1/4\) and \(A(\eta)=4(1/4)^2/2=1/8\). Independence gives

L(\eta) = \left\{\prod_{i=1}^3 \frac{1}{\sqrt{8\pi}}\exp\!\left(-\frac{y_i^2}{8}\right)\right\} \exp\!\left\{\eta\sum_{i=1}^3 y_i-3\frac{4\eta^2}{2}\right\}.

The observed sum is \(1+2+4=7\), so the parameter-dependent exponent at \(\eta=1/4\) is \((1/4)(7)-3(1/8)=7/4-3/8=11/8\). For a direct check, the original likelihood exponent is \(-[(1-1)^2+(2-1)^2+(4-1)^2]/8=-(0+1+9)/8=-10/8\). The canonical exponent including the carriers is \(-[1^2+2^2+4^2]/8+11/8=-21/8+11/8=-10/8\). The two expressions match.

What Must Stay Fixed

The phrase known variance is part of the model, not an incidental detail. The representation above uses \(\sigma^2\) as a fixed constant when identifying both the natural parameter and the carrier. If the variance is allowed to vary as an unknown parameter, the term involving \(y^2\) is no longer parameter-free. That is not the one-parameter family derived here.

A related point is that the natural parameter is not the mean itself: it is \(\mu/\sigma^2\). For a particular fixed variance, the two parameters are in one-to-one correspondence, but their numerical values differ unless the variance happens to equal one. Keeping the distinction clear prevents errors when comparing normal models with different fixed variances.

Common Mistakes

Using \(\mu\) as the natural parameter. Expanding the exponent gives the coefficient \(\mu/\sigma^2\) on \(y\), so the canonical parameter is \(\eta=\mu/\sigma^2\).
Putting the \(y^2\) term in the log-partition function. The log-partition function can depend on the parameter, not on the observed value. The term \(-y^2/(2\sigma^2)\) belongs in the carrier.
Dropping the normalizing term. The term \(A(\eta)=\sigma^2\eta^2/2\) is required so the density integrates to one; it cannot be omitted just because it does not involve \(y\).
Treating the carrier as parameter-dependent. This representation assumes \(\sigma^2\) is fixed. With that assumption, the displayed carrier does not depend on \(\mu\) or \(\eta\).
Restricting the natural parameter to positive values. Since \(\eta=\mu/\sigma^2\), and \(\mu\) can be any real number while \(\sigma^2>0\), the natural-parameter space is all real numbers.

Key takeaway: For a normal response with fixed known variance, the canonical statistic is the response itself, the natural parameter is the mean divided by the variance, the log-partition function is quadratic in that natural parameter, and the remaining response-only factor is the carrier.

Check Your Understanding

Use the fixed-variance normal density and the canonical form to answer the following.

For \(Y\sim N(5,4)\), find the natural parameter and log-partition value.
Which part of the canonical representation contains the term involving \(y^2\), and why?
What is the natural-parameter space when the variance is fixed and positive? Explain briefly.
For \(Y\sim N(-2,1)\), write the natural parameter and log-partition function value.
Why can the carrier in this tutorial be called parameter-free even though it contains the variance?