The Gaussian Dispersion Score And Its MLE

Why the Gaussian Dispersion Score Has a Simple Form

In a Gaussian generalized linear model, the dispersion parameter controls the common variance around the means. Once the means have been fitted, the observations’ departures from those means determine the likelihood for dispersion. The resulting estimate is especially simple: when the residual sum of squares is positive, the maximum-likelihood estimate is that sum divided by the number of observations.

The denominator is \(n\), not the residual degrees of freedom. That distinction is important: maximizing the likelihood and constructing an unbiased variance estimate are different tasks. We will derive the score, establish when its solution is the global maximum, and work through both fixed-mean and regression examples.

Definition: Let \(Y_1,\ldots,Y_n\) be independent Gaussian observations with fixed means \(\mu_1,\ldots,\mu_n\) and common dispersion \(\phi>0\), so that \(Y_i\sim N(\mu_i,\phi)\). For observed values \(y_1,\ldots,y_n\), define the residual sum of squares at these means by \(Q=\sum_{i=1}^n(y_i-\mu_i)^2\). The Gaussian dispersion score is the derivative of the log-likelihood with respect to \(\phi\), with the means held fixed.

Here “holding the means fixed” means that, when differentiating with respect to \(\phi\), the values \(\mu_i\) are treated as constants. In a regression model, we can first obtain fitted means and then evaluate this score at those values. This calculation concerns the dispersion coordinate of the likelihood; it does not differentiate the means with respect to regression coefficients.

Deriving the Score

For one observation, the Gaussian density with mean \(\mu_i\) and variance \(\phi\) is \(f(y_i;\mu_i,\phi)=(2\pi\phi)^{-1/2}\exp\{-(y_i-\mu_i)^2/(2\phi)\}\). Independence makes the joint likelihood the product of these densities. Taking its logarithm and collecting the terms involving the data gives

\ell(\phi) =-\frac{n}{2}\log(2\pi)-\frac{n}{2}\log\phi-\frac{Q}{2\phi}, \qquad \phi>0.

The first term is constant in \(\phi\). Differentiating the remaining terms, while \(Q\) is fixed, gives the score.

Result: For fixed means and \(Q=\sum_{i=1}^n(y_i-\mu_i)^2\), the Gaussian dispersion score is \(U_\phi(\phi)=-n/(2\phi)+Q/(2\phi^2)\), for \(\phi>0\).

The derivative of \(-n\log(\phi)/2\) is \(-n/(2\phi)\). The derivative of \(-Q/(2\phi)\) is \(Q/(2\phi^2)\), since the derivative of \(\phi^{-1}\) is \(-\phi^{-2}\). Thus the score equation is

-\frac{n}{2\phi}+\frac{Q}{2\phi^2}=0 \quad\Longleftrightarrow\quad -n\phi+Q=0 \quad\Longleftrightarrow\quad \phi=\frac{Q}{n}.

Multiplication by \(2\phi^2\) is valid because the parameter space requires \(\phi>0\). The equation gives a positive solution only if \(Q>0\). A score root by itself is not enough to establish a maximum, so the next result checks the full likelihood.

Theorem: If \(n\geq 1\) and \(Q>0\), the log-likelihood for \(\phi>0\), with the means held fixed, has a unique global maximum at \(\hat{\phi}=Q/n\). If \(Q=0\), the likelihood is unbounded as \(\phi\) decreases to zero, so there is no maximum at any \(\phi>0\).

Proof. For \(Q>0\), the score can be written as

U_\phi(\phi)=\frac{Q-n\phi}{2\phi^2}.

Because \(2\phi^2>0\), the score is positive when \(0<\phi<Q/n\), zero when \(\phi=Q/n\), and negative when \(\phi>Q/n\). Therefore the log-likelihood increases up to \(Q/n\) and decreases after it. This establishes a unique global maximum on the positive parameter space. Equivalently, the boundary limits also rule out a competing maximum: as \(\phi\) decreases to zero, the term \(-Q/(2\phi)\) drives \(\ell(\phi)\) to \(-\infty\), and as \(\phi\) increases without bound, the term \(-n\log(\phi)/2\) drives \(\ell(\phi)\) to \(-\infty\).

If \(Q=0\), the log-likelihood is \(-n\log(2\pi)/2-n\log(\phi)/2\). This tends to \(+\infty\) as \(\phi\) decreases to zero. Since zero is not in the parameter space \(\phi>0\), no positive value of \(\phi\) maximizes the likelihood. \(\square\)

Worked Calculations

Worked Example: Dispersion With Fixed Means

Suppose three independent observations have fixed means \(\mu_1=1\), \(\mu_2=4\), and \(\mu_3=4\), and the observed values are \(y_1=2\), \(y_2=3\), and \(y_3=5\). Find the Gaussian dispersion MLE.

The residuals are \(2-1=1\), \(3-4=-1\), and \(5-4=1\). Therefore \(Q=1^2+(-1)^2+1^2=3\), and \(n=3\). Since \(Q>0\), the theorem applies:

\hat{\phi}=\frac{Q}{n}=\frac{3}{3}=1.

For a direct score check, evaluate the score at \(\phi=1\): \(U_\phi(1)=-3/(2\cdot1)+3/(2\cdot1^2)=-3/2+3/2=0\). The score is positive for \(0<\phi<1\) and negative for \(\phi>1\), so this root is indeed the global maximizer.

Worked Example: An Intercept-Only Gaussian Model

Consider the observations \(2,4,6\) in a Gaussian model with an unknown intercept and common variance \(\phi\). The fitted intercept is the sample mean: \(\hat{\mu}=(2+4+6)/3=4\).

The fitted means are all \(4\), so the residuals are \(-2,0,2\). Their squared sum is \(Q=(-2)^2+0^2+2^2=8\). Holding the fitted means fixed, the dispersion score is

U_\phi(\phi)=-\frac{3}{2\phi}+\frac{8}{2\phi^2}.

Setting this equal to zero and multiplying by \(2\phi^2\) gives \(-3\phi+8=0\). Thus the likelihood estimate is \(\hat{\phi}=8/3\). The maximum is finite and unique because \(Q=8>0\).

In this example, the fitted mean uses one estimated coefficient, so there are \(3-1=2\) residual degrees of freedom. The familiar degrees-of-freedom-adjusted variance estimate is \(8/2=4\), not \(8/3\). It targets unbiasedness, whereas \(8/3\) is the likelihood maximizer. The two answers are not contradictory; they answer different estimation questions.

Worked Example: A Gaussian Regression Fit

For a simple linear regression with an intercept, take \(x=(0,1,2,3)\) and \(y=(1,2,2,5)\). The predictor and response means are \(\bar{x}=1.5\) and \(\bar{y}=2.5\). The least-squares slope is \(\hat{\beta}_1=\sum_i(x_i-\bar{x})(y_i-\bar{y})/\sum_i(x_i-\bar{x})^2\). The numerator is \(2.25+0.25-0.25+3.75=6\), and the denominator is \(2.25+0.25+0.25+2.25=5\). Therefore \(\hat{\beta}_1=6/5=1.2\), and \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=2.5-1.2(1.5)=0.7\).

The fitted means are \(0.7,1.9,3.1,4.3\). Subtracting them from the observations gives residuals \(0.3,0.1,-1.1,0.7\), so \(Q=0.3^2+0.1^2+(-1.1)^2+0.7^2=0.09+0.01+1.21+0.49=1.8\).

There are \(n=4\) observations. The dispersion score evaluated at these fitted means is \(U_\phi(\phi)=-4/(2\phi)+1.8/(2\phi^2)\), and its root is \(\hat{\phi}=1.8/4=0.45\). Checking the score at this value, \(-4/(2\cdot0.45)+1.8/(2\cdot0.45^2)=-4.444\ldots+4.444\ldots=0\). Since the residual sum of squares is positive, this is the unique maximum of the likelihood in \(\phi\) for the fitted means.

Why the Denominator Is \(n\), Not \(n-p\)

The likelihood contains one Gaussian variance contribution for each observation. With the means fixed, its log-dispersion term is \(-n\log(\phi)/2\), which produces the \(n\) in the score. The score’s other term contains the observed sum of squared residuals. Balancing these terms gives \(Q/n\).

When the means are estimated in a Gaussian linear model with \(p\) linearly independent regression coefficients, the same dispersion score evaluated at the fitted means yields the joint maximum-likelihood estimate \(Q/n\), provided \(Q>0\). This does not make \(Q/n\) unbiased. Under the model assumptions of independent \(N(X\beta,\phi)\) errors, full column rank \(p\), and \(n>p\), the residual sum of squares has expectation \((n-p)\phi\). In matrix terms, the fitted residual vector is \((I-H)Y\), where \(H\) is the rank-\(p\) projection matrix onto the columns of \(X\). Consequently,

E[Q]=\phi\,\operatorname{tr}(I-H)=(n-p)\phi, \qquad E\!\left[\frac{Q}{n-p}\right]=\phi.

The adjustment by \(n-p\) corrects the expectation of the residual sum of squares for estimating the means. It is used for an unbiased estimator under the stated conditions; it is not the maximizer of the Gaussian likelihood. The distinction is not a special feature of a particular regression calculation: it follows from the different criteria of likelihood maximization and unbiasedness.

Common Mistakes and Boundary Cases

Using the unbiased denominator in the score solution. Replacing \(n\) with \(n-p\) does not solve the Gaussian dispersion score equation. The score contains \(n\) because there are \(n\) Gaussian density normalizing factors.
Stopping after finding a score root. A root need not be a maximum in general. Here the sign of the score on either side proves it is the unique global maximum when \(Q>0\).
Differentiating the fitted means while taking the dispersion score. The dispersion score is the derivative with respect to \(\phi\) holding the means fixed. If the means are fitted, substitute their fitted values into \(Q\) for this calculation.
Ignoring the case \(Q=0\). The formula \(Q/n\) then gives the boundary value zero, which is outside the parameter space. The likelihood increases without bound as \(\phi\) approaches zero, so there is no finite positive MLE.

Key takeaway: For independent Gaussian observations with common positive variance and fixed means, the dispersion score is \(-n/(2\phi)+Q/(2\phi^2)\). If \(Q>0\), its unique likelihood maximizer is \(Q/n\). Degrees-of-freedom adjustments belong to unbiased variance estimation, not this score equation.

Check Your Understanding

Use the definitions and score derived above to answer these questions.

For \(n=5\) fixed means and residual sum of squares \(Q=10\), write the dispersion score and find its root.
Why is multiplication of the score equation by \(2\phi^2\) valid?
In a Gaussian linear model with \(n=12\) observations and \(p=3\) full-rank coefficients, what denominator appears in the dispersion MLE? What denominator appears in the usual unbiased variance estimate?
What happens to the likelihood if all observations equal their fixed means exactly?
Explain why a zero of the dispersion score must still be checked before it is called a maximum.