Inference With an Estimated Dispersion Parameter

Why Dispersion Changes Coefficient Inference

In a generalized model, the mean structure may determine how covariates affect the response, while a separate dispersion parameter controls the scale of response variability. If dispersion is known, it enters coefficient uncertainty as a fixed quantity. If it must be estimated, coefficient standard errors also depend on the uncertainty scale estimated from the data. The distinction changes Wald statistics and can change the form of likelihood-ratio comparisons.

This tutorial concerns models with independent observations, a correctly specified mean and variance structure, and a scalar dispersion parameter $\phi>0$. In the usual generalized linear model formulation, $\operatorname{Var}(Y_i)=\phi V(\mu_i)$, where $V$ is the variance function. Poisson and binomial models conventionally fix $\phi=1$; other families, such as the normal model, allow dispersion to be estimated.

Definition: Let $\beta\in\mathbb{R}^p$ be the regression coefficient vector, $X$ the $n\times p$ design matrix, and $\eta_i=x_i^\mathsf{T}\beta$ the linear predictor. Write $\mu_i=E[Y_i]$, $\operatorname{Var}(Y_i)=\phi V(\mu_i)$, and $\mu_i'=d\mu_i/d\eta_i$. Define the full-sample coefficient information without the dispersion multiplier by $$ A_n(\beta)=X^\mathsf{T}W_0(\beta)X,\qquad W_0(\beta)=\operatorname{diag}\left\{\frac{(\mu_i')^2}{V(\mu_i)}:i=1,\ldots,n\right\}. $$ Thus the expected information for $\beta$, when $\phi$ is known, is $A_n(\beta)/\phi$.

The matrix $A_n$ is full-sample information: it already accumulates contributions from all $n$ observations. Its inverse therefore has the sample-size scaling of a covariance matrix. In particular, one must not add another factor of $1/n$ to $\phi A_n^{-1}$. If instead information is defined per observation, the covariance expression includes the corresponding $1/n$.

Deriving the Coefficient Covariance

For an exponential-dispersion model with canonical parameter $\theta_i$, a typical log likelihood has terms $\{y_i\theta_i-b(\theta_i)\}/\phi+c(y_i,\phi)$. Differentiating with respect to $\beta$ gives the coefficient score $$ U_\beta(\beta,\phi)=\sum_{i=1}^n \frac{Y_i-\mu_i}{\phi V(\mu_i)}\mu_i'x_i. $$ The score has expectation zero under the model. When differentiating it again, terms multiplied by $Y_i-\mu_i$ have expectation zero; the remaining expected negative derivative is $$ I_{\beta\beta}(\beta,\phi) =E\left[-\frac{\partial U_\beta}{\partial\beta^\mathsf{T}}\right] =\frac{1}{\phi}\sum_{i=1}^n\frac{(\mu_i')^2}{V(\mu_i)}x_ix_i^\mathsf{T} =\frac{1}{\phi}A_n(\beta). $$ This derives the dispersion multiplier in coefficient information. Under regularity conditions for maximum-likelihood asymptotics—identifiability, an interior true parameter, nonsingular information, and suitable smoothness and moment conditions—the coefficient estimate consequently has approximate covariance $\phi A_n(\beta)^{-1}$.

Theorem: Suppose the generalized-model likelihood is regular, the true coefficient vector is identifiable and interior, $A_n$ is nonsingular, and $\phi$ is known. With full-sample information $A_n$, the coefficient estimate satisfies the normalized asymptotic relation $$ \frac{A_n(\beta)^{1/2}(\hat{\beta}-\beta)}{\sqrt{\phi}} \xrightarrow{d}N_p(0,I_p), $$ or, equivalently, its first-order covariance is $\phi A_n(\beta)^{-1}$. If $\phi$ is unknown and $\hat{\phi}\xrightarrow{p}\phi$, replacing $\phi$ by $\hat{\phi}$ gives the plug-in covariance estimate $\hat{\phi}A_n(\hat{\beta})^{-1}$.

The plug-in statement follows from consistency and continuity: $A_n(\hat{\beta})$ estimates the information at the true coefficient, and $\hat{\phi}$ estimates the scale multiplying its inverse. For a regular model with a fixed-dimensional coefficient and increasing information, this substitution preserves the limiting standardized normal distribution. An estimated dispersion does not automatically have an exact finite-sample distributional correction; the validity of a particular small-sample reference distribution depends on the model.

Worked Example: Known Dispersion in an Intercept-Only Poisson Model

Suppose $Y_1,\ldots,Y_4$ are independent Poisson observations with a common mean $\mu=\exp(\beta)$, and the observed counts are $2,3,1,4$. For the Poisson model, $\phi=1$, $V(\mu)=\mu$, and $\mu'=\mu$.

The log likelihood, apart from terms not involving $\beta$, is $10\beta-4\exp(\beta)$. Its score is $10-4\exp(\beta)$, so the maximum-likelihood estimate satisfies $\exp(\hat{\beta})=10/4=2.5$, giving $\hat{\beta}=\log(2.5)$.

Each diagonal entry of $W_0$ is $(\mu')^2/V(\mu)=\mu$. At the estimate, each is $2.5$, so $A_4=4(2.5)=10$. Therefore the estimated coefficient variance is $1/10=0.1$, and its standard error is $\sqrt{0.1}\approx0.316$. There is no dispersion estimate to plug in here: the conventional Poisson likelihood fixes $\phi=1$.

Wald Inference with Estimated Dispersion

For a scalar contrast $c^\mathsf{T}\beta$, the estimated standard error is obtained from the plug-in covariance. For several restrictions, let $C$ be a $q\times p$ matrix of rank $q$, and test $H_0:C\beta=d$. The Wald statistic is $$ W=(C\hat{\beta}-d)^\mathsf{T} \left\{\hat{\phi}\,C A_n(\hat{\beta})^{-1}C^\mathsf{T}\right\}^{-1} (C\hat{\beta}-d). $$ Under $H_0$, the regularity conditions above, and consistent estimation of $\phi$, $W\xrightarrow{d}\chi^2_q$.

To see why, first standardize the contrast using its true first-order covariance, $\phi C A_n(\beta)^{-1}C^\mathsf{T}$. The normalized coefficient limit gives a $q$-dimensional standard normal contrast, whose squared length has a $\chi^2_q$ limit. Replacing the true covariance by the plug-in estimate does not change that limit: consistency of $\hat{\phi}$ and the information estimate makes the ratio of estimated to true covariance converge to the identity. This is the multivariate Slutsky argument. The inverse information here is full-sample inverse information; the contrast covariance is not multiplied by an additional $1/n$.

Worked Example: Estimated Dispersion in a Normal Mean Model

Let $Y_1,\ldots,Y_5$ be independent $N(\mu,\phi)$ observations, with observed values $1,2,3,4,5$. Here the coefficient is the intercept $\beta=\mu$, and the coefficient information without dispersion is $A_5=5$.

The sample mean is $\hat{\mu}=3$. The residual sum of squares is $$ \sum_{i=1}^5(Y_i-\hat{\mu})^2=4+1+0+1+4=10. $$ For the usual unbiased variance estimate, $\hat{\phi}=10/(5-1)=2.5$, and the estimated variance of the mean is $\hat{\phi}/5=0.5$. Its standard error is $\sqrt{0.5}\approx0.707$.

For testing $H_0:\mu=0$, the estimated-dispersion statistic is $t=3/\sqrt{0.5}\approx4.243$, with $4$ degrees of freedom under the exact normal-model theory. If instead $\phi=1$ were known, the standard error would be $\sqrt{1/5}\approx0.447$, and the standardized statistic would be $3/\sqrt{0.2}\approx6.708$, with a standard normal reference distribution. The estimates are the same; the uncertainty calculation and reference distribution differ.

Likelihood-Ratio Inference and Profiling

When dispersion is unknown, likelihood-ratio inference must account for its estimation. For a restriction on $\beta$, the profile likelihood maximizes the likelihood over $\phi$ separately at each permitted value of $\beta$. Under regularity conditions, Wilks' theorem says that twice the difference between the unrestricted and restricted maximized log likelihoods converges to $\chi^2_q$. This result applies to the joint likelihood with the nuisance dispersion parameter accounted for; it does not license treating an estimated dispersion as if it were known in a known-dispersion likelihood-ratio statistic.

The normal linear model makes the distinction explicit. For a model with $p$ fitted coefficients and residual sum of squares $RSS$, maximizing over $\phi$ gives $\hat{\phi}_{ML}=RSS/n$. Comparing a restricted model with residual sum of squares $RSS_0$ to a larger model with $RSS_1$, the profile likelihood-ratio statistic is $n\log(RSS_0/RSS_1)$. By contrast, if $\phi$ is known, twice the log-likelihood difference is $(RSS_0-RSS_1)/\phi$.

Worked Example: Nested Normal Models with Unknown Dispersion

Take four observations $y=(1,2,2,5)$, with predictor values $x=(0,1,2,3)$. Compare the restricted intercept-only model with the model containing an intercept and a slope.

The restricted estimate is the mean $2.5$, and its residual sum of squares is $RSS_0=(1-2.5)^2+(2-2.5)^2+(2-2.5)^2+(5-2.5)^2=9$. For the larger model, $\bar{x}=1.5$, $\bar{y}=2.5$, $S_{xx}=5$, and $S_{xy}=6$. Thus the slope estimate is $6/5=1.2$, the intercept estimate is $2.5-1.2(1.5)=0.7$, and the fitted values are $0.7,1.9,3.1,4.3$. The residuals are $0.3,0.1,-1.1,0.7$, giving $RSS_1=0.09+0.01+1.21+0.49=1.8$.

With unknown dispersion, the profile likelihood-ratio statistic is $4\log(9/1.8)=4\log(5)\approx6.438$, asymptotically compared with $\chi^2_1$. In the normal model, the exact finite-sample comparison instead uses $$ F=\frac{(RSS_0-RSS_1)/1}{RSS_1/(4-2)} =\frac{7.2}{0.9}=8. $$ The numerator has one restriction and the larger model has two coefficients, leaving two residual degrees of freedom. If $\phi=1$ were known, the likelihood-ratio statistic would be $(9-1.8)/1=7.2$, not $4\log(5)$. These different expressions arise because the unknown-dispersion likelihood estimates the variance separately in each model.

What to Report and a Common Pitfall

For coefficient inference, report whether dispersion was fixed or estimated, how it was estimated, and how that choice enters the covariance and reference distribution. In generalized models, common dispersion estimates include Pearson-residual or deviance-based estimates adjusted by residual degrees of freedom; these are not universally identical to the maximum-likelihood estimate. Use a consistent estimator justified for the model. Exact Student $t$ or $F$ inference is a special feature of normal linear models with their stated assumptions, not a general consequence of estimating dispersion.

Wrong reasoning: “The coefficient information is $A_n$, so the covariance is $A_n^{-1}/n$; if dispersion is estimated, just replace $\phi$ by $\hat{\phi}$ in that expression.”
Correction: If $A_n=X^\mathsf{T}W_0X$ is the full-sample information without dispersion, the covariance is $\phi A_n^{-1}$, or $\hat{\phi}A_n^{-1}$ after plugging in. A factor $1/n$ appears only when using per-observation information instead.
Wrong reasoning: “Dividing every likelihood-ratio statistic by $\hat{\phi}$ gives the correct unknown-dispersion test.”
Correction: With unknown dispersion, use the likelihood profile over that nuisance parameter or an appropriate model-specific approximation. The normal-model profile likelihood produces a log residual-sum-of-squares ratio, while exact finite-sample normal comparisons use an $F$ statistic.

Key takeaway: Dispersion multiplies the coefficient covariance, not the full-sample information itself. When it is estimated, use a justified $\hat{\phi}$ in the covariance and account for nuisance-scale estimation in likelihood inference; keep the information scaling consistent throughout.

Check Your Understanding

Work through these questions using the distinction between known and estimated dispersion.

If $A_n$ denotes full-sample information without the dispersion multiplier, what is the first-order covariance of $\hat{\beta}$ when $\phi$ is known?
Why does replacing $\phi$ by a consistent $\hat{\phi}$ preserve the asymptotic Wald reference distribution under regularity conditions?
In a normal linear model, why does the profile likelihood-ratio statistic involve a logarithm of residual sum-of-squares ratios when dispersion is unknown?
What changes in a coefficient standard error when a model's dispersion is estimated rather than fixed at one?
Why is an exact $F$ reference distribution for estimated-dispersion inference not automatically available in every generalized model?