Likelihood Inference for Dispersion Parameters

Why Dispersion Requires Its Own Likelihood Analysis

In a generalized linear model, regression coefficients describe how the mean changes with predictors, while a dispersion parameter describes the scale of variation around that mean. Estimating dispersion is not always the same as dividing a residual sum of squares by a degrees-of-freedom value: the likelihood for dispersion depends on the probability model, and the regression coefficients are usually nuisance parameters when dispersion is the target.

The key likelihood operation is profiling: for each candidate dispersion value, maximize the likelihood over the regression coefficients, and then compare the resulting maximized likelihoods. This preserves the dependence between mean and dispersion estimation. It also makes clear why an estimate at, or near, a boundary requires special care.

Definition: In an exponential-dispersion model, a response $Y_i$ has density or mass function of the form $$ f(y_i;\theta_i,\phi) = \exp\left\{ \frac{y_i\theta_i-b(\theta_i)}{\phi} +c(y_i,\phi) \right\}, \qquad \phi>0. $$ Here $\theta_i$ is the canonical parameter determined by the mean model, and $\phi$ is the dispersion parameter. In the usual case, $E[Y_i]=b'(\theta_i)$ and $\operatorname{Var}(Y_i)=\phi b''(\theta_i)$. A generalized linear model specifies $\theta_i$, or equivalently the mean, through regression coefficients $\beta$ and a link function.

The exact likelihood contribution $c(y_i,\phi)$ matters: it may depend on $\phi$, and that dependence contributes to the dispersion score. Thus one cannot generally estimate $\phi$ by looking only at the factor $1/\phi$ multiplying the mean-model part of the log-likelihood. The parameter space also matters. In standard Poisson and binomial generalized linear models, dispersion is fixed at $1$; it is not a free likelihood parameter unless a different probability model is specified.

Profile Likelihood for Dispersion

Write the joint log-likelihood as $\ell(\beta,\phi)$. For each admissible $\phi$, the profile log-likelihood is the largest log-likelihood attainable by varying $\beta$:

\ell_p(\phi)=\sup_{\beta\in\mathcal{B}}\ell(\beta,\phi).

A joint maximum-likelihood estimate $(\hat{\beta},\hat{\phi})$, when it exists, has $\hat{\phi}$ maximizing $\ell_p(\phi)$. A likelihood-based confidence set for an interior value of $\phi$ can be formed by retaining the candidate values whose profile likelihood ratio is not too small. Under standard regularity conditions, the likelihood-ratio statistic for one scalar parameter is compared with a chi-squared distribution with one degree of freedom.

\mathcal{C}_{1-\alpha} = \left\{ \phi: 2\bigl[\ell_p(\hat{\phi})-\ell_p(\phi)\bigr] \leq \chi^2_{1,\,1-\alpha} \right\}.

This calibration is asymptotic, not an automatic finite-sample guarantee. It requires an identifiable model, a true parameter in the interior of its parameter space, suitable smoothness and moment conditions, and nonsingular information for the parameter of interest after accounting for nuisance parameters. If the maximum is constrained to a boundary, the usual chi-squared calibration may not apply.

Result: Suppose that for $\phi$ in an open interval the maximizer $\hat{\beta}(\phi)$ is unique, interior, and differentiable, and satisfies $\partial\ell(\hat{\beta}(\phi),\phi)/\partial\beta=0$. Then $$ \frac{d}{d\phi}\ell_p(\phi) = \frac{\partial}{\partial\phi} \ell\bigl(\hat{\beta}(\phi),\phi\bigr). $$

Proof. By definition, $\ell_p(\phi)=\ell(\hat{\beta}(\phi),\phi)$. Applying the chain rule gives $$ \frac{d}{d\phi}\ell_p(\phi) = \left(\frac{\partial\ell}{\partial\beta}\right)^{\!T} \frac{d\hat{\beta}(\phi)}{d\phi} + \frac{\partial\ell}{\partial\phi}, $$ where the derivatives on the right are evaluated at $(\hat{\beta}(\phi),\phi)$. The first term is zero by the interior score equation for $\beta$. The remaining term is the stated profile derivative. $\square$

This result explains why the profile score can be calculated by differentiating with respect to dispersion while holding the profiled regression estimate fixed at its value for that candidate dispersion. It does not say that dispersion and regression are independent; their dependence is already reflected in $\hat{\beta}(\phi)$.

Gaussian Model: Estimation and Profile-Likelihood Uncertainty

The normal model gives a particularly transparent derivation. Let $Y_i$ be independent with $Y_i\sim N(x_i^T\beta,\phi)$, where the design matrix has rank $p$, and $\phi>0$. Let $\operatorname{RSS}(\beta)=\sum_{i=1}^n(y_i-x_i^T\beta)^2$. The log-likelihood, apart from no omitted parameter-dependent terms, is

\ell(\beta,\phi) = -\frac{n}{2}\log(2\pi\phi) -\frac{\operatorname{RSS}(\beta)}{2\phi}.

For each fixed $\phi$, maximizing over $\beta$ minimizes the residual sum of squares, so the profile uses the least-squares estimate and $S=\operatorname{RSS}(\hat{\beta})$. If $S>0$, then $$ \ell_p(\phi) = -\frac{n}{2}\log(2\pi\phi)-\frac{S}{2\phi}. $$

Result: If $S>0$, the maximum-likelihood estimate of the Gaussian dispersion is $\hat{\phi}=S/n$. The profile likelihood-ratio statistic for testing a candidate $\phi_0>0$ is $$ 2\{\ell_p(\hat{\phi})-\ell_p(\phi_0)\} = n\left[ \log\left(\frac{\phi_0}{\hat{\phi}}\right) +\frac{\hat{\phi}}{\phi_0}-1 \right]. $$

Derivation. Differentiating the profile log-likelihood gives $$ \frac{d}{d\phi}\ell_p(\phi) = -\frac{n}{2\phi}+\frac{S}{2\phi^2} = \frac{S-n\phi}{2\phi^2}. $$ Since $\phi^2>0$, the derivative is positive for $\phi<S/n$ and negative for $\phi>S/n$. Therefore the unique maximum is at $\hat{\phi}=S/n$. Substituting $S=n\hat{\phi}$ into twice the log-likelihood difference yields the displayed statistic: $$ 2\{\ell_p(\hat{\phi})-\ell_p(\phi_0)\} = n\log\left(\frac{\phi_0}{\hat{\phi}}\right) +\frac{S}{\phi_0}-\frac{S}{\hat{\phi}} = n\left[ \log\left(\frac{\phi_0}{\hat{\phi}}\right) +\frac{\hat{\phi}}{\phi_0}-1 \right]. $$ The derivative sign establishes that this is a global maximum and completes the derivation. $\square$

Worked Example: Profiling the Variance in a Normal-Mean Model

Suppose five independent observations follow $N(\mu,\phi)$, with unknown mean and dispersion. The data are $1,2,2,3,2$. Find the maximum-likelihood estimates and calculate the profile likelihood-ratio statistic for $\phi_0=1$.

The least-squares estimate of the mean is $$ \hat{\mu}=\frac{1+2+2+3+2}{5}=2. $$ The residual sum of squares is $$ S=(1-2)^2+(2-2)^2+(2-2)^2+(3-2)^2+(2-2)^2=1+0+0+1+0=2. $$ Thus $\hat{\phi}=S/n=2/5=0.4$. The likelihood-ratio statistic at $\phi_0=1$ is $$ 5\left[\log\left(\frac{1}{0.4}\right)+\frac{0.4}{1}-1\right] = 5[\log(2.5)-0.6] \approx 5(0.91629-0.6) \approx 1.5815. $$ This is the profile likelihood comparison for $\phi=1$ against the fitted value $0.4$; it is not a Wald statistic based only on a standard error.

Boundary Constraints and Fixed Dispersion

A likelihood maximum may be restricted by the parameter space. For example, a model might require $\phi\geq a$ for a specified positive lower bound $a$. In the Gaussian model with $S>0$, the profile likelihood increases up to $S/n$ and decreases afterward. Consequently, maximizing over $[a,\infty)$ either leaves the unconstrained estimate unchanged or places the estimate at the lower boundary.

Result: In the Gaussian model above, under the constraint $\phi\geq a>0$, the constrained maximum-likelihood estimate is $$ \hat{\phi}_a=\max\left\{a,\frac{S}{n}\right\}. $$

Proof. If $S/n\geq a$, the unconstrained maximizer belongs to the constrained parameter space, so it remains the constrained maximizer. If $S/n<a$, then $S-n\phi<0$ for every $\phi\geq a$. The derivative of the profile log-likelihood is therefore negative throughout the constrained space, so the maximum occurs at its smallest admissible value $a$. These two cases give the stated expression. $\square$

Worked Example: A Lower-Bound Constraint Changes the Estimate

Use the data in the preceding normal example, but impose the constraint $\phi\geq 0.5$. Since $S/n=0.4<0.5$, the constrained estimate is $\hat{\phi}_{0.5}=0.5$. For comparison, evaluate twice the profile log-likelihood difference between the constrained maximum and $\phi=0.8$: $$ 2\{\ell_p(0.5)-\ell_p(0.8)\} = 5\log\left(\frac{0.8}{0.5}\right) +2\left(\frac{1}{0.8}-\frac{1}{0.5}\right) = 5\log(1.6)+2(1.25-2) \approx 2.3500-1.5000 = 0.8500. $$ The constrained maximum is at $0.5$, not at the unconstrained value $0.4$. Any likelihood-ratio comparison must use the constrained maximum in its numerator.

Worked Example: Why a Poisson GLM Has No Free Dispersion Estimate

Suppose independent counts satisfy $Y_i\sim\operatorname{Poisson}(\mu_i)$, with $\log(\mu_i)=x_i^T\beta$. The probability mass function is $$ P(Y_i=y_i)=\exp\{y_i\log(\mu_i)-\mu_i-\log(y_i!)\}. $$ There is no free $\phi$ in this model: its variance equals its mean, and the dispersion is fixed at $1$. Thus, even if a fitted model has counts whose variability seems larger than expected, there is no likelihood estimate of a free dispersion parameter under this specified Poisson distribution. Estimating such a scale requires a model with an appropriate full probability law; merely inserting $\phi$ into a variance expression does not define a likelihood.

A boundary can also be more serious than an estimate being clipped by a constraint. In the normal model with unknown mean, if all observed values are identical, then the fitted residual sum of squares is $S=0$, and $$ \ell_p(\phi)=-\frac{n}{2}\log(2\pi\phi). $$ As $\phi$ decreases toward zero, this log-likelihood increases without bound. There is no finite positive dispersion value maximizing it. This illustrates why the formal domain $\phi>0$ does not guarantee that a maximum-likelihood estimate exists.

A standard chi-squared profile-likelihood cutoff should not be used automatically when the estimate is on a boundary or the likelihood has no finite maximum. Boundary behavior can change the limiting distribution, and in cases such as the zero-residual normal example there is no ordinary interior likelihood approximation to rely on. The precise boundary analysis depends on the probability model and the way the boundary enters its parameter space.

Common Mistakes and Practical Interpretation

Ignoring the full dispersion likelihood. The term $c(y_i,\phi)$ may contribute to the dispersion score. Differentiate the full log-likelihood, not just the part containing $1/\phi$.
Using a residual degrees-of-freedom divisor as though it were always the MLE. In the Gaussian model, the likelihood estimate is $S/n$; a degrees-of-freedom adjustment is a different estimator. Other response distributions have their own likelihood equations.
Treating every generalized linear model as having an estimable dispersion. Standard Poisson and binomial models fix dispersion at $1$. A free scale parameter must belong to the specified likelihood, not just to a variance formula.
Applying an interior likelihood-ratio cutoff at a boundary without checking conditions. The usual chi-squared calibration assumes regular interior behavior. A constrained maximum or a non-finite likelihood maximum can invalidate that approximation.
Profiling incorrectly. For every candidate dispersion value, maximize over the regression coefficients before comparing likelihoods. Evaluating all candidate dispersions at a single fixed coefficient estimate is generally not the profile likelihood.

Key takeaway: Likelihood inference for dispersion starts with the full probability model and its parameter space. Profile out the regression coefficients, locate the maximum within the admissible set, and use interior likelihood-ratio calibration only when its regularity conditions are credible.

Check Your Understanding

Use the definitions and derivations above to answer the following.

Why can the dependence of $c(y_i,\phi)$ on $\phi$ affect the maximum-likelihood estimate of dispersion?
In the Gaussian model, what feature of the profile score shows that $S/n$ is a global maximum when $S>0$?
For a profile likelihood, what is maximized separately for each candidate value of $\phi$?
What does a standard Poisson GLM assume about dispersion, and why does that prevent estimating a free dispersion parameter from its likelihood?
Why is the usual chi-squared profile likelihood-ratio calibration not automatic when the estimate occurs at a boundary?