Nested Quasi-GLMs and F Tests

Why Compare Nested Quasi-GLMs?

A model with additional predictors can usually fit the observed responses at least as closely as a simpler model. The practical question is whether the improvement is large relative to the residual variation. In a quasi-binomial or quasi-Poisson model, that comparison must account for dispersion: variability may exceed what the corresponding binomial or Poisson variance function alone describes.

A nested-model F test summarizes the improvement with three quantities: the reduction in deviance, the number of added parameters, and an estimate of the dispersion. It gives an approximate test of whether the added terms are needed. It is not an exact likelihood-ratio test: a quasi-GLM specifies a mean and variance relationship, not a full response likelihood.

Set Up the Nested Comparison

For the comparison to be interpretable, fit both models to the same response observations, using the same response definition, variance function, weights, and offset. The models must differ only in their mean-model terms. Let \(D_R\) and \(D_F\) be their unscaled deviances, and let \(p_R\) and \(p_F\) be the ranks of their fitted design matrices. Then

The full model’s residual degrees of freedom are typically \(\nu=n-p_F\), where \(n\) is the number of independent observational units and \(p_F\) is the rank of its design matrix. For grouped binomial data, an observational unit is a group, not an individual trial. The degrees of freedom count independent rows of the fitted model, not the total number of trials.

When the models are fitted by maximizing the same quasi-likelihood criterion, the full model’s larger parameter space cannot produce a worse optimum. Equivalently, when the corresponding deviance is minimized, \(D_F\leq D_R\), so \(\Delta D\geq 0\). The comparison is based on this improvement, not on either deviance in isolation.

From Deviance Reduction to an Approximate F Statistic

Under the null hypothesis, the extra terms should improve the fit only by an amount consistent with random variation. In a quasi-GLM, the dispersion parameter \(\phi\) scales the variance: for example, the quasi-Poisson variance is \(\phi\mu_i\), while the quasi-binomial variance for a proportion has the corresponding binomial variance multiplied by \(\phi\). A larger \(\phi\) means that a given deviance reduction is less surprising.

The numerator is the reduction in deviance per added parameter. Dividing by \(\hat{\phi}\) puts that reduction on the scale of residual variation. A small upper-tail probability under the approximate \(F_{q,\nu}\) reference distribution is evidence against the reduced model. The reference distribution is approximate, not exact: in particular, replacing an unknown dispersion by an estimate does not in general produce an exact finite-sample F distribution.

Why This Has an F Reference Distribution

The first step in the justification is an asymptotic one. Under regularity conditions—independent observations, a correctly specified mean and variance relationship, an interior null parameter, full-rank model matrices, and a fixed number \(q\) of added terms—the improvement in quasi-likelihood is locally quadratic. The quasi-score for the added terms, after accounting for the nuisance parameters in the reduced model, is approximately normal with covariance equal to \(\phi\) times its information matrix.

To see the quadratic form behind this approximation, write \(U\) for the efficient quasi-score for the \(q\) added parameters and \(I\) for its information matrix. A second-order expansion of the quasi-likelihood around the reduced fit gives the increase in twice the quasi-log-likelihood as approximately \(U^{\mathsf T}I^{-1}U\). Under the null, \(U\) is approximately \(N(0,\phi I)\). Thus \(I^{-1/2}U/\sqrt{\phi}\) is approximately a \(q\)-dimensional standard normal vector, and

The deviance reduction is the corresponding quasi-likelihood improvement to this order, which gives \(\Delta D/\phi\approx\chi^2_q\). This is an asymptotic quadratic approximation, not an appeal to a likelihood-ratio theorem for a quasi-model.

A second approximation supplies the denominator. If the full-model residual variation is estimated by a Pearson statistic, then under a suitable large-sample approximation the Pearson statistic divided by \(\phi\) behaves roughly like \(\chi^2_\nu\). Consequently, \(\hat{\phi}=X^2_F/\nu\) acts like an estimate based on that residual variation. If the numerator and the dispersion estimate are treated as approximately independent, their ratio has the reference form

In practice, the independence and chi-square approximations are not exact for general quasi-GLMs. They explain why an F reference distribution is useful, but also why the test should be reported as approximate. Estimating dispersion from the full-model deviance instead of its Pearson statistic is another common convention; whichever estimate is used should be stated.

Practical Checks and a Common Pitfall

Before interpreting the result, verify that both fits use the same observations and response setup, that the full model really contains the reduced model, and that \(q\) is the difference in model-matrix ranks. Redundant columns do not add independent parameters, even if they appear as extra terms in a formula. Also report which dispersion estimate was used: changing \(\hat{\phi}\) changes the F statistic.

A common mistake is to calculate \((D_R-D_F)/q\) and compare it directly with a chi-square or F cutoff, ignoring dispersion. For example, in the grouped-trial calculation above, the reduction per added parameter is 4.2, but the test statistic is \(4.2/2.1=2.0\). Treating 4.2 as the F statistic would overstate the evidence because it fails to account for the estimated overdispersion.

• Do not call this an exact likelihood-ratio test. A quasi-model does not specify a full likelihood for the response.
• Do not compare non-nested models with this formula. A deviance difference has the stated interpretation only when the model spaces are nested.
• Do not mix dispersion estimates without saying so. State whether the estimate comes from Pearson residuals, deviance, or another specified source.
• Do not use the total number of binomial trials as the number of independent rows. For grouped data, use the number of groups when calculating the usual residual degrees of freedom.