Degrees of Freedom and Dispersion Ratios

Why the Denominator Matters

A Pearson statistic or residual deviance is not, by itself, a dispersion ratio. To form the usual ratio, it is divided by the model's residual degrees of freedom. That denominator is determined by how many independent response contributions there are and how many dimensions the fitted mean model uses. Counting coefficients by eye can give the wrong answer when the design matrix contains redundant columns.

This tutorial focuses on that bookkeeping. The ratios are useful summaries of variation relative to a fitted GLM, but their denominators do not make them exact measures of dispersion in every finite sample. In particular, the residual degrees of freedom are not automatically the total number of trials, the number of written model terms, or the number of parameters before accounting for constraints and aliases.

The Two Dispersion Ratios

For responses \(Y_1,\ldots,Y_n\), let \(y_i\) be the observed value and \(\hat\mu_i\) its fitted mean. Write the variance function without a dispersion multiplier as \(V(\mu_i)\), so that a model with dispersion parameter \(\phi\) has \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\). The Pearson statistic compares squared residuals with the fitted variance function:

For example, the Poisson variance function is \(V(\mu)=\mu\). For a binomial response that is a count out of \(m_i\) trials, \(V(\mu_i)=m_i p_i(1-p_i)\), where \(\mu_i=m_i p_i\). The variance function must match the response's scale: a grouped binomial count is not given the variance of a single Bernoulli trial.

The residual deviance compares the fitted model's log-likelihood with that of a saturated model, which fits each response contribution as closely as the response family permits. If \(\ell_{\mathrm{sat}}\) and \(\ell_{\mathrm{fit}}\) denote those log-likelihoods, then

The Pearson and deviance ratios use the same residual degrees of freedom, but their numerators summarize lack of fit in different ways. Neither ratio changes the fitted means; each rescales a diagnostic statistic by the residual degrees of freedom. For Poisson and binomial GLMs, the conventional baseline dispersion is one. A ratio larger than one can indicate extra variation relative to that baseline, but its interpretation also depends on whether the mean model and response assumptions are credible.

Why Model Rank Determines the Degrees of Freedom

A GLM is specified through a model matrix \(X\), whose columns encode the predictors and any intercept or other model terms. The number of columns is not necessarily the number of distinct directions in the fitted model. If some columns are exact linear combinations of others, they do not add a new direction for fitting the means. The rank \(r=\operatorname{rank}(X)\) counts the independent directions.

The usual residual degrees of freedom can be understood as the dimension left after fitting those directions. The following argument gives the precise dimension count. It applies directly to a full-rank, regular local mean model; it explains the conventional residual-df calculation, rather than asserting that a nonlinear GLM has an exact finite-sample residual distribution.

In the common GLM setup, the linear predictor is \(X\beta\). If the link is differentiable with nonzero derivative at the fitted means, the local changes in the means are obtained by multiplying changes in \(X\beta\) by a nonsingular diagonal scaling. Under these regularity conditions, that scaling does not change rank, so the local fitted-mean dimension is the rank of \(X\).

To verify this, let \(X\) be the original model matrix and let a new matrix be \(XA\), where the columns of \(XA\) span the same space as the columns of \(X\). Equal column spaces have equal dimensions, so \(\operatorname{rank}(XA)=\operatorname{rank}(X)=r\). The residual degrees of freedom are consequently \(n-r\) under either representation. If a matrix includes redundant columns, its rank is still the dimension of its column space, not its raw column count. Therefore the same denominator applies even when individual coefficients cannot all be uniquely identified. \(\square\)

Which Observations Count?

The \(n\) in \(n-r\) counts the independent response contributions represented in the fitted likelihood, not necessarily the number of people, trials, or raw records mentioned in a study. For a grouped binomial model, each grouped count contributes one response row to the model matrix, even when that row summarizes many Bernoulli trials. The rank is then subtracted from the number of grouped response rows.

Common Degrees-of-Freedom Errors

• Using \(n\) rather than \(n-r\). The mistaken reasoning is that every observed response contributes one degree of freedom and none is used by fitting. The correction is to subtract the model rank.
• Subtracting the number of written columns. This is wrong when columns are redundant. Use \(\operatorname{rank}(X)\), which counts independent fitted directions.
• Using the number of trials instead of response rows. In a grouped response model, the contribution count is the number of modeled rows, not the total number of trials summarized by them.
• Using a different denominator for the two ratios. Pearson and deviance ratios may have different numerators, but under this convention both use \(n-r\).
• Calculating a ratio when \(n-r=0\). A saturated model has no residual degrees of freedom under this count, so the ratio is undefined; it is not zero or an infinite-df estimate.

These ratios are diagnostic summaries, not automatic verdicts about a response distribution. A denominator calculated correctly cannot compensate for an unsuitable mean structure, dependence, or a mismatch between the variance function and the data. It does, however, ensure that comparisons are not distorted by a simple rank or counting error.