Degrees of Freedom for Fixed-Effect Tests

Why Degrees of Freedom Need Special Attention

In a linear mixed model, the uncertainty of a fixed-effect estimate depends on the covariance structure of the repeated observations. That structure includes variance components, which are usually estimated rather than known. Consequently, the estimated standard error of a fixed-effect estimate also has uncertainty. This matters especially when the number of independent subjects or clusters is modest: treating the standardized estimate as if it had a standard normal distribution can produce tests that are too optimistic and intervals that are too narrow.

For an ordinary linear model with independent, normally distributed errors and a common variance estimated from the residuals, a Student \(t\) reference distribution accounts for estimating that variance. In a mixed model, the analogous denominator degrees of freedom are generally not obvious. The number of observations is not, by itself, the number of independent pieces of information for every fixed effect. Degrees-of-freedom approximations such as Satterthwaite and Kenward–Roger provide a practical way to reflect some of the additional uncertainty.

Let \(Y\) be the vector of longitudinal outcomes, \(X\) the fixed-effects design matrix, and \(Z\) the random-effects design matrix. A common model is \(Y=X\beta+Zb+\varepsilon\), where \(b\) is a vector of random effects and \(\varepsilon\) is a vector of residual errors. The marginal covariance matrix of \(Y\) is \(V(\theta)\), determined by covariance parameters \(\theta\). If \(\theta\) were known, the generalized least-squares estimate would be

In practice, \(V\) is replaced by \(V(\hat{\theta})\). A single fixed-effect contrast \(L=l^\mathsf{T}\beta\) is estimated by \(\hat{L}=l^\mathsf{T}\hat{\beta}\), and its estimated variance depends on \(\hat{\theta}\). A common test statistic is \(\hat{L}/SE(\hat{L})\). The question is what reference distribution should be used for that statistic when the standard error is estimated.

Satterthwaite Approximation

The Satterthwaite method approximates the sampling distribution of an estimated variance by matching its first two moments to a scaled chi-square distribution. The idea can be shown first for a variance estimate \(Q\) that is approximately a weighted sum of independent scaled chi-square variables. If \(Q\) has mean \(m\) and variance \(v\), approximate it by \(m\chi^2_{\nu}/\nu\). That scaled chi-square has mean \(m\) and variance \(2m^2/\nu\). Matching the variance gives the approximate degrees of freedom.

For a mixed-model contrast, let \(g(\theta)=\operatorname{Var}(\hat{L}\mid\theta)\), and let \(\hat{g}=g(\hat{\theta})\) be its estimated variance. A first-order delta-method approximation estimates the variance of \(\hat{g}\) as \(\nabla g^\mathsf{T}\operatorname{Cov}(\hat{\theta})\nabla g\), evaluated at the fitted covariance parameters. Replacing \(m\) by \(\hat{g}\) and \(v\) by that estimated variance gives the Satterthwaite degrees of freedom:

This expression makes the intuition explicit: greater uncertainty in the estimated contrast variance produces fewer effective degrees of freedom. It also indicates the approximation's conditions. The variance estimate should be reasonably represented by a scaled chi-square distribution, the covariance-parameter estimates should have a usable estimated covariance matrix, and the local linear approximation to \(g(\theta)\) should be adequate. The method is an approximation, not a universal exact result for every mixed model.

From Approximate Degrees of Freedom to Inference

Once an approximate denominator degrees of freedom has been obtained, a one-contrast test uses a \(t\) reference distribution. For \(H_0:L=L_0\), the test statistic is \(t=(\hat{L}-L_0)/SE(\hat{L})\). A two-sided \(100(1-\alpha)\%\) confidence interval is \(\hat{L}\pm t_{1-\alpha/2,df}SE(\hat{L})\). The estimate and standard error retain their usual units; the \(t\) statistic and its degrees of freedom are dimensionless.

The degrees of freedom are not a correction to the point estimate. In the usual Satterthwaite calculation, the fitted contrast estimate and its estimated standard error are retained, while the reference distribution is changed to reflect uncertainty in the variance estimate. The result can materially affect the critical value and \(p\)-value when the effective degrees of freedom are small.

Kenward–Roger Adjustment

Kenward–Roger (KR) is another small-sample approach for fixed-effect inference in linear mixed models. It adjusts the estimated covariance matrix of the fixed effects to account for the fact that covariance parameters have been estimated. It also supplies approximate denominator degrees of freedom and, for multi-parameter tests, a scaling adjustment to the approximate \(F\) statistic. For a single contrast, the reported inference is commonly expressed as a \(t\) test; equivalently, its squared statistic is a one-degree-of-freedom \(F\) statistic.

Unlike the simple moment-matching expression, the full KR calculation involves model-specific covariance derivatives and finite-sample adjustments. It is generally obtained from statistical software rather than calculated by hand. Neither method is automatically preferable in every setting: their results can differ because the adjustments are constructed differently, and both rely on the fitted model and an adequate approximation.

Choosing, Checking, and Interpreting the Approximation

Both methods are intended to improve fixed-effect inference when covariance parameters are estimated and sample sizes are not large. The choice may depend on the analysis plan, software, model complexity, and the available implementation. Satterthwaite is based on approximating the uncertainty in a particular estimated variance; KR makes additional finite-sample adjustments to the fixed-effect covariance estimate and test statistic. For multi-degree-of-freedom fixed-effect tests, KR's scaling adjustment can be relevant, so a single-contrast calculation should not be substituted for a joint test.

Before trusting either result, check that the model has been fitted successfully and that the covariance structure is scientifically plausible. Degrees of freedom that are extremely small, unexpectedly sensitive to a minor model change, or unavailable from the software can signal weak information about covariance parameters or numerical difficulty. A variance estimate at or near a boundary, a nearly singular estimated covariance matrix, or very few independent subjects can also make approximations fragile. These warning signs motivate careful review of the model and design; they do not, on their own, identify a single correct alternative.

A useful distinction is between uncertainty calibration and model adequacy. Satterthwaite and KR address aspects of finite-sample uncertainty in fixed-effect inference. They do not repair a wrong mean structure, a poorly chosen residual covariance, influential subjects, or a fundamentally unsupported random-effects structure. Those issues require attention to the model itself. The ordinary normal (large-sample Wald) approximation is an alternative, but it may be unreliable when the effective number of independent units is small. Reporting the method used, the estimated degrees of freedom, the standard error, and the test or interval makes the inference easier to evaluate.

• Do not equate observations with degrees of freedom. Repeated measurements on one subject are correlated and are not generally equivalent to the same number of independent subjects.
• Do not treat fractional degrees of freedom as an error. They are a normal outcome of moment matching and are used as reported.
• Do not expect an adjustment to fix misspecification. A refined reference distribution cannot compensate for a materially unsuitable covariance model.