Degrees of Freedom in Mixed-Model Tests

Why Degrees of Freedom Matter

A fixed-effect estimate in a pseudobulk mixed model is typically reported with a standard error and a test statistic. With many independent donors, the test statistic may be well approximated by a standard normal distribution. With few donors, however, the variance components used to estimate that standard error are themselves uncertain. A reference distribution with finite degrees of freedom can better reflect this uncertainty, producing a different p-value or confidence interval from a normal approximation.

The key issue is not simply how many pseudobulk rows appear in the data table. Repeated pseudobulk samples from one donor are correlated, and the number of independent donors can be much smaller than the total number of rows. The introductory material on pseudobulk inference with few donors motivates that distinction; here the focus is how it enters fixed-effect tests through the choice of degrees of freedom.

Model and Test

Consider a gene-level response \(y_{ij}\), such as normalized log expression for donor \(i\) under condition or time point \(j\). A simple random-intercept model is

Here \(x_{ij}\) is the fixed-effect design vector, \(\beta\) is the vector of fixed coefficients, \(b_i\) is a donor-specific random intercept, and \(\varepsilon_{ij}\) is residual error. The usual model assumptions include independent donors, random effects independent of residual errors, and the stated covariance structure being adequate. For small-sample t-based inference, the relevant response or contrast errors are also assumed sufficiently close to normal. More elaborate covariance structures change the variance calculation but not the central degrees-of-freedom issue.

A t reference distribution has heavier tails than the standard normal when its degrees of freedom are small. Thus, for the same observed \(t\), a small-df test generally has a larger two-sided p-value and a larger critical value. Degrees of freedom are an approximation to the uncertainty in the estimated variance, not a direct count of cells, genes, or observations.

Satterthwaite Approximation

The Satterthwaite method approximates the sampling distribution of an estimated contrast variance by matching its first two moments to a scaled chi-square distribution. The basic argument is clearest when a variance estimate is a sum of independent components. Suppose \(\hat v=\sum_k\hat v_k\), each component has expectation \(v_k\), and approximately \(\hat v_k\sim v_k\chi^2_{\nu_k}/\nu_k\). Then

Approximate \(\hat v\) by \(v\chi^2_{\nu}/\nu\). This scaled chi-square has expectation \(v\) and variance \(2v^2/\nu\). Equating its variance to the variance of \(\hat v\) gives

In a mixed model, the contrast variance depends on estimated variance components, and the exact distribution need not be a simple sum of independent terms. Software commonly uses a Satterthwaite approximation based on how the estimated contrast variance changes with those components and on the estimated covariance of the components. In a delta-method formulation, if \(g\) is the gradient of the contrast variance with respect to the variance components and \(\widehat{\mathrm{Cov}}(\hat{\theta})\) is their estimated covariance matrix, then an approximate variance for the estimated contrast variance is \(g^{\mathsf T}\widehat{\mathrm{Cov}}(\hat{\theta})g\). Matching moments yields

where \(\hat v\) is the estimated variance of the contrast. This explains why the degrees of freedom can be non-integer and can differ across contrasts in the same fitted model: different contrasts depend differently on uncertain variance components.

How Common Strategies Differ

Kenward–Roger and Satterthwaite are commonly used for Gaussian linear mixed models, but their results need not match. Kenward–Roger modifies the estimated covariance of fixed effects as well as the reference distribution; Satterthwaite focuses on approximating the degrees of freedom. Neither method repairs a misspecified covariance structure, confounding, or too little independent replication. For models fitted with a non-Gaussian response, the available small-sample procedures and their justification depend on the model and software; one should not assume that a Gaussian mixed-model correction transfers automatically.

Donor-Level Replication and a Paired Example

A particularly clear benchmark is a paired design in which each donor contributes one pseudobulk observation under each of two conditions. If donor differences are independent and normally distributed with a common variance, testing the mean difference is a one-sample t test on the donor-level differences. With \(D\) complete donors, its degrees of freedom are exactly \(D-1\) under those assumptions. A random-intercept model for the same balanced two-condition design expresses the pairing differently, but the within-donor contrast is still the information that identifies the condition effect.

Diagnostics, Limitations, and Common Mistakes

Degrees of freedom are part of the inferential method, so their value should be inspected and reported alongside the test statistic and p-value. In software, check which method was requested, whether the reported degrees of freedom are contrast-specific, and whether the fitting procedure actually returned the requested approximation. Different software defaults can produce different tests from the same fitted model. When Satterthwaite and Kenward–Roger give notably different results, that discrepancy signals sensitivity to small-sample variance estimation; it is not a reason to select whichever p-value is preferred.

• Using total rows as independent replication. Repeated rows within a donor are correlated under the model. Identify the independent sampling units and use a model that represents the repeated structure.
• Assuming \(N-p\) is always the right df. That quantity is familiar from simple linear regression, but a mixed model estimates covariance components as well. It does not generally capture the uncertainty of a mixed-model contrast variance.
• Assuming fractional degrees of freedom are an error. Moment matching can produce non-integer values; the t distribution is defined for positive real degrees of freedom.
• Treating the approximation as a remedy for poor design. No degrees-of-freedom method creates donor replication. Very few donors, strong imbalance, boundary variance estimates, or a poorly supported covariance model can make inference fragile.
• Reporting a p-value without its test specification. State the contrast, test statistic, degrees-of-freedom method and value, and p-value. This makes clear whether the result used a finite-df approximation or a large-sample reference.

The appropriate strategy depends on the model, design, and software implementation. For a Gaussian mixed model with a small number of independent donors, a validated Satterthwaite or Kenward–Roger procedure is often preferable to an unqualified normal approximation. In a simple paired design, donor-level differences provide a transparent reference analysis. In every case, the assumptions behind the chosen reference distribution matter: normality of the relevant errors, independent donors, and a plausible covariance structure cannot be replaced by a degrees-of-freedom calculation.