Degrees of Freedom for Few-Cluster GEE Tests

Why Degrees of Freedom Matter With Few Clusters

In GEE, the sandwich covariance estimator is justified by large-sample arguments in which the number of independent clusters grows. With few clusters, the estimated uncertainty can be highly variable, and a standard normal or chi-square reference may give tests that are too liberal. A finite-sample reference distribution can account, approximately, for uncertainty in the estimated variance as well as uncertainty in the contrast estimate.

Degrees of freedom are not an automatic property of a GEE coefficient. They depend on how the variance estimate is constructed and on the approximation used to calibrate the test. In particular, there is no single universal Satterthwaite degrees-of-freedom formula for every robust GEE covariance estimator. The useful general principle is to approximate the estimated variance by a scaled chi-square distribution, matching its first two moments.

Scalar GEE Contrasts and the Satterthwaite Approximation

Definition: For a fitted GEE coefficient vector $\hat{\beta}$, a scalar contrast is $\hat{\psi}=c^{T}\hat{\beta}$, where $c$ is a specified nonzero vector. Its estimated robust variance is $\hat{V}_{c}=c^{T}\hat{\Sigma}c$, where $\hat{\Sigma}$ is the estimated covariance matrix of $\hat{\beta}$. To test $H_{0}:c^{T}\beta=\psi_{0}$, the usual contrast statistic is $T=(\hat{\psi}-\psi_{0})/\sqrt{\hat{V}_{c}}$.

Suppose the estimated variance can be expressed as a sum of independent component estimates, $\hat{V}_{c}=\sum_{j=1}^{J}\hat{V}_{j}$. Write $V_{j}=E[\hat{V}_{j}]$, and suppose each component has variance $2V_{j}^{2}/\nu_{j}$, where $\nu_{j}>0$ is its component degrees of freedom. These assumptions describe a moment approximation; they do not follow automatically from fitting a GEE.

Definition: The Welch–Satterthwaite approximation assigns the total variance estimate the effective degrees of freedom $$ \nu=\frac{\left(\sum_{j=1}^{J}V_{j}\right)^{2}}{\sum_{j=1}^{J}V_{j}^{2}/\nu_{j}}. $$ In practice, unknown component variances are replaced by their estimated values. The resulting $\hat{\nu}$ is an estimated, generally non-integer degrees of freedom.

The formula gives less weight to components that are estimated with low precision. It is useful only when the variance components and their degrees of freedom have a defensible interpretation under the chosen approximation. Merely having one score contribution per cluster does not, by itself, establish that each contribution is an independent variance-component estimate with a known degrees of freedom.

Why the Moment-Matching Formula Has This Form

Theorem: Suppose $\hat{V}_{1},\ldots,\hat{V}_{J}$ are independent, with $E[\hat{V}_{j}]=V_{j}$ and $\operatorname{Var}(\hat{V}_{j})=2V_{j}^{2}/\nu_{j}$. Let $V=\sum_{j=1}^{J}V_{j}>0$. A scaled chi-square variable $a\chi^{2}_{\nu}$, matched to the mean and variance of $\hat{V}_{c}=\sum_{j=1}^{J}\hat{V}_{j}$, has $\nu=V^{2}/\sum_{j=1}^{J}V_{j}^{2}/\nu_{j}$ and $a=V/\nu$.

Proof. Independence makes the variance of the sum equal to the sum of the component variances:

E[\hat{V}_{c}]=\sum_{j=1}^{J}V_{j}=V,\qquad \operatorname{Var}(\hat{V}_{c})=2\sum_{j=1}^{J}\frac{V_{j}^{2}}{\nu_{j}}.

For a chi-square variable with $\nu$ degrees of freedom, the mean is $\nu$ and the variance is $2\nu$. Therefore $a\chi^{2}_{\nu}$ has mean $a\nu$ and variance $2a^{2}\nu$. Matching the mean requires $a\nu=V$, so $a=V/\nu$. Matching the variance then requires

2a^{2}\nu=2\sum_{j=1}^{J}\frac{V_{j}^{2}}{\nu_{j}} \quad\Longrightarrow\quad \frac{V^{2}}{\nu}=\sum_{j=1}^{J}\frac{V_{j}^{2}}{\nu_{j}} \quad\Longrightarrow\quad \nu=\frac{V^{2}}{\sum_{j=1}^{J}V_{j}^{2}/\nu_{j}}.

Thus the stated scaled chi-square matches both moments. Matching two moments does not prove that the actual variance estimate has a chi-square distribution; that distribution is an approximation. $\square$

Worked Example: Combining Two Variance Components

Suppose a scalar GEE contrast has two estimated variance components, $\hat{V}_{1}=0.16$ and $\hat{V}_{2}=0.09$. Their assigned component degrees of freedom are $\nu_{1}=4$ and $\nu_{2}=8$, respectively. Apply the Welch–Satterthwaite approximation.

The total estimated variance is $\hat{V}_{c}=0.16+0.09=0.25$. Substituting the component estimates into the formula gives

\hat{\nu} =\frac{(0.16+0.09)^{2}}{0.16^{2}/4+0.09^{2}/8} =\frac{0.0625}{0.0064+0.0010125} =\frac{0.0625}{0.0074125} \approx 8.43.

The estimated standard error is $\sqrt{0.25}=0.5$. The components do not simply contribute $4+8=12$ degrees of freedom: their variance-estimation precision differs, so moment matching gives about $8.43$. This calculation is justified as an approximation only if the component estimates and their assigned degrees of freedom are appropriate for the variance decomposition being used.

From a Variance Approximation to a t Test

For a finite-sample t reference, a further approximation is needed: the standardized contrast numerator must behave approximately like a standard normal variable and be approximately independent of the estimated variance. This independence is exact in some classical normal-theory settings, but generally is not exact for a GEE sandwich estimator.

Theorem: If $Z\sim N(0,1)$ and $U\sim\chi^{2}_{\nu}$ are independent, then $Z/\sqrt{U/\nu}$ has a Student t distribution with $\nu$ degrees of freedom. Consequently, if under the null the contrast numerator is approximately normal and independent of a variance estimate approximated by $(V/\nu)\chi^{2}_{\nu}$, a t reference with $\nu$ degrees of freedom is a moment-based finite-sample approximation.

Derivation. By definition, a Student t variable with $\nu$ degrees of freedom is the ratio of an independent standard normal variable and the square root of an independent chi-square variable divided by its degrees of freedom. If $\hat{V}_{c}$ is approximated by $(V/\nu)U$, then

\frac{\hat{\psi}-\psi_{0}}{\sqrt{\hat{V}_{c}}} \ \approx\ \frac{Z}{\sqrt{U/\nu}}.

The right-hand side has the stated t distribution under the theorem's independence and distribution assumptions. For GEE, these assumptions are approximations, not an exact consequence of the sandwich formula. $\square$

For a two-sided level-$\alpha$ test, compare $|T|$ with the $1-\alpha/2$ quantile of $t_{\hat{\nu}}$, or report the corresponding t-based p-value. A confidence interval uses the same reference: $\hat{\psi}\pm t_{\hat{\nu},1-\alpha/2}\sqrt{\hat{V}_{c}}$. As $\nu$ increases, the t distribution approaches the standard normal distribution.

Worked Example: How a t Reference Changes a Contrast Test

Suppose $\hat{\psi}=0.80$, the null value is $\psi_{0}=0$, and the robust standard error is $0.50$. Suppose a defensible variance-component approximation gives $\hat{\nu}=8.43$, as in the preceding calculation.

The observed statistic is

T=\frac{0.80-0}{0.50}=1.60.

Using a standard normal reference gives a two-sided p-value of approximately $2\{1-\Phi(1.60)\}=0.110$. Using a t reference with $8.43$ degrees of freedom gives a larger two-sided p-value, approximately $0.15$, because the t distribution has heavier tails at this small degrees of freedom. At the 5% level, both references fail to reject the null in this example; their p-values nevertheless show that the reference choice affects the strength of the evidence.

For comparison, the 95% normal critical value is about $1.96$, while the 95% two-sided t critical value for about $8.43$ degrees of freedom is about $2.30$. The corresponding intervals are approximately $0.80\pm1.96(0.50)=( -0.18,1.78)$ under the normal reference and $0.80\pm2.30(0.50)=( -0.35,1.95)$ under the t reference. The latter is wider because it reflects a finite-sample calibration.

Joint Contrasts and F References

For $q$ restrictions, write the null hypothesis as $H_{0}:C\beta=d$, where $C$ has rank $q$. The joint Wald statistic is

W=(C\hat{\beta}-d)^{T}(C\hat{\Sigma}C^{T})^{-1}(C\hat{\beta}-d).

Under the usual large-cluster asymptotics and a nonsingular contrast covariance matrix, $W$ is compared with a chi-square distribution with $q$ degrees of freedom. A finite-sample alternative is to compare $W/q$ with an $F$ distribution having $q$ numerator degrees of freedom and an approximate denominator degrees of freedom $\nu$. This is a distinct calibration from applying a scalar t test separately to each contrast. The denominator degrees of freedom for a joint test require an approximation appropriate to that joint covariance; a scalar contrast's $\hat{\nu}$ cannot automatically be reused.

Worked Example: A Two-Restriction Wald Test

Suppose a joint GEE test has $q=2$ restrictions and observed Wald statistic $W=8$. For illustration, suppose a chosen joint moment-matching method supplies denominator degrees of freedom $\nu=8$. The finite-sample statistic and reference are

F=\frac{W}{q}=\frac{8}{2}=4,\qquad F\sim F_{2,8}\ \text{under the approximate null reference}.

For $F_{2,8}$, the upper-tail probability at 4 is $\{8/(8+2\cdot4)\}^{4}=0.5^{4}=0.0625$. The asymptotic chi-square reference instead gives $P(\chi^{2}_{2}\geq8)=e^{-8/2}\approx0.0183$. The finite-sample F calibration therefore does not reject at the 5% level, whereas the chi-square calibration does. This example demonstrates why the reference distribution can matter with few clusters; the assumed denominator degrees of freedom must still be justified by the method used.

Choosing and Interpreting Degrees of Freedom

A Satterthwaite calculation should begin with a specified variance approximation, not with a desired test distribution. Some methods estimate variance components and their degrees of freedom directly; others use moment matching for a contrast or for a vector of contrasts. A simple rule based on the number of clusters minus the number of fitted coefficients is sometimes used as a rough conservative convention, but it is not the Welch–Satterthwaite formula and is not universally valid for every GEE design or covariance estimator.

The number of independent clusters, their sizes, the leverage of individual clusters, and the way the robust covariance is adjusted can all affect small-sample behavior. A leverage-adjusted covariance estimator may change the estimated variance, but it does not by itself specify the reference distribution or degrees of freedom. A report should state the covariance estimator, the contrast, the reference distribution, and how its degrees of freedom were obtained.

Use a scalar t approximation when a contrast-level variance approximation and its degrees of freedom are available and defensible.
Use a joint F approximation only when the denominator degrees of freedom are calibrated for the joint test, rather than borrowed automatically from one coefficient.
Retain the asymptotic reference as a comparison when useful, while recognizing that it can be optimistic with few independent clusters.
Do not interpret a fractional degrees of freedom as a literal cluster count. It summarizes the variability represented by a moment approximation.

Common Pitfall

Common mistake: “There are 12 clusters, so every coefficient test should use $t_{11}$, and every two-coefficient test should use $F_{2,11}$.”

Correction: A cluster-count rule may be used as a stated convention in some settings, but it is not a universal consequence of GEE. A Satterthwaite degrees of freedom depends on how the contrast variance is estimated and on the variability of that estimate. For joint tests, the relevant approximation must also account for the joint covariance. State the method and its assumptions; do not treat a convenient cluster-count formula as an exact result.

The practical lesson is that finite-sample degrees of freedom are a calibration choice supported by assumptions. They can make inference more cautious when variance estimation is unstable, but they cannot repair a poorly estimated covariance matrix or guarantee correct size in every few-cluster design.

Check Your Understanding

Use these questions to check both the calculation and the assumptions behind a finite-sample reference.

For independent variance components, explain why the Satterthwaite denominator contains a sum of squared component variances divided by component degrees of freedom.
A contrast has estimated variance $0.25$ and estimated degrees of freedom $8.43$. What standard error and reference distribution would you use for its approximate two-sided test?
Why does a t reference generally have heavier tails than a standard normal reference when its degrees of freedom are small?
Why should the denominator degrees of freedom for a joint F test not automatically be copied from a scalar contrast?
What additional assumptions are needed to interpret a Satterthwaite t approximation as more than a moment-matching device?