Why Complete-Data Degrees Of Freedom Matter
Multiple imputation introduces uncertainty because different plausible completed datasets can produce different estimates. But that is not the only source of uncertainty in an analysis. Even if every missing value were known, an estimate from a modest-sized sample can have a reference distribution with finite degrees of freedom. A pooled analysis must not confuse uncertainty from a limited number of imputations with the sampling limitations of the complete-data analysis.
The distinction matters for inference. A reference distribution with few degrees of freedom has heavier tails than a standard normal distribution, so it generally leads to a larger critical value and wider confidence intervals. Rubin’s large-sample degrees-of-freedom calculation reflects the finite number of imputations, but by itself it does not fully represent finite complete-data degrees of freedom.
The regression formula counts independent residual information after estimating the \(k\) linearly independent regression coefficients, including the intercept if it is in the model. It is not generally correct to use \(n-1\), nor is \(\nu_{\mathrm{com}}\) always determined by sample size alone. For other analyses, the appropriate complete-data degrees of freedom depend on the model and the reference distribution used for the complete-data estimate.
What Rubin’s Large-Sample Degrees Of Freedom Capture
For a scalar estimand, let \(Q_j\) be its estimate and \(U_j\) its estimated variance in completed dataset \(j\), for \(j=1,\ldots,m\). The pooled estimate is \(\bar Q\), the mean of the \(m\) estimates. The pooled within-imputation variance is \(\bar U\), the mean of the \(U_j\)'s, and the between-imputation variance is \(B\), the sample variance of the \(Q_j\)'s. Rubin’s total variance estimate is \(T=\bar U+(1+1/m)B\).
The relative increase in variance \(r\) is the additional variance contribution, including the finite-\(m\) correction, relative to the pooled within-imputation variance. Rubin’s large-sample degrees of freedom are denoted here by \(\nu_{\mathrm{old}}\):
This calculation primarily reflects how much the imputations disagree and how many imputations were used. For positive finite \(r\), \(\nu_{\mathrm{old}}\) is greater than \(m-1\); as \(r\) increases, it decreases toward \(m-1\) from above. As \(r\) approaches zero from above, it grows without bound. Thus, a small \(r\) can produce a very large \(\nu_{\mathrm{old}}\), even when the complete-data analysis has only a modest number of degrees of freedom.
That is the key point: a large \(\nu_{\mathrm{old}}\) does not establish that the complete-data analysis has large-sample information. When \(\nu_{\mathrm{com}}\) is finite, an inference procedure that uses only \(\nu_{\mathrm{old}}\) may use a reference distribution with tails that are too light for the complete-data sampling uncertainty. Small-sample pooling methods incorporate this additional information; simply relabeling \(\nu_{\mathrm{old}}\) as the complete-data degrees of freedom would be incorrect.
Why Regression Residual Degrees Of Freedom Have This Role
Consider a complete-data linear regression with a fixed, full-rank design matrix \(X\), independent normally distributed errors with mean zero and common variance, and \(k\) linearly independent columns in \(X\). Under these assumptions, the usual coefficient estimate has a \(t\) reference distribution with \(n-k\) degrees of freedom when its standard error uses the residual variance estimate.
Here \(\hat\beta_a\) estimates coefficient \(\beta_a\), \(s^2\) estimates the error variance using the \(n-k\) residual degrees of freedom, and the expression in square brackets is the \(a\)th diagonal element of \((X^\mathsf{T}X)^{-1}\). The finite degrees of freedom arise because the error variance is estimated, not known. In a multiple-imputation analysis, each completed dataset still requires an appropriate complete-data analysis. Variation across imputed datasets adds uncertainty, but does not make the residual degrees of freedom disappear.
This derivation is specific to the stated regression assumptions. In a different model, one should use the degrees of freedom associated with that model’s complete-data inference, rather than applying \(n-k\) automatically. The same caution applies if design rank changes between completed datasets: the analysis and its degrees of freedom need to be examined rather than presumed identical.
Worked Examples
Worked Example: A Regression With 36 Complete-Data Degrees Of Freedom
Suppose a hypothetical study has \(n=42\) observations and a regression design matrix of rank \(k=6\). Across \(m=10\) imputations, suppose the pooled within-imputation variance is \(\bar U=0.0400\) and the between-imputation variance is \(B=0.0100\). The pooled estimate is \(\bar Q=0.800\). Find the complete-data degrees of freedom and compare them with \(\nu_{\mathrm{old}}\).
With the stated regression model and a common full-rank design, the complete-data residual degrees of freedom are \( \nu_{\mathrm{com}}=n-k=42-6=36 \). The relative increase in variance is \( r=(1+1/10)(0.0100)/0.0400=0.2750 \). Therefore,
So the large-sample Rubin calculation gives about \(193.5\) degrees of freedom, while the complete-data regression has 36 residual degrees of freedom. The difference warns that \(\nu_{\mathrm{old}}\) alone does not reflect all the finite-sample information relevant to this analysis.
For scale, \(T=0.0400+1.1(0.0100)=0.0510\), so the pooled standard error is \(\sqrt{0.0510}\approx0.2258\). A standard-normal 95% critical value is 1.960, whereas the \(t_{36}\) critical value is about 2.028. Using the latter only as a complete-data reference benchmark gives \(0.800\pm2.028(0.2258)\), or approximately \([0.3421,1.258]\). This is a comparison of reference distributions, not a claim that \(\nu_{\mathrm{com}}\) alone supplies the final pooled degrees of freedom.
Worked Example: A Large Imputation-Based Value Does Not Mean Large Complete-Data Information
Consider a hypothetical analysis with \(m=5\), \(\bar U=0.1200\), \(B=0.0096\), and \(\bar Q=1.40\). Assume the complete-data analysis has \(\nu_{\mathrm{com}}=12\). Calculate the total variance and \(\nu_{\mathrm{old}}\), then explain their relationship to \(\nu_{\mathrm{com}}\).
First, \(r=(1+1/5)(0.0096)/0.1200=0.096\). Hence,
The total variance is \(T=0.1200+1.2(0.0096)=0.13152\), and the pooled standard error is \(\sqrt{0.13152}\approx0.3627\). Despite the large value of \(\nu_{\mathrm{old}}\), the complete-data analysis has only 12 degrees of freedom. The large value arises because the between-imputation contribution is small relative to \(\bar U\); it does not mean that the complete-data sampling distribution has become normal.
For comparison, using the \(t_{12}\) critical value of about 2.179 as a complete-data benchmark would give a margin of \(2.179(0.3627)\approx0.7903\), rather than the standard-normal margin \(1.960(0.3627)\approx0.7109\). The benchmark illustrates why finite complete-data degrees of freedom can affect uncertainty even when \(\nu_{\mathrm{old}}\) is very large.
Worked Example: When the Number Of Imputations Is The More Obvious Constraint
In a hypothetical analysis, let \(m=5\), \(\bar U=0.0400\), \(B=0.0640\), and \(\bar Q=1.50\). Suppose \(\nu_{\mathrm{com}}=100\). Calculate \(r\), \(\nu_{\mathrm{old}}\), and an interval using the \(t\) reference distribution associated with \(\nu_{\mathrm{old}}\).
The relative increase is \(r=1.25(0.0640)/0.0400=2.00\), so
The total variance is \(T=0.0400+1.25(0.0640)=0.1200\), giving \(SE=\sqrt{0.1200}\approx0.3464\). A \(t_9\) 95% critical value is about 2.262, so the interval using \(\nu_{\mathrm{old}}\) is \(1.50\pm2.262(0.3464)\), approximately \([0.7164,2.284]\).
Here \(\nu_{\mathrm{old}}=9\) is much smaller than \(\nu_{\mathrm{com}}=100\). The example shows that finite imputation-related degrees of freedom can be an important constraint even when the complete-data analysis has many degrees of freedom. It does not imply that complete-data degrees of freedom can be ignored in other settings.
Choosing And Checking The Degrees Of Freedom
The relevant degrees of freedom depend on the source of uncertainty and on the assumptions behind the analysis. A useful diagnostic is to calculate \(\nu_{\mathrm{com}}\) and \(\nu_{\mathrm{old}}\) separately before drawing conclusions from either one. A large discrepancy, such as a very large \(\nu_{\mathrm{old}}\) alongside a small \(\nu_{\mathrm{com}}\), is a signal to use a small-sample pooling approach that accounts for complete-data information as well as imputation uncertainty.
| Quantity | What it describes | Typical role |
|---|---|---|
| \(\nu_{\mathrm{com}}\) | Degrees of freedom for the complete-data analysis | Reflects finite-sample information under the complete-data model |
| \(\nu_{\mathrm{old}}\) | Degrees of freedom from the large-sample Rubin calculation | Reflects imputation variation and the finite number of imputations |
| Normal reference | Large-sample approximation | Can be inadequate when relevant degrees of freedom are small |
A tempting shortcut is to use the smaller of \(\nu_{\mathrm{old}}\) and \(\nu_{\mathrm{com}}\). This cap may seem cautious, but it is only a heuristic: it does not, by itself, provide a principled calculation that combines both sources of information. Conversely, using the normal distribution whenever \(\nu_{\mathrm{old}}\) is large can understate uncertainty when \(\nu_{\mathrm{com}}\) is limited. A finite-sample pooling adjustment is the appropriate alternative when both sources matter.
Check the complete-data degrees of freedom against the actual analysis. In ordinary least squares, verify the design-matrix rank, not just the number of named predictors. In other models, do not assume that the regression residual formula applies. Also check that the completed-data analyses estimate the same scalar quantity and use compatible variance estimates; otherwise, a single common \(\nu_{\mathrm{com}}\) may not adequately summarize the situation.
Common Pitfalls And Limitations
- Using sample size as the degrees of freedom. In regression, the standard residual degrees of freedom are \(n-k\), not automatically \(n\) or \(n-1\). Count the estimable model parameters through the design-matrix rank.
- Treating \(\nu_{\mathrm{old}}\) as a complete-data quantity. It is driven by \(m\) and the between-imputation contribution. A large value does not establish that the completed-data estimate has a large-sample reference distribution.
- Assuming more imputations fix limited sample information. More imputations can reduce Monte Carlo error in pooled calculations, but do not increase the number of observed individuals or the complete-data residual degrees of freedom.
- Assuming a \(t\) reference is exact for every model. Exact \(t\) results require their stated model assumptions. For other analyses, the complete-data reference distribution and degrees of freedom may be approximate or defined differently.
- Using a simple minimum as a universal correction. Taking the smaller of two degrees-of-freedom values is not a general derivation of a finite-sample pooling method.
The calculations here rely on a scalar estimand, valid completed-data estimates and variance estimates, and imputations that adequately represent the intended missing-data uncertainty. Departures from these conditions can cause poor inference that no degrees-of-freedom adjustment alone can repair. Complete-data degrees of freedom are one essential part of the reference-distribution question, not a substitute for checking the imputation and analysis models.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- A hypothetical regression has \(n=58\) observations and design-matrix rank \(k=9\). What are its complete-data residual degrees of freedom?
- For \(m=4\) and \(r=1\), calculate \(\nu_{\mathrm{old}}\). What does it say, and what does it not say?
- For \(m=5\), \(r=0.1\), and \(\nu_{\mathrm{com}}=10\), calculate \(\nu_{\mathrm{old}}\). Explain why it should not be interpreted as the complete-data degrees of freedom.
- Why does increasing the number of imputations not increase the complete-data residual degrees of freedom?
Solutions. (1) \(58-9=49\) residual degrees of freedom, under the stated ordinary least squares setup. (2) \(\nu_{\mathrm{old}}=(4-1)(1+1)^2=12\). It summarizes the imputation-related calculation for this \(m\) and \(r\); it does not give the complete-data degrees of freedom, which must come from the analysis model. (3) \(\nu_{\mathrm{old}}=4(1+10)^2=484\). Its large value follows from small \(r\), while the complete-data analysis still has 10 degrees of freedom. (4) Increasing \(m\) can improve the Monte Carlo precision of pooling, but it does not change the observed sample size, model rank, or residual degrees of freedom of the complete-data analysis.