Degrees of Freedom for Scaled Variance Estimates

Why Degrees of Freedom Matter in a Scaled Analysis

A reference-scaled analysis uses variability in the reference treatment to determine how much uncertainty is compatible with the decision criterion. The estimated reference variance is not the true variance, however. In a finite sample, its precision depends on the information available to estimate it. Degrees of freedom summarize that information in many standard variance calculations: with fewer degrees of freedom, the estimate is less precise and its confidence interval is wider.

This matters directly for a confidence-bound decision. In the FDA linearized criterion, the quantity to be bounded includes a squared treatment difference and a term proportional to the reference variance. When the reference variance is uncertain, substituting its point estimate alone does not account for that uncertainty. A conservative upper bound must reflect how little the data establish about the variance, as well as uncertainty in the treatment difference.

The calculations below explain the role of degrees of freedom using a transparent variance-pivot argument. A fitted mixed model may require an approximate degrees-of-freedom method, and the resulting calculation must follow the applicable analysis specification. The simple chi-square calculation is a teaching framework, not a replacement for a prescribed regulatory procedure.

From a Variance Estimate to a Lower Confidence Bound

Suppose \(s_R^2\) estimates the within-subject reference variance \(\sigma_{wR}^2\), measured on the log scale, and suppose the model supports the pivot \(\nu_v s_R^2/\sigma_{wR}^2\sim\chi^2_{\nu_v}\). Here \(\nu_v\) is the degrees of freedom for the variance estimate, not necessarily the degrees of freedom for the treatment contrast. For a lower one-sided confidence bound with confidence level \(1-\gamma\), let \(\chi^2_{1-\gamma,\nu_v}\) be the \(1-\gamma\) quantile of the chi-square distribution. Then

To justify the direction of the bound, the pivot has probability \(\gamma\) of being below its \(\gamma\) quantile and probability \(1-\gamma\) of being below its \(1-\gamma\) quantile. Equivalently,

Rearranging this event gives \(\sigma_{wR}^2\geq \nu_v s_R^2/\chi^2_{1-\gamma,\nu_v}\), which yields the stated lower bound. A lower bound is relevant here because the reference variance is subtracted in the scaled criterion: a smaller plausible variance gives less credit from scaling and therefore a larger, more conservative value of the criterion.

As \(\nu_v\) decreases, the chi-square distribution is more dispersed. Its upper quantile is larger relative to its degrees of freedom, so \(L_v\) tends to be farther below \(s_R^2\). Thus two studies with the same variance estimate can produce different lower bounds if their variance estimates have different degrees of freedom.

Carrying Variance Uncertainty Into a Confidence-Bound Decision

Let \(\delta\) denote the population log-scale treatment difference and let \(\hat{\delta}\) be its estimate. For the FDA linearized criterion, the population quantity of interest has the form

where \(\theta\) is the nonnegative scaling constant specified by the analysis (\(\theta\geq0\)). The decision criterion requires an upper confidence bound for \(g\) to be below zero, alongside any separate point-estimate condition. The construction that follows illustrates why degrees of freedom for both components matter; it does not replace the exact confidence-bound procedure required for a particular application.

Assume the treatment contrast has estimated standard error \(SE_\delta\) and an applicable \(t\) reference distribution with \(\nu_\delta\) degrees of freedom. A two-sided confidence interval for \(\delta\) with confidence \(1-\alpha/2\) has upper endpoint in absolute value

This is an upper bound on \(|\delta|\), so \(\delta^2\leq\delta_U^2\) whenever \(|\delta|\leq\delta_U\). Separately, construct a lower bound \(L_v\) for \(\sigma_{wR}^2\) with confidence \(1-\alpha/2\). The probability that either of these two bounds fails is at most the sum of their individual failure probabilities, \(\alpha/2+\alpha/2=\alpha\). This Bonferroni argument does not require the two estimates to be independent. On the event that both bounds hold,

Thus \(U_g=\delta_U^2-\theta L_v\) is a conservative upper bound under the stated coverage assumptions. If \(U_g<0\), the bound supports \(g<0\) at a confidence level of at least \(1-\alpha\). If \(U_g\geq0\), this bound does not establish that \(g<0\); it does not prove that \(g\geq0\).

For \(\alpha=0.05\), each component bound has confidence 97.5%. The contrast bound uses \(t_{0.9875,\nu_\delta}\), while the variance lower bound uses \(\chi^2_{0.975,\nu_v}\). The two degrees of freedom can differ because the contrast and the variance component are estimated from different sources of information.

Effective Degrees of Freedom in a Model-Based Estimate

In a simple normal model, the variance degrees of freedom follow directly from the sampling design. In a mixed model, a reference-variance estimate may combine information from multiple components, and no single integer count may describe its uncertainty exactly. One common approximation is the Satterthwaite method: match the estimated variance of a quantity to the variance of a scaled chi-square variable, then use the matching degrees of freedom.

For example, suppose \(Q=\sum_j a_jS_j^2\), where the variance estimates \(S_j^2\) are independent and satisfy \(\nu_j S_j^2/\sigma_j^2\sim\chi^2_{\nu_j}\). The coefficients \(a_j\) are treated as fixed. Then \(E(Q)=\sum_j a_j\sigma_j^2\), and

Approximating \(Q\) by a scaled chi-square variable \(c\chi^2_{\nu_{\mathrm{eff}}}\) gives \(E(Q)=c\nu_{\mathrm{eff}}\) and \(\operatorname{Var}(Q)=2c^2\nu_{\mathrm{eff}}\). Eliminating \(c\) yields

Because the component variances are unknown, a practical calculation substitutes their estimates. This moment-matching approximation is justified only to the extent that the independence and component-distribution assumptions are reasonable. REML estimation by itself does not guarantee that a variance component has an exact chi-square pivot or that this particular degrees-of-freedom approximation is the required one.

Diagnostics, Alternatives, and Limits

A degrees-of-freedom calculation should be checked as part of the model-based analysis, not selected only because it produces a desired decision. Verify which estimate the degrees of freedom apply to, whether the software reports a variance-component degree of freedom or a contrast degree of freedom, and whether the assumed sampling distribution is exact or approximate. Confirm that the variance estimate is positive and that the model and design provide information to distinguish reference variability from other sources of variability. If the fitted model is singular, the estimate is at a boundary, or the reported degrees of freedom are unexpectedly small, a nominal chi-square calculation may not represent the actual uncertainty well.

The main alternatives answer related but distinct questions. A direct chi-square interval is exact under its normal variance model and known degrees of freedom. A Satterthwaite approximation accommodates some combinations of independent variance estimates, but its accuracy depends on the approximation assumptions. Other model-based methods may be specified for particular mixed-model analyses; their uncertainty calculations and degrees of freedom must be interpreted according to that method. A point-estimate substitution, by contrast, avoids a degrees-of-freedom calculation but does not provide a confidence bound for the unknown reference variance.

The generic upper-bound construction also has limits. It uses separate bounds for the contrast and variance and combines them conservatively; another prescribed procedure may account for their joint distribution more directly. The result depends on the chosen confidence allocation, the validity of the contrast standard error and reference distribution, and the model used for the reference variance. Therefore, the calculation illustrates how finite-sample degrees of freedom can affect a scaled decision, but it does not establish that a particular implementation meets every regulatory requirement.

Common Pitfall