Covariate-Adjusted Statistics and Boundary Calibration

Why Covariance Matters After Adjustment

In a multi-arm trial, each treatment-control statistic is affected by the control observations it shares with the other statistics. Covariate adjustment can change the variances of the treatment-control contrasts and their dependence. Boundary calibration therefore needs the joint null distribution of the adjusted statistics, not just a standard error for each one separately.

The derivation below uses a linear model with a common covariate slope. The covariate values may differ across arms; adjustment accounts for those values under the model. For exact multivariate normal calibration, assume the model is correctly specified, the design matrix is fixed (or condition on it), and errors are independent \(N(0,\sigma^2)\). In particular, the model assumes a common error variance and no treatment-by-covariate interaction. A treatment-control statistic is evaluated under its null hypothesis when constructing a null distribution.

Covariance at One Look

The Frisch–Waugh–Lovell result gives a useful way to see what adjustment does: first remove the linear association with the nuisance covariates from both the outcome and treatment indicators, then estimate the treatment effects from what remains. Define \(M_C=I-C(C'C)^{-1}C'\), assuming \(C\) has full column rank. The adjusted information matrix for the treatment coefficients is \(J=D'M_CD\). If \(J\) is nonsingular, then

Consequently, for treatment contrasts \(c_i'\beta\) and \(c_j'\beta\),

This is the general covariance formula for the specified linear model. It accommodates unequal arm sizes and covariate imbalance through \(J\); it does not require treatment indicators to be uncorrelated with covariates. The covariance of standardized statistics follows by dividing each covariance by the product of the corresponding standard errors. If \(\sigma^2\) is unknown but estimated, this Gaussian calculation describes the known-variance normal case; substituting an estimated variance requires the relevant estimated-variance calibration rather than treating the resulting statistics as exactly normal.

Covariance Across Looks

At successive looks, the data are usually nested: an earlier look’s participants are included in later analyses. Even when each look uses covariate-adjusted regression, the cross-look covariance can be derived directly from the linear weights of the estimates. For look \(s\), let \(X_s=[D_s\ C_s]\) be its full design matrix, and let \(c\) select a treatment contrast from the coefficient vector. The contrast estimate can be written as \(w_{c,s}'Y_s\), where \(w_{c,s}=X_s(X_s'X_s)^{-1}c\), assuming full rank. Embed the earlier-look weight vector in the later data vector by assigning zero weights to participants not yet observed. If the errors are independent with variance \(\sigma^2\), then for looks \(s\le t\),

Only the shared participants contribute to this inner product. Dividing by the two marginal standard errors gives the correlation of the estimates or, equivalently, of their standardized statistics when the same positive scaling is used. This weight formula remains valid when the covariate adjustment differs across looks, provided each estimator is the stated linear-model estimator and the error assumptions hold. It is safer than assuming that adjustment leaves the familiar unadjusted information-fraction correlation unchanged.

Using the Joint Null Distribution for Calibration

Collect the standardized treatment-control statistics across arms and looks into a vector \(Z\). Under the stated Gaussian model and a specified intersection null, \(Z\) has a multivariate normal distribution with mean zero and correlation matrix \(R\). The off-diagonal entries of \(R\) are calculated from the covariances just derived and the corresponding standard errors. A one-sided MaxT boundary with common critical value \(b\) is calibrated by choosing \(b\) so that the probability of at least one null statistic exceeding it equals the chosen familywise level \(\alpha\):

Here \(i\) indexes treatment arms and \(k\) indexes looks; \(P_0\) denotes probability under the specified joint null. With stage-specific boundaries \(b_k\), replace the event by \(\{\exists i,k:Z_{i,k}\ge b_k\}\). The probability is evaluated using the full joint correlation matrix, including both shared-control and repeated-look dependence. Calibration is commonly performed by multivariate normal integration or simulation followed by a one-dimensional root search for a common boundary.

Derivation and Calibration Checks

The covariance formula follows from ordinary least squares after projecting out the nuisance covariates. Write \(Y=D\beta+C\gamma+\varepsilon\). Multiplication by \(M_C\) removes the component in the column space of \(C\), because \(M_CC=0\). Thus \(M_CY=M_CD\beta+M_C\varepsilon\). The least-squares estimator based on this residualized regression is \(\hat{\beta}=(D'M_CD)^{-1}D'M_CY\). Substituting the model and using \(M_C^2=M_C=M_C'\) yields

Applying the contrast vectors on the left and right gives the covariance of any two adjusted contrasts. For the cross-look result, express each estimator as a weighted sum of outcomes. Independence gives zero covariance for distinct participants, while each shared participant contributes \(\sigma^2\) times the product of that participant’s two weights. Summing those contributions gives \(\sigma^2w_{c,s}'w_{d,t}\).

Before using the resulting matrix for boundary calibration, check that every model matrix has full rank, the contrast standard errors are positive, and the covariance matrix is symmetric and positive semidefinite (up to numerical tolerance). Also check that its diagonal entries agree with the separately calculated marginal variances and that correlations lie between \(-1\) and \(1\). A singular matrix can occur when statistics are linearly redundant; use a method that handles the resulting lower-dimensional distribution rather than pretending the matrix is nonsingular.

Interpretation, Alternatives, and Limitations

Covariate adjustment can improve precision when baseline covariates explain outcome variation, but the covariance structure depends on the design and the fitted model. A common shortcut is to compute each adjusted standard error and then treat the statistics as independent. That is wrong when the control group is shared: it omits the covariance and generally misstates the probability of crossing at least one boundary. The correction is to obtain the off-diagonal entries as well as the marginal variances, then calibrate the joint event.

Bonferroni is a useful alternative when a simple, dependence-robust upper bound is preferred: allocate error across tests and use marginal critical values. It can be conservative because it does not exploit the joint covariance. MaxT calibration uses a specified joint null distribution and can be less conservative, but it is only as reliable as that distribution and the model assumptions used to obtain it. For strong familywise error control, calibration under the global null alone is not sufficient in general; the relevant intersection nulls or a valid subset-pivotality argument must also support the procedure, as in the earlier material on closed testing and subset pivotality.

The exact normal calibration here relies on Gaussian errors with known variance. With an estimated variance, small samples, heteroscedasticity, covariate-model misspecification, or data-dependent changes to the adjustment model, the stated joint normal distribution may not be adequate. Check the model and design, and use a calibration that matches the actual statistic and its assumptions. These concerns are distinct from simply calculating a covariance matrix correctly.