Why Adjust Interim Scores?
At an interim analysis, arms may have different observed distributions of baseline prognostic factors simply by chance. If a factor predicts outcome, an unadjusted treatment-control difference can partly reflect this imbalance rather than a treatment contrast at a common covariate profile. A covariate-adjusted score attempts to remove that component before arms are ranked. The adjustment can change the ranking, but it does not automatically make the new ranking more reliable: the model and its assumptions matter.
Consider a randomized multi-arm trial with one shared control. Let \(Y\) be an outcome for which larger values are better, and let \(X\) be a baseline covariate measured before treatment. The discussion below concerns treatment-control point estimates used as selection inputs, not a general selection rule. Scores should be expressed on the same outcome scale when arms are compared.
Model and Conditions for a Common-Slope Adjustment
For a single baseline covariate, a useful model at the interim analysis is
where \(i\) indexes participants, \(j=0\) denotes control, \(K\) is the number of experimental arms, \(c\) is a chosen reference value, and \(\tau_0=0\). The treatment-control effect for arm \(j\) is \(\tau_j\). The coefficient \(\beta\) describes the common linear association between the baseline covariate and outcome across all arms.
The adjustment relies on important conditions. The covariate must be measured before randomization or treatment, the linear relationship should be adequate over the observed range, and the covariate-outcome slope should be common across arms. The model also assumes independent errors with mean zero and a common variance for the usual regression-based inference. Randomization supports unbiased unadjusted comparisons in expectation, but it does not guarantee that the realized covariate means are identical at an interim look. The regression adjustment additionally needs enough covariate overlap across arms to estimate a meaningful contrast.
If the treatment effect is constant over \(X\), both adjusted and unadjusted differences target the same treatment effect in expectation. In a particular interim sample, however, the estimates can differ because arm covariate means differ and because the adjusted model estimates its slope. With several baseline covariates, replace \(\beta(X-c)\) by \(\boldsymbol{\beta}^{T}(\mathbf{X}-\mathbf{c})\); each covariate must be included according to a prespecified model rather than chosen after inspecting which specification produces a preferred ranking.
How the Adjusted Contrast Is Obtained
Fit a regression with an indicator for each arm and a common covariate slope. For a single covariate, the fitted outcome at a common reference value can be expressed through arm-specific intercepts. The estimated intercept for group \(g\) is \(\bar{Y}_g-\hat{\beta}(\bar{X}_g-c)\). Subtracting the control intercept from the treatment-arm intercept eliminates \(c\), giving:
The first term is the unadjusted score. The second term estimates the outcome difference attributable to the observed baseline-covariate imbalance under the common-slope model. The formula extends to several covariates by replacing the product with an inner product.
Worked Calculations
Worked Example: Adjustment Reverses the Interim Ranking
Hypothetical data. Suppose higher outcome values are better, and a baseline score \(X\) is prognostic: higher values tend to predict higher outcomes. At an interim look, the group means are as follows. The pooled within-arm regression sums are \(S_{XY}=30\) outcome-score units and \(S_{XX}=15\) score-squared units.
| Group | Mean outcome \(\bar{Y}\) | Mean baseline score \(\bar{X}\) |
|---|---|---|
| Control | 10 | 4 |
| Arm A | 14 | 5 |
| Arm B | 12 | 3 |
The common-slope estimate is \(\hat{\beta}=S_{XY}/S_{XX}=30/15=2\) outcome units per baseline-score unit. The unadjusted scores are \(14-10=4\) for A and \(12-10=2\) for B, so A ranks higher without adjustment.
For A, the covariate difference is \(5-4=1\). Its adjusted score is \(4-2(1)=2\) outcome units. For B, the covariate difference is \(3-4=-1\). Its adjusted score is \(2-2(-1)=4\) outcome units. Under this fitted model, the ranking reverses: B has the larger adjusted score. A's participants had a higher mean prognostic score than control, which contributes positively to A's observed outcome difference; adjustment subtracts that contribution.
This calculation does not establish that B is truly better. It shows that the selection input depends on the chosen score and its assumptions. Both scores use the same interim participants and should be understood as estimates, not as error-free arm effects.
Worked Example: Adjusting for Two Baseline Covariates
Hypothetical data. Suppose two prespecified baseline covariates are included in the common-slope model. The fitted slopes are \(\hat{\beta}_1=1.5\) outcome units per unit of \(X_1\) and \(\hat{\beta}_2=-0.8\) outcome units per unit of \(X_2\). The interim group means are:
| Group | \(\bar{Y}\) | \(\bar{X}_1\) | \(\bar{X}_2\) |
|---|---|---|---|
| Control | 50 | 4.0 | 6.0 |
| Arm A | 54 | 5.0 | 5.0 |
| Arm B | 55 | 3.5 | 6.5 |
The unadjusted scores are \(54-50=4\) for A and \(55-50=5\) for B, so B ranks higher. For A, the covariate-mean differences are \(1.0\) and \(-1.0\). The estimated prognostic contribution is \(1.5(1.0)+(-0.8)(-1.0)=2.3\). Thus A's adjusted score is \(4-2.3=1.7\) outcome units.
For B, the differences are \(3.5-4.0=-0.5\) and \(6.5-6.0=0.5\). The estimated prognostic contribution is \(1.5(-0.5)+(-0.8)(0.5)=-0.75-0.40=-1.15\). Therefore B's adjusted score is \(5-(-1.15)=6.15\) outcome units. The ranking remains B above A, but the separation between their scores is different: the unadjusted difference is \(1\) outcome unit, whereas the adjusted difference is \(6.15-1.7=4.45\) outcome units.
This example illustrates why adjustment should be applied consistently to every arm, using the same covariates and fitted model. Comparing an adjusted score for one arm with unadjusted scores for others would not be a coherent ranking.
When a Single Adjusted Score Is Not Enough
A common-slope model assumes that the treatment-control contrast does not vary with the covariate. If treatment effects differ across covariate values, an arm's effect depends on which covariate profile is the target. The following hypothetical model makes that issue explicit. Let the control mean be the reference and suppose the treatment contrasts are \(\Delta_A(x)=1+2x\) and \(\Delta_B(x)=2-x\), in outcome units. At \(x=0\), A's contrast is \(1\) and B's is \(2\), so B ranks higher. At \(x=1\), A's contrast is \(3\) and B's is \(1\), so A ranks higher. There is no single ranking independent of the target covariate distribution.
Worked Example: Interaction Changes Which Arm Ranks Highest
Suppose the target population has mean covariate value \(1\), and the treatment effects follow the covariate-dependent contrasts above. For a linear contrast, the average treatment effect over this target population is the contrast evaluated at its mean. Therefore A's target-population contrast is \(1+2(1)=3\) outcome units, while B's is \(2-1=1\) outcome unit. At the reference value \(x=0\), the corresponding contrasts are \(1\) and \(2\), respectively. The ranking changes because the treatment-by-covariate interaction makes the treatment effect depend on \(x\).
A common-slope adjustment cannot represent these effects. If such interaction is plausible and important, the model and the target population for averaging effects need to be specified explicitly; selecting using a single common-slope score would obscure the changing contrasts.
Diagnostics, Alternatives, and Limitations
Before using adjusted scores, examine whether the covariates are measured at baseline, whether there is reasonable overlap across arms, whether outcome-covariate patterns are approximately linear, and whether treatment-by-covariate interaction is substantial. At a small interim look these checks can be noisy; they are not a license to search many models and retain whichever one yields a desired arm ranking. A fitted slope based on little within-arm covariate variation may be unstable, so the adjustment itself can add noise.
| Selection input | What it uses | Main consideration |
|---|---|---|
| Unadjusted difference | Difference between observed arm and control means | Simple and transparent, but can reflect chance covariate imbalance in the interim sample |
| Common-slope adjusted difference | Unadjusted difference minus the modeled covariate contribution | Can account for prognostic baseline imbalance; depends on a suitable, stable common-slope model |
| Covariate-specific or standardized contrasts | Estimated effects at a stated covariate value or averaged over a stated target distribution | Needed when effects vary across covariate values; the target must be clear |
If adjustment is prespecified and the model is appropriate, it may improve precision, particularly when the covariate strongly predicts outcome. It is not guaranteed to do so in every interim sample, especially when many covariates are fitted relative to the available data. Unadjusted differences remain a useful reference and may be preferable when the adjustment model is poorly supported. Because selection uses interim data, an adjusted ranking is still a data-dependent decision; the score alone does not establish that the overall multi-stage design has acceptable operating characteristics.
Common Mistakes
- Assuming adjustment must preserve the ranking. It can reverse a ranking when arms have different covariate means and the covariate is prognostic. Calculate each arm's adjustment term rather than relying on the unadjusted ordering.
- Calling the adjusted score the observed mean difference. The adjusted score is a model-based contrast. State which covariates and model produced it.
- Adjusting for a post-randomization variable. Such a variable may be affected by treatment, so adjusting for it can change the comparison in problematic ways. Use baseline covariates for the adjustment described here.
- Ignoring effect modification. If treatment effects vary with a covariate, one common adjusted contrast may hide different effects across covariate profiles. Specify the target population and model the interaction when needed.
- Choosing covariates to favor an arm. Selecting a model after seeing which one produces a preferred ranking undermines the intended prespecification and can make rankings unstable.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- An arm's unadjusted score is \(3.0\) outcome units. Its covariate mean exceeds control by \(0.5\) units, and the fitted common slope is \(2.0\) outcome units per covariate unit. What is its adjusted score?
- Why can adjusted and unadjusted scores target the same treatment effect in expectation but differ in an interim sample?
- What additional issue must be addressed if the treatment effect varies with a baseline covariate?
- Why is it inappropriate to adjust for a variable measured after treatment starts using the baseline adjustment described here?
Solutions. (1) The adjusted score is \(3.0-2.0(0.5)=2.0\) outcome units. (2) Under a constant treatment effect and a valid common-slope model, both estimate the same contrast in expectation, but realized covariate means can differ across randomized arms, and the adjusted score also uses an estimated slope. (3) The effect then depends on covariate value, so specify a target covariate value or distribution and fit an interaction or otherwise estimate the corresponding standardized contrast. (4) A post-treatment variable may itself be affected by treatment; adjusting for it can alter the treatment comparison rather than merely account for baseline imbalance.