Baseline-Covariate-Adjusted Randomization Statistics

Why adjust a randomization statistic?

A baseline covariate can predict outcome variation even though it is measured before treatment. Adjusting for such a covariate can make a treatment contrast easier to interpret and can reduce the influence of chance baseline imbalance. In an adaptive trial, however, the adjustment does not replace the randomization distribution. The test statistic must be evaluated under the assignment paths the trial’s adaptive rule could actually have generated.

The central distinction is between adjusting the statistic and changing the assignment mechanism used for inference. Baseline information can enter the statistic, but the reference distribution still comes from the design. Reassigning treatment labels as if the trial had used fixed-allocation complete randomization can give the wrong reference distribution when assignment probabilities depended on the observed history.

Definition: Let $Z=(Z_1,\ldots,Z_n)$ denote the assignment vector, with $Z_i=1$ for treatment and $Z_i=0$ for control. Let $X_i$ be a vector of prespecified baseline covariates and $Y_i$ the observed outcome. For a fixed coefficient vector $\beta$, define the adjusted outcome $R_i=Y_i-\beta^\mathsf{T}X_i$. If assignment path $z$ has at least one unit in each arm, an adjusted difference in means is $$ T(z;Y,X)=\frac{\sum_{i=1}^n z_iR_i}{\sum_{i=1}^n z_i} -\frac{\sum_{i=1}^n(1-z_i)R_i}{\sum_{i=1}^n(1-z_i)}. $$

The components of $X_i$ must be measured before assignment, and the adjustment rule—including the covariates and how $\beta$ is obtained—should be specified without choosing it to make the observed result look more favorable. The coefficient vector may come from an external source or a prespecified fit that does not depend on the assignment labels. A common intercept is unnecessary in this contrast: subtracting the same constant from every $R_i$ changes neither arm mean difference nor the statistic.

The statistic is a function of the observed outcomes, baseline covariates, and a candidate assignment. To form its randomization distribution, keep the outcomes and baseline covariates fixed under the sharp null and calculate the statistic for every feasible assignment path. Do not keep the observed treatment labels inside the calculation when evaluating a different path.

Exactness under a sharp null

For a Fisher randomization test, the relevant null is the sharp null that every unit would have the same outcome under either assignment: $Y_i(1)=Y_i(0)$ for all $i$. Under this null, the observed outcome for each unit is known under every assignment path. This makes it possible to evaluate $T(z;Y,X)$ for each path without imputing unobserved outcomes. The path probabilities must be those of the actual adaptive assignment mechanism, including its dependence on the observed history.

Write $\pi(z)$ for the probability that the complete adaptive procedure generates path $z$, under the sharp null. These probabilities sum to one over the feasible paths. For a two-sided test based on the absolute value of the contrast, the randomization p-value is

p_{\mathrm{rand}}=\sum_{z\in\mathcal{Z}}\pi(z)\, \mathbf{1}\!\left\{\left|T(z;Y,X)\right|\geq \left|T(z_{\mathrm{obs}};Y,X)\right|\right\},

where $\mathcal{Z}$ is the set of feasible paths, $z_{\mathrm{obs}}$ is the observed path, and $\mathbf{1}\{\cdot\}$ equals one when its condition holds and zero otherwise. The weak inequality includes ties. For a one-sided alternative, the extremeness condition instead follows the prespecified direction.

The justification is the same randomization argument used for other Fisher tests: under the sharp null, $Y$ and $X$ are fixed, so the statistic’s only random input is the assignment path. The observed path is drawn from the distribution assigning probability $\pi(z)$ to path $z$. Therefore, the probability mass of paths at least as extreme as the observed path is a valid tail probability under that design. Discreteness can make the test conservative at some significance levels, but does not make the p-value a probability that the null is true.

Worked Example: Computing the Observed Adjusted Contrast

Suppose six hypothetical participants have one baseline covariate $X$, measured in units for which a one-unit increase predicts a two-point increase in outcome. Before looking at this trial’s treatment contrast, investigators specify $\beta=2$ outcome points per covariate unit. The observed treatment group has $(X,Y)=(1,8),(2,10),(3,12)$, and the control group has $(X,Y)=(0,5),(1,7),(2,8)$. Assume the outcomes are measured in points and the covariate is genuinely baseline.

The adjusted outcomes are $R_i=Y_i-2X_i$. For the treatment group they are $8-2(1)=6$, $10-2(2)=6$, and $12-2(3)=6$. Their mean is $18/3=6$ points. For control they are $5-2(0)=5$, $7-2(1)=5$, and $8-2(2)=4$, with mean $14/3=4.667$ points.

Thus the observed adjusted contrast is $$ T(z_{\mathrm{obs}};Y,X)=6-\frac{14}{3} =\frac{4}{3}=1.333\text{ points}. $$

The unadjusted difference is $10-20/3=3.333$ points. The groups differ in their average covariate values, so this prespecified adjustment removes the part of that contrast attributed to the specified covariate slope. This calculation alone is not a test; the test also requires evaluating the same statistic over the adaptive design’s paths.

Recalculate along the adaptive paths

For each feasible path, treatment labels change but the observed outcomes and baseline covariates remain attached to their participants. With a fixed $\beta$, calculate the $R_i$ once and then recompute the treatment and control means for each path. Weight each result by that complete path’s probability, not by an invented fixed-allocation distribution.

A coefficient estimated using treatment labels is different. For example, an ANCOVA fit containing a treatment indicator generally produces a coefficient that depends on the candidate assignment. Such an approach can still define a randomization statistic, but its fitting and extraction steps must be specified in advance and repeated for every path. Computing that fit only for the observed assignment and carrying its result over to other paths does not evaluate the stated statistic.

Specify the adjustment.
Choose baseline covariates and a coefficient or fitting algorithm before assessing the test result. State which outcomes and assignment labels the algorithm uses.

Define the statistic on the path space.
Check that it can be calculated for every feasible assignment path. If it requires both arms, the design’s support must ensure both arms occur, or a different statistic is needed.

Evaluate the design distribution.
For each path, apply the same statistic rule and obtain its probability from the adaptive procedure. Exact enumeration or a correctly implemented Monte Carlo randomization procedure can be used.

Compare with the observed path.
Sum the probabilities of paths at least as extreme as the observed path, using the direction and extremeness rule specified for the test.

Worked Example: A Weighted Adaptive Randomization Test

Consider four hypothetical participants with baseline covariates $X=(0,1,2,3)$ and observed outcomes $Y=(10,12,11,15)$, in outcome points. The prespecified slope is $\beta=1.5$ points per covariate unit. A hypothetical sequential adaptive rule assigns exactly one participant to treatment. After incorporating its response history, the rule has three feasible complete paths: participant 1 treated with probability $0.50$, participant 2 treated with probability $0.30$, or participant 3 treated with probability $0.20$. For instance, these probabilities could arise from a first branch of probability $0.50$, followed on the other branch by probabilities $0.60$ and $0.40$, giving $0.50$, $0.50(0.60)=0.30$, and $0.50(0.40)=0.20$. Suppose the observed path treats participant 2. All paths have both arms, and the outcomes and covariates are available for every participant.

The adjusted outcomes are $10-1.5(0)=10$, $12-1.5(1)=10.5$, $11-1.5(2)=8$, and $15-1.5(3)=10.5$. Their total is $39$. When exactly one participant is treated, the adjusted contrast for treating participant $i$ is that participant’s adjusted outcome minus the mean for the other three.

For participant 1, the contrast is $10-(10.5+8+10.5)/3=10-29/3=0.3333$ points. For participant 2, it is $10.5-(10+8+10.5)/3=10.5-28.5/3=1.000$ point. For participant 3, it is $8-(10+10.5+10.5)/3=8-31/3=-2.333$ points.

The observed absolute statistic is $1.000$ point. Paths 2 and 3 are at least as extreme, so the two-sided p-value is $$ p_{\mathrm{rand}}=0.30+0.20=0.50. $$

This is the probability, under the sharp null and this specified adaptive mechanism, of obtaining an adjusted contrast at least as extreme in absolute value as the observed one. It is not a 50% probability that the sharp null is true. The calculation uses the path probabilities; treating the three paths as equally likely would instead give $2/3$, which is not the test defined by this design.

Choices, diagnostics, and limitations

A simple adjusted difference in means is transparent and can be used with a fixed, externally justified coefficient. An unadjusted difference is simpler but may be less informative when baseline covariates predict outcomes. A regression-based statistic can incorporate several covariates or estimate a treatment coefficient, but then its full fitting procedure must be applied to every assignment path. These choices concern the statistic; none licenses replacing the adaptive path probabilities with those of a different design.

Before trusting a calculation, check that each covariate was measured before treatment, the adjustment was specified rather than selected after inspecting many candidate results, and the statistic is defined across the design’s entire support. Verify that path probabilities include all relevant sequential assignment probabilities and sum to one. For Monte Carlo evaluation, also check that paths are sampled from the actual adaptive mechanism and that the statistic is recomputed for every sampled path.

A covariate-adjusted Fisher test is exact for the sharp null when the randomization mechanism is correctly represented and the statistic is applied consistently. That exactness does not automatically give an exact test of a weak null such as a zero average treatment effect when individual effects may differ. Adjusting for baseline covariates does not change that distinction. The statistic also cannot repair missing or incorrectly recorded assignment histories, an unmodeled change to the allocation rule, or outcomes affected by treatment in ways that violate the assumptions underlying the sharp-null analysis.

Worked Example: Combining Two Baseline Covariates

Four hypothetical participants have two centered baseline covariates, $X_1$ in decade units and $X_2$ in ten-point units, and outcomes in points: $(X_1,X_2,Y)=(0,0,8),(1,0,11),(0,2,12),(1,2,14)$. The prespecified coefficients are $\beta_1=0.5$ points per $X_1$ unit and $\beta_2=0.2$ points per $X_2$ unit. Suppose participant 4 is treated and the other three are controls.

Calculate $R_i=Y_i-0.5X_{i1}-0.2X_{i2}$. The residualized values are $8$, $11-0.5=10.5$, $12-0.4=11.6$, and $14-0.5-0.4=13.1$ points. The treated mean is $13.1$; the control mean is $(8+10.5+11.6)/3=30.1/3=10.033$ points. Therefore, $$ T=13.1-10.033=3.067\text{ points}. $$

The unadjusted contrast is $14-(8+11+12)/3=14-10.333=3.667$ points. The $0.600$-point difference between the two contrasts equals $0.5(1-1/3)+0.2(2-2/3)=0.333+0.267=0.600$ points, the specified covariate contribution to the difference in group means. This arithmetic illustrates the adjustment; it does not by itself establish a causal effect or a p-value.

Common mistake

Wrong reasoning: “The covariate-adjusted statistic for the observed assignment is $1.0$, so I can compare it with all fixed-allocation permutations that preserve the observed treatment count.” That comparison uses a reference distribution that may not match an adaptive trial’s assignment probabilities.

Correction: Keep the observed outcomes and baseline covariates fixed under the sharp null, but apply the adjustment and statistic to each path generated by the adaptive rule. Weight the results by the rule’s path probabilities. Also ensure that any assignment-dependent fitting is repeated on each path; otherwise the statistic being tested is not the one actually calculated.

Key takeaway: Baseline adjustment changes the statistic, not the design. For sharp-null randomization inference, define the adjustment clearly, recompute the statistic on every feasible adaptive assignment path, and use the probabilities of those paths.

Check Your Understanding

Answer each question, then compare with the solutions below.

Why can baseline covariates be held fixed while computing a Fisher randomization distribution under a sharp null?
A statistic uses a regression coefficient fitted with the treatment indicator in the model. What must be done when evaluating a different assignment path?
In the weighted example, why is the two-sided p-value $0.50$, rather than $2/3$?
Does an exact covariate-adjusted Fisher test of a sharp null automatically test a zero weak average effect exactly? Explain.
What should be done if a proposed difference-in-means statistic is undefined on a feasible assignment path?

Solutions. (1) Under the sharp null, every unit’s outcome is unchanged by assignment, while a baseline covariate is measured before assignment. Both are therefore fixed across the candidate paths. (2) Refit the model and extract the statistic using that candidate path’s treatment labels, following the same prespecified algorithm used for the observed path. (3) The probabilities of the two extreme paths are $0.30$ and $0.20$; the design weights paths unequally, so their total probability is $0.50$, not two out of three equally weighted paths. (4) No. Exactness follows for the sharp null, which fixes every unit’s outcome under either assignment; a weak average-effect null permits individual effects to differ and needs separate justification. (5) Define a statistic that is valid on the full path space, or use a design whose support excludes the problematic paths. Do not silently discard feasible paths from the reference distribution.