Covariate Adjustment in Randomized Experiments

Why Adjust for Covariates in a Randomized Experiment?

Random assignment makes treatment and control groups comparable in expectation, but a particular assignment can still leave them different in their baseline characteristics. If pretreatment characteristics also predict the outcome, adjusting for them can improve precision. The goal is not to repair confounding, as in an observational comparison: the assignment already identifies a causal effect. The goal is to use relevant baseline information without changing the effect being estimated or undermining the experiment’s design.

Adjustment must respect the timing and structure of the experiment. A baseline measure is recorded before assignment and cannot be caused by treatment. A post-treatment measure may itself be affected by treatment. Controlling for the latter can remove part of the effect of interest or introduce bias.

Potential Outcomes and the Target Effect

Consider a completely randomized experiment with \(N\) units, of which \(n_1\) are assigned treatment and \(n_0=N-n_1\) are assigned control. Let \(Z_i=1\) indicate that unit \(i\) is assigned treatment and \(Z_i=0\) indicate control. Each unit has two fixed potential outcomes, \(Y_i(1)\) and \(Y_i(0)\), and the observed outcome is \(Y_i=Z_iY_i(1)+(1-Z_i)Y_i(0)\). Assume consistency, no interference between units, and assignment by the stated randomization procedure.

Write \(\bar{Y}_1\) and \(\bar{Y}_0\) for the observed outcome means in the treatment and control groups, and \(\bar{X}_1\) and \(\bar{X}_0\) for their covariate means. The unadjusted estimator is \(\hat{\tau}_{\mathrm{unadj}}=\bar{Y}_1-\bar{Y}_0\). It is unbiased over the random assignment distribution for the finite-population effect. Covariate adjustment changes how the comparison is calculated, not the assignment mechanism or the estimand it is intended to estimate.

Regression Adjustment and ANCOVA

A common ANCOVA regression includes a treatment indicator and pretreatment covariates with common slopes: \(Y_i=\alpha+\tau_{\mathrm{ANCOVA}}Z_i+(X_i-\bar{X})'\beta+\varepsilon_i\). Here \(\bar{X}\) is the covariate mean across all randomized units, and the prime denotes a transpose. The coefficient on \(Z_i\) is the adjusted treatment estimate. With one covariate and a fitted common slope \(\hat{\beta}\), the coefficient has the form \(\hat{\tau}_{\mathrm{ANCOVA}}=(\bar{Y}_1-\bar{Y}_0)-(\bar{X}_1-\bar{X}_0)'\hat{\beta}\). Thus, it subtracts the outcome difference predicted by the observed covariate imbalance.

A common-slope regression assumes the covariate-outcome relationship is the same in both arms. A fully interacted regression allows it to differ: \(Y_i=\alpha+\tau_{\mathrm{int}}Z_i+(X_i-\bar{X})'\beta+Z_i(X_i-\bar{X})'\gamma+\varepsilon_i\). Centering the covariates makes \(\tau_{\mathrm{int}}\) the fitted treatment-control difference at the overall covariate mean. Under random assignment, this regression adjustment can improve precision without requiring the common-slope model to be correct. For a modest number of prespecified baseline covariates, the interacted specification is a useful default when treatment-effect modification by those covariates is plausible.

Neither regression makes randomization unnecessary. The covariates should be chosen based on pretreatment information, preferably before outcomes are examined. Adding many covariates or flexible terms can consume degrees of freedom and make estimates unstable, especially in small experiments. Adjustment can improve precision, but it does not guarantee a smaller standard error in every realized sample.

Uncertainty That Respects the Design

For the unadjusted difference under complete random assignment, define \(S_1^2\) and \(S_0^2\) as the finite-population variances of the potential outcomes under treatment and control, and \(S_\tau^2\) as the finite-population variance of the unit effects \(Y_i(1)-Y_i(0)\); each variance uses denominator \(N-1\). The randomization variance is

The final term cannot generally be estimated because each unit reveals only one potential outcome. Omitting this nonnegative term gives the conservative Neyman variance estimator \(s_1^2/n_1+s_0^2/n_0\), where \(s_1^2\) and \(s_0^2\) are the observed sample variances in the two arms. Its square root is a design-based standard error. The formula relies on the assignment mechanism; if randomization was blocked, the analysis should preserve those blocks rather than treat the experiment as an unrestricted complete randomization.

For a covariate-adjusted regression, ordinary model-based standard errors can rely on a constant error variance or a correctly specified outcome model. A heteroskedasticity-robust standard error instead allows residual variance to differ across units. One common choice is HC2. For a regression design matrix \(W\), fitted residuals \(\hat{e}_i\), and diagonal leverage values \(h_{ii}\), its sandwich covariance estimate is

Here \(w_i\) is the row of \(W\) for unit \(i\), and the standard error for the treatment coefficient is the square root of its estimated variance. In a fully interacted regression, this robust standard error is design-compatible for large randomized samples under regularity conditions, including bounded leverage and a fixed, modest number of covariates. It is not an exact finite-sample guarantee. For small samples, high leverage, or unusual assignment schemes, use inference methods justified for that design and report the limitations. Under blocked randomization, include the blocks in the adjustment and use uncertainty calculations that reflect assignment within blocks.

Why Post-Treatment Adjustment Can Mislead

A post-treatment covariate is measured after assignment and may be affected by treatment. Controlling for it can change the question from the total treatment effect to a direct effect that excludes pathways through that variable. It can also create bias when the variable is a common effect of treatment and another cause of the outcome: conditioning on a common effect can induce an association between its causes. Randomization protects the treatment comparison before such conditioning; it does not automatically protect a comparison after selecting or adjusting on a treatment-affected variable.

Common Mistakes and Practical Guidance

Covariate adjustment is most useful when it is tied to the design, the target effect, and a defensible uncertainty calculation. The following distinctions help prevent common errors.

For a blocked or stratified experiment, carry the design into both estimation and uncertainty assessment. Block indicators or stratum-specific comparisons can preserve the original assignment structure. Covariate adjustment complements the design; it does not replace it.