Chance Baseline Imbalances in Randomized Trials

Why Baseline Groups Can Differ After Randomization

Random assignment makes treatment allocation independent of participants’ baseline characteristics in the design. It does not promise that the groups in one completed trial will have identical ages, baseline scores, or other characteristics. Even when the allocation procedure is carried out correctly, chance can place more participants with a particular characteristic in one group than the other.

This distinction matters when reading or reporting a trial. A baseline difference is a description of the groups that were formed. By itself, it does not establish that the allocation procedure failed, nor does it show that treatment assignment caused the baseline difference. Baseline characteristics were measured before treatment, and their group differences arise from which participants happened to be assigned to each group.

Definition: A baseline imbalance is an observed difference between the treatment groups in a characteristic measured before treatment. Under random assignment, such a difference can occur by chance. Randomization balances a baseline characteristic in expectation: across repeated assignments made according to the same design, the expected group means are equal. It does not require equality in every realized assignment.

The phrase “in expectation” refers to the average over the assignments that the randomization procedure could have produced, not to an average over future participants. In a particular trial, the assigned groups are fixed, and their observed characteristics may differ. The size of a difference should therefore be interpreted in light of the allocation design and the amount of random variation it permits.

A Randomization Calculation for a Baseline Difference

Consider a trial using complete randomization: exactly \(n_T\) of \(N\) participants are assigned to treatment, every allocation of that size is equally likely, and the remaining \(n_C=N-n_T\) are assigned to control. Suppose a baseline variable \(x\) is measured for all participants before assignment. Define the observed difference as treatment minus control:

D=\bar{x}_T-\bar{x}_C.

For this calculation, the \(N\) observed baseline values are treated as fixed; the source of randomness is the assignment. Define their finite-population variance using denominator \(N-1\):

S_x^2=\frac{1}{N-1}\sum_{i=1}^{N}(x_i-\bar{x})^2.

Under complete randomization, the mean of \(D\) over all allowed assignments is zero, and its variance is:

\operatorname{Var}(D)=S_x^2\left(\frac{1}{n_T}+\frac{1}{n_C}\right).

The square root of this variance is the standard deviation of the baseline difference across the random assignments permitted by this design. It can be used to describe how large the observed difference is relative to randomization variability. It is not automatically a normally distributed test statistic, and dividing by it does not by itself produce a p-value.

Why the Randomization Variance Has This Form

A short derivation makes the conditioning clear. Under complete randomization, the treatment group is a simple random sample of \(n_T\) participants from the fixed set of \(N\). Its mean has expected value \(\bar{x}\) and variance \((1-n_T/N)S_x^2/n_T\). Because the control group contains everyone else, \(D\) can be written in terms of the treatment-group mean:

\bar{x}_C=\frac{N\bar{x}-n_T\bar{x}_T}{n_C}, \qquad D=\frac{N}{n_C}(\bar{x}_T-\bar{x}).

Thus the expected difference is zero. To find its variance, multiply the variance of \(\bar{x}_T\) by the square of \(N/n_C\):

\begin{aligned} \operatorname{Var}(D) &=\left(\frac{N}{n_C}\right)^2 \left(1-\frac{n_T}{N}\right)\frac{S_x^2}{n_T}\\ &=\left(\frac{N}{n_C}\right)^2 \frac{n_C}{N}\frac{S_x^2}{n_T}\\ &=\frac{N}{n_Tn_C}S_x^2 =S_x^2\left(\frac{1}{n_T}+\frac{1}{n_C}\right). \end{aligned}

The final equality uses \(N=n_T+n_C\). This result depends on complete randomization and fixed group sizes. If a trial used a different allocation design, its design must guide any calculation of randomization variability; the formula should not be applied without checking that the design matches its assumptions.

Worked Example: A Binary Baseline Characteristic

Suppose a hypothetical trial has 60 participants, of whom 30 have a particular baseline characteristic and 30 do not. Exactly 30 participants are assigned to treatment by complete randomization. In one assignment, 18 participants with the characteristic enter treatment and 12 enter control. Describe the imbalance and compare it with its randomization variability.

Let \(x=1\) for a participant with the characteristic and \(x=0\) otherwise. The treatment proportion is \(18/30=0.600\), and the control proportion is \(12/30=0.400\). Therefore, the observed difference in proportions is

$$ D=0.600-0.400=0.200. $$

The overall proportion is \(30/60=0.500\). The finite-population variance of these 60 fixed binary values is

S_x^2=\frac{60}{59}(0.5)(1-0.5)=\frac{15}{59}\approx0.2542.

Under the stated complete-randomization design, the standard deviation of the difference across assignments is

SE(D)=\sqrt{\left[\frac{60}{59}(0.5)(0.5)\right] \left(\frac{1}{30}+\frac{1}{30}\right)} =\sqrt{\frac{1}{59}} \approx0.1302.

The observed difference is \(0.200/0.130189\approx1.536\) randomization standard deviations from zero. This ratio describes the difference relative to the variation expected across assignments under the design; it is not, on its own, a p-value. The correct contextual report is that the treatment group had a 20-percentage-point higher observed prevalence of the characteristic. The groups are not perfectly balanced, but this observation alone does not show that randomization failed.

Describe the Difference Before Interpreting It

A useful report begins with the groups, the measured characteristic, and its units. For a continuous characteristic, give group means and their difference in the original units. For a binary characteristic, give group proportions and their difference in percentage points. This makes the imbalance understandable without implying that a particular threshold separates acceptable from unacceptable randomization.

A standardized mean difference can provide a scale-free description of a continuous baseline difference. It divides the difference in group means by the pooled within-group standard deviation. For groups with sample sizes \(n_T\) and \(n_C\), and sample standard deviations \(s_T\) and \(s_C\), the pooled standard deviation is:

s_{\mathrm{pooled}}= \sqrt{\frac{(n_T-1)s_T^2+(n_C-1)s_C^2} {n_T+n_C-2}}.

The standardized mean difference is \(D/s_{\mathrm{pooled}}\). It expresses the observed difference in units of pooled standard deviations, not in the original measurement units. It is a descriptive measure: no universal cutoff determines whether randomization succeeded, and standardization does not turn a baseline difference into evidence about the allocation process.

Worked Example: Reporting a Continuous Baseline Difference

In a hypothetical trial, the treatment group has 40 participants with a mean baseline age of 52 years and a sample standard deviation of 10 years. The control group also has 40 participants, with a mean age of 49 years and a sample standard deviation of 8 years. Calculate and interpret the raw and standardized differences.

With treatment minus control as the direction, the raw difference is \(52-49=3\) years. The pooled standard deviation is

s_{\mathrm{pooled}} =\sqrt{\frac{(40-1)(10^2)+(40-1)(8^2)}{40+40-2}} =\sqrt{\frac{39(100)+39(64)}{78}} =\sqrt{82} \approx9.055\text{ years}.

The standardized mean difference is

\frac{52-49}{\sqrt{82}}\approx0.331.

The treatment group’s observed mean age is 3 years higher, or about 0.331 pooled standard deviations higher. Both numbers describe this realized allocation. Neither number establishes that the assignment mechanism was defective; assessing the mechanism requires information about how assignment was generated and implemented.

Why Many Baseline Tests Can Mislead

A table of baseline characteristics may contain many variables. If each is tested separately at a chosen significance level, some results can appear statistically unusual simply because many comparisons were made. For example, if 20 valid tests each use a 5% significance level and all their null hypotheses are true, the expected number of p-values below 0.05 is \(20(0.05)=1\). This expectation does not require the tests to be independent. It does not say that exactly one such result will occur, or give the probability of at least one.

A small baseline-test p-value is calculated relative to a statistical null model or randomization distribution. It is not the probability that randomization failed, and a larger p-value is not proof that the groups are identical. Baseline tests should not be used as a pass-or-fail certificate for randomization. Descriptive differences, considered in context and alongside the design, are generally more informative for understanding the realized groups.

Worked Example: One Unusual Difference Among Many

Suppose a hypothetical trial reports 20 baseline characteristics, and one comparison has a p-value below 0.05. If, for illustration, all 20 comparisons correspond to true null hypotheses and each test has a 5% Type I error rate, the expected count below 0.05 is

E(\text{number below }0.05)=20(0.05)=1.

This calculation does not prove that the observed result is a chance fluctuation; it shows why a single threshold-crossing result among many comparisons cannot by itself diagnose failed randomization. The next step is to describe the size and units of the difference and review how allocation was carried out. If there were a documented deviation, that record—not the baseline p-value alone—would be evidence relevant to implementation.

What Can and Cannot Be Concluded

Randomization is a property of the assignment procedure, while baseline balance is a feature of the groups produced by one assignment. A marked difference can be a reason to check the allocation and data records carefully, but it is not a verdict on the procedure. Relevant checks include whether the intended random sequence was used, whether assignments were concealed as planned, whether participants were enrolled before their assignments were known, and whether assignment records were entered accurately.

The calculation of randomization variability also has a boundary: it describes assignments allowed by the assumed design. The complete-randomization formula above does not automatically apply if assignment was restricted or organized differently. The design details matter. Finally, baseline imbalance and later outcome analysis are related but distinct issues. A baseline difference is not evidence that a treatment changed a pre-treatment variable. Decisions about adjustment belong to the planned analysis and should not be made solely because one baseline comparison looks unusual.

Common Mistakes

Expecting exact equality. Random assignment balances characteristics in expectation, not necessarily in a single trial. Chance differences are compatible with proper randomization.
Treating a baseline p-value as a test of whether randomization occurred. It summarizes a comparison under a statistical reference distribution; it does not give the probability that the allocation procedure failed.
Calling a nonsignificant result proof of balance. Failure to cross a significance threshold does not establish that the groups are identical or that every important characteristic is well matched.
Ignoring the allocation design. A variability calculation is justified only when its assumptions match the actual assignment procedure.
Reporting only a label such as “imbalanced.” State the group values, the direction and size of the difference, and its units so readers can judge its practical scale.

Key takeaway: A baseline difference describes the groups formed in one randomized trial. It may arise by chance even when allocation was correctly implemented. Interpret its magnitude in context, use calculations that match the randomization design, and evaluate procedural concerns using allocation records rather than baseline significance tests alone.

Check Your Understanding

Try these questions without looking back.

What does it mean to say that randomization balances a baseline characteristic “in expectation,” and why does that not imply identical groups in every trial?
In a completely randomized trial with fixed group sizes, what does the standard deviation of the baseline mean difference describe?
A baseline comparison has a p-value below 0.05 among many characteristics. Why does this not, by itself, show that randomization failed?
Why must the allocation design be checked before using the complete-randomization variance formula?
For the age example, state the raw difference and standardized mean difference in context. What does each measure communicate?