Model-Based and Randomization-Based Dependence

Why the Dependence Depends on the Sampling Framework

Two treatment-versus-control contrasts share the control mean, so they are generally not independent. But the shared control is not the whole story: dependence also reflects how participants entered the groups and what is treated as random. Under an independent-sampling model, outcomes in separate groups are independent and the common control mean creates positive covariance. Under complete randomization from a fixed finite population, group membership is mutually exclusive; this introduces additional dependence, summarized by the finite-population covariance of the treatment effects.

The distinction matters because a covariance is defined relative to a probability model. A model-based calculation treats outcomes as random under a specified sampling model. A randomization-based calculation treats the potential outcomes of the finite study population as fixed and the assignment as random. The formulas need not agree, even when they concern the same observed design.

Independent-Sampling Group-Mean Model

Suppose the observations in arm \(a\) are \(Y_{a1},\ldots,Y_{an_a}\). Assume observations are independent within and across arms, with \(E(Y_{ak})=\mu_a\) and \(\operatorname{Var}(Y_{ak})=\sigma_a^2\). In particular, the control observations used in both contrasts are the same observations, while the treatment-arm observations are independent of one another and of the control observations. Finite variances are required; normality is not needed for the covariance result.

For distinct treatment arms \(i\) and \(j\), expand the covariance of the two contrasts. Covariances between different arms vanish by the independence assumption, leaving the variance of the shared control mean:

This model therefore gives a nonnegative covariance, strictly positive when the control variance is positive. The corresponding contrast variances are \(\sigma_i^2/n_i+\sigma_0^2/n_0\) and \(\sigma_j^2/n_j+\sigma_0^2/n_0\). Thus the correlation is the shared-control covariance divided by the product of the two contrast standard deviations. It depends on the variances and sample sizes, not on the treatment-effect means.

Complete Randomization in a Finite Population

Now suppose there are \(N\) fixed study units, each with a fixed potential outcome \(Y_k(a)\) under every arm \(a\). Complete randomization assigns exactly \(n_a\) units to each arm, with every allocation having the prescribed group sizes equally likely. Assume one arm per unit, no interference between units, and consistency: the observed outcome for a unit is its potential outcome under its assigned arm. In this framework, the assignment is random; the potential outcomes are not.

Write \(\bar{Y}(a)=N^{-1}\sum_{k=1}^N Y_k(a)\) for the finite-population mean under arm \(a\), and define the finite-population covariance between two sets of potential outcomes by \(S_{ab}=(N-1)^{-1}\sum_{k=1}^N [Y_k(a)-\bar{Y}(a)][Y_k(b)-\bar{Y}(b)]\). The finite-population treatment effect for unit \(k\) in arm \(i\) is \(\tau_i(k)=Y_k(i)-Y_k(0)\); its mean is \(\bar{\tau}_i=\bar{Y}(i)-\bar{Y}(0)\). Define \(S_{\tau_i,\tau_j}\) by the same covariance formula, using the two lists of individual treatment effects.

The difference-in-means estimator \(D_i\) is unbiased over assignments for \(\bar{\tau}_i\). However, the treatment and control groups are not independently sampled from the fixed population: assignments without replacement make group means dependent. For distinct treatment arms \(i\ne j\), the resulting randomization covariance is

Here \(\operatorname{Cov}_{R}\) denotes covariance over the random assignments. The first term resembles the shared-control contribution in the independent-sampling model. The second term adjusts it for the finite-population relationship between the two treatment effects. Consequently, randomization-based covariance can be positive, zero, or negative.

Deriving the Randomization Covariance

A short derivation makes the source of the correction explicit. Let \(\bar{Y}_a^{\,obs}\) be the observed mean in arm \(a\). Under complete randomization, the mean of a fixed potential-outcome list in arm \(a\) has variance \((1/n_a-1/N)S_{aa}\). For two distinct arms \(a\) and \(b\), their observed means have covariance \(-S_{ab}/N\). To see the latter identity, center each potential-outcome list by its finite-population mean and let \(I_{ka}\) indicate whether unit \(k\) is assigned to arm \(a\). For \(k\ne l\), \(P(I_{ka}=1,I_{lb}=1)=n_an_b/[N(N-1)]\); for \(k=l\), this joint probability is zero. Since each centered list sums to zero, the sum of cross-products over \(k\ne l\) is the negative of the sum over \(k=l\), yielding \(-S_{ab}/N\) for the covariance of the two arm means.

Now expand the covariance of the two contrasts. The distinct-arm mean covariance identities give

Expanding the product of the centered individual effects shows \(S_{\tau_i,\tau_j}=S_{ij}-S_{i0}-S_{0j}+S_{00}\). Substitution gives the stated result. This derivation also shows why the correction concerns the association of individual treatment effects, not merely the observed association between group means.

For reference, the randomization variance of one contrast is \(\operatorname{Var}_{R}(D_i)=S_{ii}/n_i+S_{00}/n_0-S_{\tau_i,\tau_i}/N\). This is a separate same-contrast formula; the cross-covariance formula for distinct arms should not be applied with \(i=j\) in its place.

Choosing the Framework and Checking Assumptions

The two approaches answer different probability questions. The model-based formula applies when the arm observations are modeled as independent samples, with the control observations reused in both contrasts. The randomization formula applies when the study population is fixed and the assignment is complete randomization with fixed group sizes. A randomized experiment does not, by itself, license either formula without specifying the target and assumptions: an analyst may use a model for outcomes, condition on a finite population and random assignment, or combine sources of randomness in a broader model.

Useful checks are therefore design-specific. Confirm whether units were independently sampled into groups or assigned from a fixed population, whether group sizes were fixed, and whether the same control observations enter both contrasts. For the randomization calculation, verify that the treatment-effect covariance uses centered lists and divisor \(N-1\). If assignment was blocked, clustered, adaptive, or unequal-probability rather than complete, the formula derived here generally does not apply unchanged.

A common mistake is to say, “The contrasts share a control, so their covariance must be positive and equal to the control-mean variance.” That conclusion is correct only under the independent-sampling assumptions. Under complete randomization, omitting \(-S_{\tau_i,\tau_j}/N\) can give the wrong magnitude or even the wrong sign. The correction may be hard to estimate from observed data because each unit reveals only its outcome under its assigned arm; the formula describes the randomization covariance, not necessarily a directly observable quantity.