Shared-Control Covariance Across Repeated Looks

Why Shared-Control Covariance Matters

In a multi-arm trial, each treatment may be compared with the same control group. That creates dependence between treatment-control statistics: a control mean that happens to be low makes several treatment contrasts look larger at once. Repeated looks add another source of dependence, because observations available at an earlier look may also contribute to a later look. To describe the joint behaviour of the statistics, both kinds of overlap must be represented.

The focus here is the covariance induced by that shared sampling. It is a property of the data and contrast construction, not a consequence of choosing a particular testing boundary. The key calculation is to find the covariance between the two contrast numerators and then divide by their standard errors.

Model and Covariance Formula

Suppose there are treatment arms indexed by \(i\) and control indexed by \(0\). At look \(k\), let \(n_{ik}\) and \(n_{0k}\) be the cumulative numbers of participants observed in treatment arm \(i\) and control. Assume these are fixed by the design, and that participants observed at an earlier look remain in the cumulative data at later looks. Write \(\bar{x}_{ik}\) and \(\bar{x}_{0k}\) for the corresponding sample means.

For the exact calculation below, assume individual outcomes are independent, normally distributed, and have the same known variance \(\sigma^2\) in every arm. Treatment groups are independent of each other and of the control group. Within an arm, the observations at one look are a subset of those at a later look. Under these assumptions, a standardized treatment-control statistic is

Its expectation may be nonzero when the treatment effect is nonzero, but its variance is one. Covariance between two such statistics is the covariance of their centered versions, so the same formula applies under the null and under fixed treatment effects.

Here \(\mathbf{1}\{i=j\}\) is one when the two statistics use the same treatment arm and zero otherwise. The first numerator term is treatment-sample overlap; it appears only for the same arm. The second is control-sample overlap; it appears for every pair of arms because both contrasts subtract control means. The variance factor \(\sigma^2\) cancels between the covariance of the contrast numerators and the standard errors.

Derivation From Sample-Mean Overlap

Let \(m_{ik}\) denote the centered sample mean in treatment arm \(i\) at look \(k\), and let \(m_{0k}\) denote the centered control sample mean. For nested samples of sizes \(n_{ik}\) and \(n_{il}\), the two treatment means share \(\min(n_{ik},n_{il})\) observations. Each shared observation contributes \(\sigma^2/(n_{ik}n_{il})\) to their covariance. Thus

The same reasoning gives \(\operatorname{Cov}(m_{0k},m_{0l})=\sigma^2\min(n_{0k},n_{0l})/(n_{0k}n_{0l})\) for the control means. Expand the covariance between the two contrast errors:

Independence makes the two cross-arm, treatment-control terms zero. The treatment-mean covariance is zero for different treatment arms and is the nested-sample expression above for the same arm. The control term remains in either case. Finally, divide by the product of the two standard errors, \(\sigma\sqrt{1/n_{ik}+1/n_{0k}}\) and \(\sigma\sqrt{1/n_{jl}+1/n_{0l}}\). This yields the stated formula.

The positive sign on the shared-control contribution is important. Both contrasts subtract the control mean; therefore a downward fluctuation in that mean shifts both contrasts upward, rather than shifting them in opposite directions.

Reading a Multi-Arm, Multi-Look Correlation Matrix

A joint covariance matrix is obtained by applying the formula to every pair of statistics. Because each \(Z_{ik}\) has variance one, its covariance matrix is also its correlation matrix. For example, suppose two arms and the common control each have 40 participants at look 1 and 80 at look 2. Order the statistics as \(Z_{A1}, Z_{A2}, Z_{B1}, Z_{B2}\). The pairwise correlations are: 0.7071 for the two looks within either arm; 0.5000 for different arms at the same look; and 0.3536 for different arms at different looks. Therefore the matrix is

The entries reflect two distinct overlaps: same-arm statistics share treatment data, and all statistics share control data. Under the exact normal model, these statistics have a joint multivariate normal distribution with this correlation matrix and with means determined by their treatment effects. The matrix, not a collection of separate marginal distributions, describes their joint variation.

Assumptions, Diagnostics, and Alternatives

The formula depends on the actual overlap structure. Before using it, check that sample sizes are cumulative, that the stated participants really are shared across looks, and that treatment groups are independent. If control observations at two looks overlap in \(q\) participants rather than being nested, replace \(\min(n_{0k},n_{0l})\) by \(q\) in the control-mean covariance term. If treatment observations overlap only partially, use their actual overlap count in the corresponding term as well. If the overlap is unknown, the covariance is not determined by the sample sizes alone.

Normal outcomes with known common variance give an exact joint-normal result. With non-normal outcomes, the same covariance calculation can be an approximation when sample sizes are sufficiently large and the mean estimates are well behaved. If variances differ across arms, the shared control term is governed by the control variance and treatment terms by their respective treatment variances; the common-\(\sigma^2\) expression no longer applies as written. Using a variance estimate shared across statistics can also make the joint distribution depart from the exact known-variance normal model.

A useful diagnostic is to inspect the design’s participant-level allocation and look schedule, rather than infer overlap from labels such as “stage 1” and “stage 2.” Also check that every calculated correlation lies between \(-1\) and \(1\), that the matrix is symmetric, and that its diagonal entries are one. A matrix that fails these checks usually signals an overlap or standard-error calculation error.

The main alternatives answer different needs. If each treatment arm has a genuinely separate control group, cross-arm control covariance is absent, although repeated looks within an arm can remain correlated. A method that does not use the joint covariance, such as a conservative allocation of error across separate tests, can avoid estimating this dependence but may not use the shared-control structure efficiently. A joint multivariate approach requires the covariance matrix and can represent the dependence directly. These are not interchangeable assumptions: treating shared-control statistics as independent changes the joint probabilities used in multi-arm inference.

Common Pitfall

The covariance calculation matters because simultaneous decisions across arms and looks depend on how often statistics cross their thresholds together. Marginal standard errors alone cannot provide that joint information. At the same time, the formula has limits: it describes the specified sampling and overlap model, not arbitrary adaptive changes, incomplete observations, or a design whose variance structure is different.