Masked Covariance Estimation For Repeated-Measures Outcomes

Why covariance matters in a blinded re-estimation

When each participant is measured at multiple visits, the observations from the same participant are generally related. Planning a comparison based on a change score, an average across visits, or another linear contrast therefore requires more than the separate variance at each visit: it requires the covariances among visits. Those covariances determine the variability of the planned contrast, and hence the sample size needed to detect a specified effect.

In a blinded re-estimation, treatment assignments are not used to form the nuisance estimate. A simple pooled covariance matrix can be calculated from all participants’ repeated outcomes without treatment labels. This is useful, but pooling has a key limitation: if treatment groups have different mean outcome profiles, the pooled covariance includes a component from those differences. The resulting estimate is not automatically the within-treatment covariance needed for planning.

Pooled covariance estimation and its assumptions

Suppose there are \(N\) participants with complete measurements at the same \(m\) visits. Let \(\bar{Y}\) be the vector of pooled sample means, calculated separately at each visit. The usual pooled sample covariance matrix is

This calculation uses participant-level vectors, not separate visit-specific data sets. Each diagonal entry is a sample variance, and each off-diagonal entry is the corresponding sample covariance. The estimator targets a common within-group covariance matrix when participants are independent, the visit measurements are comparably defined, and the groups have the same covariance structure. For the straightforward blinded interpretation, it also requires that differences between group mean profiles are negligible or absent. Complete measurements at the included visits are assumed for the formula as written.

The last condition matters because pooled centering subtracts the overall visit means, not the separate means in each treatment group. To see the consequence, suppose there are two groups with fixed mean vectors \(\mu_0\) and \(\mu_1\), a common within-group covariance \(\Sigma\), and \(n_0\) and \(n_1\) participants. Write \(\delta=\mu_1-\mu_0\). The pooled sample covariance has an expected within-group contribution \(\Sigma\), plus a between-group contribution:

The multiplier is nonnegative, and \(\delta\delta^\mathsf{T}\) adds variation in directions where the group mean profiles differ. For large \(N\), the multiplier is approximately \(\pi_0\pi_1\), where \(\pi_g=n_g/N\). Thus, with balanced groups, the approximate added term is \(\delta\delta^\mathsf{T}/4\). Masking does not make this term disappear: it prevents the analyst from subtracting separate group means.

From the covariance matrix to the planned contrast

A contrast is a weighted sum of visit outcomes. Its variance follows directly from the covariance matrix. In a two-visit change score, for example, \(w=(-1,1)^\mathsf{T}\), so the participant’s score is follow-up minus baseline. Expanding the matrix product gives

The sign of the covariance term depends on the contrast weights. Positive covariance reduces the variance of a difference between visits, because participants with above-average baseline outcomes tend also to have above-average follow-up outcomes. For a weighted average with weights of the same sign, positive covariance instead contributes positively to the variance.

For a two-group comparison with independent participants, common contrast variance \(\sigma_C^2=w^\mathsf{T}\Sigma w\), and \(n_0,n_1\) participants per group, the variance of the difference in mean contrasts is \(\sigma_C^2(1/n_0+1/n_1)\). Under equal allocation with \(n\) participants per group, the usual normal-approximation planning formula for a two-sided test is

where \(\Delta_C\) is the target between-group difference in the planned contrast, \(\alpha\) is the Type I error rate, and \(1-\beta\) is the target power. This calculation assumes the specified contrast and effect are appropriate, the common variance is a reasonable planning value, observations are independent across participants, and the normal approximation is suitable. In practice, the calculated value is rounded up to a whole participant per group.

Why masking can change the contrast variance estimate

The pooled covariance estimate can be inserted into the contrast formula, but the group-mean contribution also projects onto that contrast. Applying \(w\) to the pooled covariance decomposition gives

This shows precisely which part of the group mean difference affects the planned contrast variance estimate: it is the squared difference between groups on that contrast. A pooled matrix may be contaminated in some directions but not others. If \(w^\mathsf{T}\delta=0\), this between-group component does not inflate the variance estimate for that particular contrast, even though other entries of the matrix may be affected. In general, however, the contrast-specific contamination cannot be assessed from pooled data alone because the group labels and the contrast difference are masked.

Derivation, diagnostics, and alternatives

The covariance decomposition follows by centering within each group. Let \(\bar{Y}_g\) be the sample mean in group \(g\); then $$ \sum_{i=1}^{N}(Y_i-\bar{Y})(Y_i-\bar{Y})^\mathsf{T} =\sum_{g=0}^{1}\sum_{i\in g}(Y_i-\bar{Y}_g)(Y_i-\bar{Y}_g)^\mathsf{T} +\sum_{g=0}^{1}n_g(\bar{Y}_g-\bar{Y})(\bar{Y}_g-\bar{Y})^\mathsf{T}. $$ The cross-products vanish exactly because the within-group deviations sum to zero in each group. For two independent groups with common covariance \(\Sigma\), the expected within-group scatter is \((N-2)\Sigma\), and the expected between-group scatter is \(\Sigma+(n_0n_1/N)\delta\delta^\mathsf{T}\); dividing their sum by \(N-1\) gives the stated expression.

Before using a pooled estimate, check the data and the design against its assumptions. Confirm that each row corresponds to one participant and columns correspond to the intended visits, that units and visit definitions are consistent, and that the matrix is symmetric. A covariance matrix should also be positive semidefinite; a computed matrix with a materially negative eigenvalue can indicate data-processing or numerical problems. Inspect visit-specific variances, correlations, scatterplots, and the number of participants contributing at each visit. Strongly changing variances, unusual patterns over time, or substantial incomplete measurements warn that a single common covariance estimate may not represent the planned analysis.

A central diagnostic limitation is that treatment masking hides the very group mean profiles needed to assess the between-group term. A large pooled covariance or a change in its entries can therefore reflect a true change in within-participant variability, a treatment mean-profile difference, or both. The pooled estimate alone cannot distinguish these explanations. Prespecifying which visits, contrast, covariance estimator, and assumptions will be used helps keep the nuisance update interpretable.

An unstructured covariance estimate allows each variance and covariance to differ, but requires enough participants to estimate its many entries reliably. Structured alternatives impose simpler patterns: compound symmetry uses a common variance and common within-person correlation, while an autoregressive structure makes correlations decline with visit separation. Such structures can stabilize estimation when justified by the measurement schedule and scientific setting, but a convenient structure should not be assumed solely because the unstructured estimate is noisy. Independence is another possible planning assumption; it sets off-diagonal covariances to zero and can badly misstate the contrast variance when repeated outcomes are correlated.

The method’s main limitation is identifiability: without treatment labels or additional modeling assumptions, pooled repeated outcomes generally do not identify the within-group covariance matrix separately from treatment mean differences. It is most straightforward when treatment mean differences are negligible for covariance-estimation purposes, or when the planning procedure explicitly accepts the pooled estimate as an approximation. An alternative is a model-based covariance estimate that uses treatment assignments, if unmasking is permitted and the design’s statistical safeguards allow it. Another alternative is to prespecify a defensible structured covariance model. These approaches trade masking, modeling assumptions, and precision differently; none makes the underlying assumptions irrelevant.