Avoiding Double Counting in Integrated Evidence

Why Overlapping Cohorts Change the Evidence

Suppose two molecular studies report genetic association estimates for the same variant, one for gene expression and one for protein abundance. If the studies include some of the same participants, their estimates may be correlated: sampling variation in a shared participant can affect both estimates. Treating the two results as independent can therefore make the combined evidence appear more precise than it is.

The key is to distinguish two estimates from two independent pieces of information. Estimates from different molecular layers can still share participants, controls, genotype measurements, or correlated phenotypes. The appropriate joint analysis represents this dependence in a covariance matrix. It also makes clear when a single combined effect is a meaningful target and when the layer-specific effects should instead be tested jointly without averaging them.

Definition: Let $\hat{\boldsymbol{\beta}}=(\hat{\beta}_1,\ldots,\hat{\beta}_K)^\mathsf{T}$ be association estimates for $K$ molecular layers, and let $\boldsymbol{\beta}$ be the corresponding vector of true effects. Their covariance matrix is $\Sigma$, whose diagonal entries are the variances of the estimates and whose off-diagonal entries are their covariances. A covariance-aware analysis uses the joint sampling distribution of the estimates, rather than treating each estimate as independent.

Set Up the Joint Likelihood

For large-sample association estimates, a standard working model is $\hat{\boldsymbol{\beta}}\sim N(\boldsymbol{\beta},\Sigma)$. This approximation assumes that the estimates are approximately unbiased for the specified effects, that their sampling distribution is adequately approximated by a multivariate normal distribution, and that $\Sigma$ correctly describes their sampling variances and covariances. Each estimate must also refer to the same genetic variant and compatible allele coding and effect scale. If allele directions or units differ, align or transform them before combining results.

The covariance matrix is not merely a technical adjustment. Its entries reflect how the estimates vary together under repeated sampling. Overlap can contribute to covariance, as can correlated molecular phenotypes and shared analysis features. Overlap does not, by itself, determine the covariance: the association design and the relationship between the measured traits also matter.

If the scientific target is a single common effect $\theta$, so that $\boldsymbol{\beta}=\theta\mathbf{1}$, the likelihood can be used to estimate that effect efficiently. This common-effect assumption says the variant has the same effect on every layer on the analyzed scale. It is not implied simply because the layers have been measured in overlapping participants. When a common effect is not scientifically justified, retain the vector of effects and use a joint test instead.

Common-effect estimator: Under the multivariate normal model with known positive-definite covariance matrix $\Sigma$, and the restriction $\boldsymbol{\beta}=\theta\mathbf{1}$, the generalized least-squares estimate and its variance are $$ \hat{\theta}= \frac{\mathbf{1}^\mathsf{T}\Sigma^{-1}\hat{\boldsymbol{\beta}}} {\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}, \qquad \operatorname{Var}(\hat{\theta})= \frac{1}{\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}}. $$ The standard error is the square root of this variance.

This estimator weights evidence according to the full covariance structure. Highly correlated estimates generally contribute less additional information than equally precise estimates that are independent. The formula does not license combining effects with incompatible meanings; the common-effect restriction still needs a scientific rationale.

Worked Example: Combining Two Correlated Association Estimates

Suppose two hypothetical molecular layers give estimates of the same genetic effect, in compatible units: $\hat{\beta}_1=0.30$ and $\hat{\beta}_2=0.34$. Each has standard error $0.20$, and their estimated sampling correlation is $0.60$. Assume for this example that the common-effect model is appropriate and that the covariance estimate is reliable.

The variances are $0.20^2=0.0400$, and the covariance is $0.60(0.20)(0.20)=0.0240$. Thus

\Sigma= \begin{pmatrix} 0.0400 & 0.0240\\ 0.0240 & 0.0400 \end{pmatrix}.

With equal marginal variances and a common effect, the generalized least-squares estimate is the average of the two estimates. Its variance is the variance of their average:

\hat{\theta}=\frac{0.30+0.34}{2}=0.3200, \qquad \operatorname{Var}(\hat{\theta}) =\frac{0.0400+0.0240}{2}=0.0320, \qquad SE(\hat{\theta})=\sqrt{0.0320}=0.1789.

A normal-approximation 95% confidence interval is $0.3200\pm1.96(0.1789)$, or approximately $(-0.0307,\,0.6707)$ effect units. Under the stated model, this interval is produced by a procedure that would cover the common effect in 95% of repeated samples. It is not a statement that there is a 95% probability that the fixed effect lies in this particular interval.

Deriving the Common-Effect Estimate

The estimator follows directly from maximizing the joint normal likelihood. Apart from a constant, the log likelihood for $\theta$ is minus one half of the covariance-weighted squared discrepancy between the observed estimates and the common-effect model:

\ell(\theta)=-\frac{1}{2} (\hat{\boldsymbol{\beta}}-\theta\mathbf{1})^\mathsf{T} \Sigma^{-1} (\hat{\boldsymbol{\beta}}-\theta\mathbf{1}).

Differentiate with respect to $\theta$. Since $\Sigma$ is symmetric, the derivative of the quadratic expression gives

\frac{d\ell(\theta)}{d\theta} =\mathbf{1}^\mathsf{T}\Sigma^{-1} (\hat{\boldsymbol{\beta}}-\theta\mathbf{1}).

Setting this derivative equal to zero yields $\mathbf{1}^\mathsf{T}\Sigma^{-1}\hat{\boldsymbol{\beta}} =\theta\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}$, and hence the estimator in the definition above. The estimator is a linear combination of the estimates. Its variance is

\operatorname{Var}(\hat{\theta}) = \frac{\mathbf{1}^\mathsf{T}\Sigma^{-1} \operatorname{Var}(\hat{\boldsymbol{\beta}}) \Sigma^{-1}\mathbf{1}} {(\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1})^2} = \frac{1}{\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}},

because $\operatorname{Var}(\hat{\boldsymbol{\beta}})=\Sigma$, so the numerator reduces to $\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}$. This derivation explains why omitting off-diagonal covariances changes both the weights and the reported uncertainty.

Testing a Common Effect or Testing the Vector

A common-effect estimate answers a focused question only if the effects can reasonably be represented by one $\theta$. Sometimes the scientific question is instead whether there is evidence of association in any of several layers, while allowing the effects to differ. Under the multivariate normal model and a specified positive-definite $\Sigma$, the Wald statistic for $H_0:\boldsymbol{\beta}=\mathbf{0}$ is $Q=\hat{\boldsymbol{\beta}}^\mathsf{T}\Sigma^{-1}\hat{\boldsymbol{\beta}}$. If the null holds and the normal model and covariance are correct, $Q$ has a chi-square distribution with $K$ degrees of freedom, approximately when the normal model is an asymptotic approximation. This is a joint test; it does not assert that the effects are equal or identify a causal pathway.

Worked Example: Jointly Testing Two Layer-Specific Effects

In a hypothetical study, suppose the estimates for two molecular layers are $0.20$ and $0.30$, each with standard error $0.20$, and their estimated correlation is $0.50$. The target is a joint test of whether both layer-specific effects are zero; no common-effect assumption is made. Assume the estimates are approximately jointly normal and the covariance matrix is correctly specified.

The covariance is $0.50(0.20)(0.20)=0.0200$, giving $\Sigma=\begin{pmatrix}0.0400&0.0200\\0.0200&0.0400\end{pmatrix}$. For this matrix, its inverse is $\begin{pmatrix}33.333&-16.667\\-16.667&33.333\end{pmatrix}$, with entries rounded for display. The joint Wald statistic can be calculated as

Q= \frac{0.20^2+0.30^2-2(0.50)(0.20)(0.30)} {0.20^2(1-0.50^2)} = \frac{0.0700}{0.0300} =2.333.

Under $H_0$, the reference distribution is chi-square with $df=2$. For two degrees of freedom, the upper-tail probability is $\exp(-Q/2)=\exp(-1.1667)\approx0.3114$. Thus the joint test does not provide strong evidence against the null at conventional significance levels. The result does not show that both effects are exactly zero; it describes the evidence from this joint test under its assumptions.

Sensitivity Analysis When Overlap Is Uncertain

In some integrated analyses, the separate studies report their marginal standard errors but do not provide enough information to estimate cross-layer covariance precisely. Replacing an unknown covariance with zero is not a neutral choice: it asserts that the estimates are uncorrelated. A defensible alternative is to state a set of plausible correlations or overlap scenarios, calculate the combined uncertainty under each, and assess whether the substantive conclusion changes.

For two estimates with the same standard error $s$ and sampling correlation $\rho$, under a common-effect model the average has standard error $s\sqrt{(1+\rho)/2}$. This formula follows from $\operatorname{Var}((\hat{\beta}_1+\hat{\beta}_2)/2) =(s^2+s^2+2\rho s^2)/4$. The assumed correlation must be feasible: for a two-estimate covariance matrix, $-1\leq\rho\leq1$, and the matrix must be positive definite for inversion-based procedures. With more layers, the entire proposed correlation matrix must be positive definite, not just each pairwise correlation.

Worked Example: How an Unknown Correlation Changes the Conclusion

Suppose two hypothetical layers each estimate the same common effect as $0.32$, each with standard error $0.20$. Their cross-layer sampling correlation is uncertain. Consider $\rho=0$, $0.50$, and $0.80$ as sensitivity scenarios. The same observed estimates are used in each scenario; only the assumed covariance changes. For each scenario, test $H_0:\theta=0$ using a two-sided normal Wald test.

For $\rho=0$, $SE=0.20\sqrt{(1+0)/2}=0.1414$, so $z=0.32/0.1414=2.263$ and the two-sided p-value is approximately $0.02365$. For $\rho=0.50$, $SE=0.20\sqrt{1.50/2}=0.1732$, so $z=1.848$ and $p\approx0.0646$. For $\rho=0.80$, $SE=0.20\sqrt{1.80/2}=0.1897$, so $z=1.687$ and $p\approx0.0917$.

The p-value is below $0.05$ in the independence scenario but above $0.05$ in the two positive-correlation scenarios. This is evidence that the conclusion is sensitive to the unknown dependence, not evidence that one of the scenarios is true. The analysis should report the assumed range and seek better covariance information where possible.

Common Pitfalls and Practical Checks

A covariance-aware analysis is only as credible as its inputs and target. Before combining estimates, check whether they describe the same variant, effect allele, scale, and estimand. Confirm whether the intended result is a common effect or a joint test of distinct layer-specific effects. Then identify how the covariance was obtained or, if it is uncertain, which sensitivity scenarios are scientifically plausible.

Wrong reasoning: “The estimates come from different omics layers, so they are independent.” Correction: Different outcomes do not guarantee independent sampling errors. Shared participants and correlated measurements can induce covariance.
Wrong reasoning: “The cohorts overlap, so the estimates must have a particular positive correlation.” Correction: Overlap can contribute to dependence, but the magnitude and even direction of covariance also depend on the design and trait relationship. Estimate it or examine plausible alternatives.
Wrong reasoning: “A combined estimate is appropriate whenever there are several molecular estimates.” Correction: Pooling into one $\theta$ imposes a common-effect model. If layer effects have different scientific meanings or may differ, use a vector-based question such as the joint Wald test instead.
Wrong reasoning: “A smaller standard error proves stronger biological evidence.” Correction: Precision depends on the covariance assumptions as well as the data. Report the assumptions and interpret the result as statistical evidence, not as proof of a mechanism or causal effect.

When participant-level data are available, covariance can be estimated from the joint analysis or from the relevant joint score or influence-function contributions. When only summary estimates are available, use documented overlap and design information if it supports a covariance estimate; otherwise show how results change across a justified range. A sensitivity analysis does not remove uncertainty about overlap, but it makes the consequences of that uncertainty visible.

Key takeaway: Integrated evidence must be weighted using the joint covariance of its estimates. Independence is an assumption to justify, not a default created by measuring different molecular layers. If cross-layer dependence is uncertain, show how the estimate, standard error, and inference change over plausible covariance scenarios.

Check Your Understanding

Answer each question using the covariance structure and scientific target in the situation.

Two molecular association estimates have standard errors $0.10$ and $0.12$. What additional quantity is needed to specify their joint covariance matrix?
Why can overlapping cohorts make an independence-based standard error too small?
What scientific assumption is imposed when estimating one common effect across several layers?
For two equally precise estimates with positive sampling correlation, how does the variance of their average compare with the variance calculated under independence?
If the cross-layer correlation is not known, what should a sensitivity analysis vary, and what should it report?