Why Unequal Assay Panels Require an Explicit Comparison Plan
Disease cohorts are often collected at different times, in different laboratories, or for different scientific aims. One cohort might have RNA sequencing and DNA methylation, while another has RNA sequencing and proteomics. Even when both cohorts address the same disease question, their combined data do not automatically form one comparable multi-omics table. The first task is to decide which measurements represent the same biological feature in a sufficiently comparable way, and which conclusions must remain specific to an assay or cohort.
The aim is not to force every cohort into a single complete matrix. Instead, define the part of the evidence that can be compared, preserve assay-specific results with clear labels, and state the limits of each conclusion. This separation makes cross-cohort evidence more interpretable and reduces the risk of treating absent measurements as biological absence.
Define the Comparison Before Combining Results
Let \(c\) index cohorts, \(i\) index subjects within a cohort, \(p\) index assay platforms, and \(g\) index biological features. For a disease-versus-control analysis, a cohort-specific effect for feature \(g\) might be the adjusted difference in mean log-expression between disease and control subjects. Write that estimate as \(\hat{\beta}_{cg}\), with standard error \(SE_{cg}\). The exact model depends on the study design, but the target must be stated in the same scientific terms across cohorts before estimates are combined.
For example, a log2-expression difference can be comparable across cohorts if the gene identifiers, expression processing, contrast definition, and adjustment strategy are aligned sufficiently to give the estimates the same interpretation. Raw counts from different sequencing runs are not themselves directly comparable merely because both are called expression. Nor does sharing a gene name guarantee comparability if one platform measures a different transcript region or uses a different feature definition.
A practical comparison plan therefore records, for each cohort, the available assay, the feature identifiers and annotation versions, the measurement scale, quality-control exclusions, and the disease contrast. Ambiguous feature mappings should be excluded from the shared set or reported as uncertain, rather than resolved by silently choosing one mapping. The shared set is often the intersection of eligible features after these checks; the broader union can still be retained for cohort-specific reporting.
List which platforms and feature families were measured in each cohort, and distinguish an unmeasured feature from a measured feature with a low or undetected signal.
Use stable identifiers and documented annotation rules. Exclude ambiguous mappings when they prevent a defensible one-to-one comparison.
Specify the same biological contrast and compatible effect scale. State how normalization, covariate adjustment, and eligibility criteria relate across cohorts.
Summarize comparable effects across cohorts where justified; label results from nonshared assays as specific to the cohorts that measured them.
Worked Examples
Worked Example: Constructing a Shared Feature Set
Suppose two hypothetical disease cohorts have these panels after basic quality control. Cohort A measured RNA expression for 12,000 genes and proteins for 900 targets. Cohort B measured RNA expression for 10,500 genes and metabolites for 180 compounds. The gene lists share 9,200 stable gene identifiers. Among these, 300 map ambiguously between the cohorts’ annotation files, leaving 8,900 genes with defensible shared identities.
The RNA shared-feature set is therefore \(9{,}200-300=8{,}900\) genes, provided that the expression summaries also target comparable gene-level quantities and both cohorts use compatible transformations for their effects. Cohort A’s 900 protein targets are not part of that shared RNA set: they are a different assay and biological measurement, even if some proteins correspond to genes in the RNA panel. The 180 metabolites in Cohort B are likewise not RNA features.
A defensible report can compare disease-versus-control RNA effects for the 8,900 shared genes. It can separately describe proteins measured in Cohort A and metabolites measured in Cohort B as cohort-specific evidence. It cannot say that a protein result was replicated in Cohort B when that cohort did not assay the protein. The distinction is about what was measured, not whether a value happened to pass a significance threshold.
Worked Example: Combining Comparable Effects for a Shared Gene
In a hypothetical analysis, two independent cohorts estimate the disease-control difference in log2 expression for the same mapped gene, with compatible normalization and adjustment. Cohort A reports \(\hat{\beta}_{1g}=0.60\) log2 units and \(SE_{1g}=0.20\) log2 units. Cohort B reports \(\hat{\beta}_{2g}=0.80\) log2 units and \(SE_{2g}=0.25\) log2 units. Assume each estimate is approximately normally distributed around a common underlying effect, the standard errors are correctly estimated, and the cohorts are independent.
Under a fixed-effect model, inverse-variance weights are \(w_c=1/SE_{cg}^2\). Thus \(w_1=1/(0.20)^2=25\) and \(w_2=1/(0.25)^2=16\). The weighted estimate is
Its standard error under this model is \(SE(\hat{\beta}_{g})=\sqrt{1/\sum_cw_c}=\sqrt{1/41}=0.1562\) log2 units. An approximate 95% confidence interval is
The interval describes uncertainty in the common-effect estimate under the stated model; it is not a 95% probability that the fixed effect lies in this particular realized interval. The positive estimate indicates higher expression in disease than control on the specified log2 scale, on average under the model. The calculation is appropriate only if both cohort estimates refer to the same gene-level contrast and a common effect is scientifically defensible. If those conditions are doubtful, present cohort-specific estimates and uncertainty rather than presenting the pooled number as a universal effect.
Worked Example: Reporting a Cohort-Specific Assay Finding
Suppose a hypothetical metabolomics assay was run only in Cohort B. For a particular metabolite, the estimated disease-control difference is \(0.50\) log2 abundance units with \(SE=0.18\) log2 units. The cohort-specific model gives an approximate 95% confidence interval of
This result supports a positive disease-control difference for the measured metabolite in Cohort B under that analysis. Cohort A did not measure the metabolite, so it contributes neither a confirming estimate nor a zero effect. The proper wording is that the association was observed in Cohort B and could not be assessed in Cohort A with its available panel. It is not evidence of cross-cohort replication.
Why Inverse-Variance Pooling Has the Form It Does
The weighting in the previous example gives more influence to estimates with smaller variance. A short derivation shows the basis. Suppose independent cohort estimates \(Y_c=\hat{\beta}_{cg}\) are unbiased for one common effect \(\beta_g\), with known variances \(v_c=SE_{cg}^2\). Consider a weighted average \(\tilde{\beta}_g=\sum_c a_cY_c\), where \(\sum_c a_c=1\). It is unbiased because \(E(\tilde{\beta}_g)=\sum_c a_c\beta_g=\beta_g\). Independence gives variance \(\operatorname{Var}(\tilde{\beta}_g)=\sum_c a_c^2v_c\).
To minimize that variance subject to weights summing to one, use a Lagrange multiplier \(\lambda\) for the constraint. Differentiating \(\sum_c a_c^2v_c-\lambda(\sum_ca_c-1)\) with respect to \(a_c\) gives \(2a_cv_c-\lambda=0\), so \(a_c=\lambda/(2v_c)\). The constraint implies \(\lambda/2=1/\sum_c(1/v_c)\), hence \(a_c=(1/v_c)/\sum_j(1/v_j)\). Therefore the minimum-variance unbiased weighted average has the inverse-variance form used above. This derivation depends on the common-effect, independence, and variance assumptions; it does not establish that those assumptions fit every cross-cohort analysis.
Diagnostics, Alternatives, and Limits
Before interpreting a shared-feature result, check whether the feature mapping is stable, whether units and effect directions agree, and whether the contrast and covariate adjustment are sufficiently aligned. Review cohort-specific estimates and their uncertainty rather than relying only on a combined estimate. A sign reversal or markedly different magnitudes can signal that a single summary obscures relevant differences; it warrants investigation and cautious reporting, not automatic deletion of an inconvenient cohort. Also check whether a feature is genuinely unmeasured in a cohort, rather than recorded under a different identifier or filtered during quality control.
Several approaches are useful, but answer different questions:
| Approach | What it can answer | Main caution |
|---|---|---|
| Analyze the shared feature set | How do comparable features relate to disease across cohorts? | The shared set may omit biologically important features measured in only one cohort. |
| Combine cohort-specific effects | What is a summary effect for a shared feature under a stated common-effect model? | Pooling is not justified by shared feature names alone; effect targets and assumptions must align. |
| Report cohort-specific panels separately | What was observed in the assays available to each cohort? | These findings do not establish cross-cohort replication for unmeasured features. |
| Restrict analysis to one platform type | Can a narrower, consistently measured set support a clear comparison? | It may discard complementary evidence and limit the scientific scope. |
A common mistake is to code a feature absent from a cohort’s panel as zero and then include it in a cross-cohort average. This confuses not measured with measured and having a zero value; the resulting summary has no valid interpretation as a shared-feature effect. Keep measurement availability explicit. Another mistake is to standardize every cohort’s values and assume that this makes unlike targets equivalent. Standardization changes numerical scale, but it does not repair a mismatched feature definition, assay target, or scientific contrast.
The shared set can also be narrower than the biologically interesting set, and cohort-specific evidence may be less independently validated than shared evidence. Results from an assay available in one cohort can generate hypotheses, but should be labelled as such and accompanied by the cohort, assay, effect scale, and uncertainty. Cross-cohort integration is strongest when it makes clear both what can be compared and what cannot.
Check Your Understanding
Answer each question, then check the brief solutions.
- Two cohorts both measure a gene, but one reports transcript-level effects and the other reports gene-level effects. Is the feature automatically comparable? State the issue to resolve.
- A disease-control effect is \(0.40\) with \(SE=0.20\) in one cohort and \(0.60\) with \(SE=0.30\) in an independent cohort. Under a fixed common-effect model, calculate the inverse-variance estimate and its standard error.
- A protein was measured only in Cohort A and has a positive disease-control estimate. What can be said about Cohort B, which did not assay that protein?
- Why is replacing an unmeasured feature with zero invalid for a shared-feature comparison?
Solutions. (1) No. Establish that both measurements target a compatible biological quantity, with defensible feature mapping and effect interpretation; shared wording alone is insufficient. (2) The weights are \(1/(0.20)^2=25\) and \(1/(0.30)^2=11.111\). The estimate is \([25(0.40)+11.111(0.60)]/(36.111)=16.667/36.111=0.4615\). Its standard error is \(\sqrt{1/36.111}=0.1664\), in the effect’s units. (3) The association is observed in Cohort A under its analysis; Cohort B provides no estimate and cannot be called a replication or a nonreplication. (4) Zero is a measured value, whereas unmeasured means no assay result exists. Substitution invents data and changes the meaning of the comparison.