Cross-Cohort Replication Across Sequencing Pipelines

Why Cross-Cohort Replication Needs a Shared Measurement Plan

A rare-variant finding can be tested in a new cohort only if the two cohorts measure a sufficiently comparable genetic target and phenotype. Exome capture kits may cover different exons; sequencing and calling pipelines may represent the same allele differently; and cohorts may use different diagnostic criteria or ancestry compositions. If those differences are left implicit, an apparent difference in gene-level results can reflect measurement or study design rather than a meaningful difference in evidence.

Replication does not require every cohort to observe the same rare alleles. Different participants can carry different alleles in the same gene. It does require a pre-specified basis for deciding which genomic positions and variant classes enter the gene-level analysis, how unobserved genotypes are treated, and which phenotype and ancestry comparisons are intended. The earlier material on replication criteria for gene-level rare-variant findings concerns the criteria for interpreting a replication result; here the focus is making the cross-cohort inputs comparable.

Harmonize the Genomic Target and Callability

Start by expressing the target in a common genome assembly and coordinate system. If cohorts use different assemblies, transform coordinates and verify that the resulting intervals and alleles are consistent. For each gene, define the region being tested—for example, specified coding exons or a defined set of splice-adjacent bases—and record how that region was selected. A gene name alone is not a sufficiently precise target: gene boundaries, transcripts, and exon definitions can differ across annotation releases.

For each cohort, determine which bases in that target are adequately covered and callable under quality rules set in advance. Coverage thresholds are design choices, not universal constants. They should be chosen to support reliable rare-variant genotypes, applied consistently within a cohort, and reported with the resulting callable fraction. Because callability can vary by participant and position, a cohort-wide statement such as “the gene was sequenced” is not enough.

One conservative strategy is to use only the genomic territory that is callable under the planned rules in every cohort. Let \(C_j\) be the set of target positions that meet the callability criteria in cohort \(j\). The shared territory is the intersection of those sets. If a study instead retains cohort-specific territory, it must specify how different eligible regions are handled; otherwise, the cohorts may be testing different variant opportunities.

The intersection cannot be larger than any cohort’s callable set. This follows directly from the definition of intersection: every position in \(C_{\mathrm{shared}}\) must belong to each \(C_j\). Thus, restricting analysis to shared territory improves comparability but can reduce the number of testable bases and the number of observable rare alleles. That trade-off should be decided before looking at replication results.

At the participant level, define how much of the shared target must be callable for a sample to contribute, and apply the rule without reference to case status. At the variant level, set criteria for depth, genotype quality, and other relevant quality measures. A variant with a confident reference genotype is different from a position with insufficient data: only the former supports classifying that participant as a noncarrier. If one cohort has systematically poorer callability among cases than controls, this also threatens comparability and should be addressed in the design and quality-control plan.

Harmonize Variant Definitions and Gene-Level Inputs

A shared region is only one part of a shared analysis. The same underlying allele can be encoded differently across pipelines, especially for insertions and deletions or multiallelic sites. Normalize variant representation against the same reference assembly, align alleles, and reconcile equivalent records before applying filters. Use a consistent annotation release, transcript policy, and gene assignment. Document how consequences such as protein-truncating, missense, or splice-region variants are classified.

Next, pre-specify the variant mask: the rules that determine which variants enter a gene-level test. These may include functional consequences, quality filters, and a minor-allele-frequency threshold and reference definition. If frequency thresholds depend on ancestry, specify the ancestry-specific frequency source and the handling of variants with uncertain frequency. A mask must have the same meaning across cohorts; matching only the label “damaging” is inadequate if annotation rules differ.

The gene-level analysis inputs include more than a list of variant names. They include the eligible variant set, participant genotypes or genotype-derived scores, callability and missingness handling, variant weights if used, phenotype, covariates, and the analysis method. Select the same test and input rules for each cohort, or explain in advance how any unavoidable differences will be reconciled. Do not convert an uncallable genotype into a reference genotype simply to create a complete matrix.

Align Phenotype and Ancestry Comparisons

A gene-level association can be compared across cohorts only when the phenotype has a sufficiently consistent meaning. Define the case criteria, control criteria, measurement window, diagnostic source, and handling of uncertain or intermediate records. If one cohort includes broad clinical diagnoses and another requires a narrower adjudicated definition, the analyses may address different outcomes. A replication plan can choose a common definition, or specify a primary definition and justified cohort-specific adaptations, but it should make that choice before testing.

Control selection matters as well. Controls should be assessed for the relevant phenotype with comparable care, rather than treated as unaffected solely because no diagnosis appears in a record. Excluding ambiguous participants may improve definition consistency, though it can reduce sample size. Record how many participants are excluded under the harmonized rules and ensure those rules do not depend on genetic results.

Ancestry must be handled because rare-allele frequencies can differ across populations. Decide whether the cohorts will be ancestry-matched, analyzed in pre-defined strata, or adjusted using appropriate ancestry covariates. Apply a coherent strategy to cases and controls, and avoid interpreting broad social categories as precise measures of genetic ancestry. If cohort-specific estimates are combined, specify the target ancestry groups and how the estimates will be compared; a crude pooled comparison can confound differences in ancestry composition with differences in case status.

Pre-Specify the Analysis and Its Interpretation

Before running the replication analysis, write down the phenotype model, covariates, variant mask, gene boundaries, test, and missingness rules. For example, a binary phenotype may be analyzed with a logistic regression model whose predictors include a gene-level genetic score and pre-specified covariates. Such a model requires a correctly defined binary outcome, an appropriate relationship between the predictors and the log odds, and observations whose dependence structure is handled by the analysis. If participants are related, a method that assumes independent observations without accounting for that relationship may not be appropriate.

The genetic score itself must be defined consistently. A burden score may count or weight qualifying alleles, while other gene-level procedures can accommodate different patterns of effects. The choice should reflect the intended genetic alternative and be held constant across cohorts when possible. This tutorial does not require a particular test: the critical design point is that the same eligible variants, score construction, covariate strategy, and inferential procedure are used or that differences are explicitly pre-specified.

A replication plan should also state what result would count as supporting replication, including the direction or form of the expected effect where applicable, the significance criterion, and how the gene-level analysis fits any multiplicity procedure. A non-significant result is not proof of no association. In particular, fewer callable bases, fewer participants, or fewer qualifying alleles can lower power even when the underlying effects are similar. The analysis should report the tested territory, sample counts, callability, and the defined gene-level result so readers can assess what was actually replicated.

Common Pitfalls and Why the Plan Matters

The most consequential mistake is to treat a variant that was not called as a reference genotype. This inflates the apparent number of noncarriers and can bias comparisons if callability differs between cohorts or case groups. The correction is to retain missingness explicitly and require adequate callability under rules set before the association analysis.

• Assuming the same gene name means the same target. Different assemblies, transcripts, and exon boundaries can change which bases are tested. Specify the coordinates and annotation policy.
• Comparing different masks as if they were identical. A difference in consequence annotation or frequency reference changes the gene-level input. Harmonize definitions and document versions.
• Using crude ancestry-pooled comparisons when groups are imbalanced. Different rare-allele frequencies across ancestry strata can create a compositional difference. Match, stratify, or adjust in a pre-specified way.
• Expecting identical rare alleles in both cohorts. Replication is usually about consistent evidence under a comparable gene-level definition, not the repeated observation of every individual allele.
• Interpreting a non-significant result as evidence of no effect. A narrower shared target or smaller effective sample can reduce power. Report these design limits rather than treating significance alone as a measure of comparability.

A well-planned cross-cohort replication makes the comparison interpretable even when sequencing platforms differ. It separates variation in the biological signal from variation in what each pipeline could reliably observe, while preserving the distinction between a confidently observed noncarrier and an unknown genotype.