Replicating Cell-Type Genetic Associations

What Is Being Replicated?

A cell-type genetic association is not just a variant near a gene. It is a defined claim about a genetic effect on an expression measure in a specified cell population, under a stated analysis and covariate adjustment. Independent replication asks whether that claim is supported in a separate set of donors, rather than merely whether a second analysis produces a small p-value.

Before looking at results, specify the target association: the gene or transcript, variant, cell population, effect allele, expression scale, effect measure, and relevant adjustment variables. This makes it possible to tell whether two studies tested the same hypothesis. If one study tests a gene-level pseudobulk expression measure in classical monocytes and another tests a transcript-level measure in all monocytes, the results may be informative about related biology, but they are not automatically direct replications.

Definition: Independent replication is an evaluation of a prespecified genetic association in a separate donor sample, using a sufficiently comparable phenotype, cell population, variant or justified proxy, effect-allele orientation, and analysis target. Replication evidence concerns the estimated effect and its uncertainty in context; it does not require the replication result to cross the same significance threshold as the discovery result.

Donors—not cells—are the independent sampling units for donor-level replication. A replication set should not reuse discovery donors, even if it contains newly sequenced cells or a different assay of the same donors. Reusing donors can make the second result dependent on the first and can overstate how well the association generalizes.

Harmonize the Association Before Comparing Results

Four kinds of alignment are especially important: cell identity, gene and variant identity, expression measurement, and covariate adjustment. A disagreement in any one can look like a failure to replicate even when the underlying biological effect is compatible.

Harmonization target: A comparable replication analysis should refer to the same biological cell population, the same gene or clearly justified molecular feature, the same variant (or a proxy with known linkage disequilibrium and allele phase), the same effect allele, a comparable expression scale, and a clearly described adjustment set.

Cell labels. Cell names are not universally interchangeable. One data set may distinguish classical and nonclassical monocytes, while another reports a broad monocyte category. Compare definitions, markers, annotation procedures, and state composition—not just label text. A broad label can average effects that differ across subtypes. Likewise, a cell-state definition that depends on variables affected by genotype may change the target association; earlier material on collider bias from conditioning on cell states explains why such conditioning requires care.

Gene and variant identity. Confirm genome build, chromosome position, reference and alternate alleles, and gene identifier or transcript definition. The same rs identifier can be represented differently across files, and the same gene symbol can refer to different transcript-level measurements. A nearby proxy variant is not automatically equivalent: its relationship to the tested variant depends on linkage disequilibrium and allele phase in the relevant population.

Effect allele. An effect estimate is tied to a specified allele count. If one analysis reports the effect per G allele and another reports the effect per A allele at the same diploid biallelic variant, their estimates have opposite signs when the allele orientation is reversed. For a diploid biallelic genotype, the A-allele count is two minus the G-allele count. More generally, at fixed ploidy \(P\), the count of one allele is \(P\) minus the count of the other allele. Do not use the diploid complement for a genotype with a different ploidy. For a variant with a third allele, this two-allele sign reversal is not an adequate harmonization rule.

Expression and adjustment. A coefficient per unit of log-expression is not numerically comparable to a coefficient per standard deviation unless the scale conversion is known. Normalization, transformation, filtering, and aggregation can also alter the phenotype. Covariates matter because adjusted estimates describe effects conditional on the chosen variables. Compare adjustment sets and their definitions, including ancestry variables, technical factors, and relevant donor characteristics. A different covariate set may change precision or, when it changes confounding control or the target population, the estimate itself.

A Practical Replication Workflow

Write down the target claim.
Specify gene, cell population, variant, effect allele, expression measure, effect scale, and covariates before inspecting the replication result.

Check independence and data quality.
Verify donor nonoverlap and examine whether assay quality, cell yield, genotype quality, and sample composition make the comparison interpretable.

Harmonize biological and technical identities.
Align cell definitions, gene identifiers, genome build, variant alleles, expression transformations, and covariate definitions. Record any mismatch that cannot be resolved.

Estimate and compare effects.
Report effect estimates with standard errors or confidence intervals on a common orientation and, where possible, a common scale. Interpret direction, magnitude, and precision together.

Investigate discordance without overclaiming.
Consider power, cell composition, measurement, population differences, and model adjustment. Distinguish a measurement mismatch from plausible biological heterogeneity, and state what the data do not establish.

Comparing Effects and Their Uncertainty

For a quantitative expression phenotype, a common starting point is the estimated genetic effect \(\hat{\beta}\), its standard error \(SE\), and a confidence interval. The precise model can depend on the assay and data structure; the essential replication comparison is between estimates that refer to the same allele, population, and scale. A p-value alone is not an effect-size comparison.

If the discovery and replication estimates are independent and expressed on the same scale and allele orientation, their difference is \(\hat{\beta}_{D}-\hat{\beta}_{R}\). Its standard error can be calculated from the two standard errors when the estimates are independent:

SE(\hat{\beta}_{D}-\hat{\beta}_{R}) = \sqrt{SE_D^2+SE_R^2}.

This follows because the variance of a difference of independent estimates is the sum of their variances: \(\operatorname{Var}(\hat{\beta}_{D}-\hat{\beta}_{R})=\operatorname{Var}(\hat{\beta}_{D})+\operatorname{Var}(\hat{\beta}_{R})\). Dividing the observed difference by this standard error gives a standardized comparison under the usual approximate normal sampling model. The calculation is not appropriate as written for overlapping donor samples, nor does a nonsignificant difference prove the effects are identical.

Worked Example: Reversing an Allele Orientation

Suppose a hypothetical discovery analysis estimates a gene-expression effect of \(0.24\) log2-expression units per G allele, with \(SE_D=0.08\). An independent replication analysis reports \(-0.20\) units per A allele, with \(SE_R=0.09\), at the same diploid biallelic variant. Harmonize the orientation and assess the evidence.

For diploid genotypes at this biallelic site, the A dosage is two minus the G dosage. Thus, if the model is written per A allele, the slope has the opposite sign from the slope per G allele. Re-expressing the replication estimate per G allele gives \(\hat{\beta}_R=+0.20\), with the same standard error, \(0.09\).

The replication estimate is positive and its two-sided test statistic against zero is:

z_R=\frac{0.20}{0.09}=2.22.

The corresponding two-sided normal-approximation p-value is about \(0.026\). A 95% confidence interval is \(0.20\pm1.96(0.09)\), or \(0.0236\) to \(0.376\) log2-expression units per G allele.

The harmonized difference is \(0.24-0.20=0.04\). Its standard error is \(\sqrt{0.08^2+0.09^2}=\sqrt{0.0145}=0.120\), so the standardized difference is \(0.04/\sqrt{0.08^2+0.09^2}=0.3322\), with a two-sided p-value of about \(0.74\). These hypothetical estimates have the same direction, and their difference is small relative to its uncertainty. This is compatible with replication, but it does not establish exact equality of effects.

Worked Comparisons Across Cell Labels and Scales

Worked Example: A Broad Cell Label Can Hide Subtype Effects

Imagine a hypothetical discovery study reporting a positive effect in classical monocytes. A separate study measures a broad monocyte population containing 40% classical and 60% nonclassical monocytes. On a common expression scale, suppose the true subtype-specific effects are \(0.30\) units per allele in classical monocytes and \(-0.10\) units per allele in nonclassical monocytes. What effect might the broad label show?

Under the simplifying assumptions that the proportions are fixed across genotype groups, the measurements are on the same scale, and the broad-population effect is adequately represented by the composition-weighted average, the expected broad effect is:

0.40(0.30)+0.60(-0.10) = 0.12-0.06 = 0.06 \text{ units per allele}.

The broad-label estimate is much smaller than the classical-monocyte effect, despite a positive classical effect being present. If the proportions instead varied with genotype, a simple weighted-average interpretation could fail because the mixture itself would change across genotypes. The broad estimate would then combine within-subtype expression effects with composition differences. The appropriate conclusion is that the labels target different populations; the smaller broad-label estimate alone does not show that the subtype-specific result is false.

Worked Example: Converting a Standardized Effect to Expression Units

Suppose a hypothetical discovery study reports an effect of \(0.18\) standard deviations of expression per effect allele, with \(SE_D=0.08\). The discovery expression standard deviation is \(0.50\) log2-expression units. An independent replication reports \(0.10\) log2-expression units per the same allele, with \(SE_R=0.04\). Compare the estimates on the log2 scale.

Multiplying the standardized discovery effect and its standard error by the expression standard deviation gives:

\hat{\beta}_D=0.18(0.50)=0.090 \text{ log2 units per allele}, \qquad SE_D=0.08(0.50)=0.040 \text{ log2 units per allele}.

The replication estimate is \(0.100\) log2 units per allele, so the difference is \(0.090-0.100=-0.010\). Assuming independent donors and comparable adjustment, the standard error of that difference is \(\sqrt{0.040^2+0.040^2}=0.0566\). The standardized difference is \(-0.010/0.0566=-0.177\), giving a two-sided p-value of about \(0.86\). The replication estimate itself has statistic \(0.100/0.040=2.50\), with a two-sided p-value of about \(0.012\). The estimates are close on the converted scale; comparing \(0.18\) directly with \(0.10\) would have been invalid because the former is measured in standard deviations and the latter in log2-expression units.

This conversion is justified only if the reported standardized effect is defined as change in expression standard deviations per allele and the stated \(0.50\) is the standard deviation used to standardize that phenotype. If the studies use different standardization populations or transformations, the conversion may not put them on a genuinely equivalent scale.

When Replication Appears to Fail

A replication result can be nonsignificant because its estimate is imprecise, even when its point estimate is close to discovery. Conversely, a small p-value does not guarantee that the effect is meaningfully similar: a precise estimate in a large sample can be statistically different from discovery while remaining close in practical magnitude, or it can differ substantially. Report the estimates, uncertainty, and relevant context rather than reducing the assessment to a shared significance threshold.

If direction or magnitude differs, first verify allele orientation, phenotype scale, variant identity, and cell-label comparability. Then consider whether differences in cell-state composition, ancestry, environment, assay, or adjustment could plausibly alter the measured association. Such differences are hypotheses to investigate, not automatic explanations. A measurement mismatch should be described as a limitation of comparability; a biological explanation requires evidence that distinguishes it from sampling variation and technical differences.

Weak evidence: the replication estimate is too imprecise to distinguish the target effect from zero or from a range of meaningful effects.
Measurement mismatch: the studies use non-equivalent cell labels, expression definitions, scales, or variants, so the estimates do not directly target the same quantity.
Potential heterogeneity: after harmonization, effects still differ in a way that may reflect genuine population or biological differences, but the cause needs further assessment.

A useful report states what matched, what did not, the harmonized estimates and intervals, and how much evidence exists for a difference. Avoid describing a nonsignificant replication as proof of no effect, or a matching sign as decisive replication when the estimate is highly uncertain.

Common Mistake

Common mistake: “The discovery p-value was below 0.05, but the replication p-value was above 0.05, so the association failed to replicate.” This reasoning treats significance as a property of the effect rather than a joint function of effect size and precision. The correction is to align the hypothesis and scales, compare effect estimates and uncertainty, and determine whether the replication data are precise enough to support or contradict the discovery-sized effect.

Key takeaway: Replication is a comparison of clearly specified, independent estimates—not a contest between p-values. Harmonize the biological population, allele, phenotype scale, and adjustment before interpreting agreement or disagreement.

Check Your Understanding

Answer each question by identifying the target of comparison, not just the statistical result.

Why does sequencing new cells from the same donors not by itself provide independent replication?
A replication report uses the opposite allele from discovery at a diploid biallelic variant. What must be done to compare the effect directions?
Discovery tests a defined monocyte subtype, while replication tests all monocytes. Give one reason the estimates could differ even if the subtype association is real.
Discovery reports effects per standard deviation of expression and replication reports effects per log2-expression unit. What information is needed before comparing magnitudes?
Why does a nonsignificant replication p-value not prove that the genetic effect is absent?