Replication Estimands: Effect, Direction, and Population

Why a Replication Claim Needs an Estimand

A reported genetic association is not just a variant name and a p-value. It refers to a particular contrast in genotype, an outcome defined and measured in a particular way, an effect scale, a statistical model, and a population. A replication claim is interpretable only when these pieces identify what quantity is meant to be reproduced. Two studies can analyze the same locus and outcome label yet estimate different quantities because they code alleles differently, use different phenotype definitions, or target different populations.

This tutorial focuses on defining that quantity—the estimand—and on separating two claims that are often conflated. Replication of direction asks whether an effect points the same way under aligned coding. Replication of magnitude asks whether the effect sizes are sufficiently alike on a stated scale. Neither claim is fully specified by saying that a result is “replicated.”

Definition: A genetic association estimand is the population quantity that an analysis aims to estimate. To define it, state the genetic contrast and allele coding, the outcome and its scale, the model and effect measure, and the target population. An association estimand describes a statistical relationship; by itself, it does not establish that changing genotype would cause a change in outcome.

Specify the Genetic Contrast and Effect Measure

For a biallelic variant, let \(A\) be the allele designated as the effect allele and let \(G\) be its dosage, usually taking values from 0 to 2. With an additive dosage model, the model treats each additional copy of \(A\) as adding the same amount on the model’s effect scale. The other allele is the reference allele. The labels “effect” and “reference” are coding choices; they do not imply that one allele is beneficial or harmful.

The outcome and effect scale must also be explicit. For a quantitative outcome, a linear regression coefficient is a mean difference in outcome units per additional copy of the effect allele, conditional on included covariates. For a binary outcome, a logistic regression coefficient is a change in log odds per additional copy, conditional on the model covariates; exponentiating it gives the per-allele odds ratio. These are different estimands, even if both analyses use the same variant and participants.

Model definition: Under an additive linear model, \(E(Y\mid G,C)=\beta_0+\beta_GG+\gamma^\mathsf{T}C\), where \(Y\) is a quantitative outcome and \(C\) is the specified covariate vector. The conditional mean difference per additional effect allele is \(\beta_G\), in units of \(Y\). Under an additive logistic model, \(\log\{P(Y=1\mid G,C)/[1-P(Y=1\mid G,C)]\}=\beta_0+\beta_GG+\gamma^\mathsf{T}C\); the conditional odds ratio per additional effect allele is \(\exp(\beta_G)\). Each interpretation depends on the stated model being an adequate description for the estimand of interest.

The additive assumption is not automatic. It constrains the genotype effect: on the chosen model scale, the difference between dosage 0 and 1 equals the difference between dosage 1 and 2. A dominant, recessive, genotype-category, or dosage-by-environment analysis targets a different contrast. Therefore, the model form belongs in the estimand description, not merely in the methods details.

Worked Example: Interpreting a Per-Allele Logistic Effect

Suppose a hypothetical study analyzes disease status using an additive logistic model. The effect allele is G, dosage \(G\) ranges from 0 to 2, and the estimated coefficient is \(\hat{\beta}_G=0.182\). The outcome is disease status, and the model adjusts for age and sex.

The effect measure is a conditional log odds difference per additional G allele. Exponentiating the coefficient gives the per-allele odds ratio:

\exp(\hat{\beta}_G)=\exp(0.182)=1.200

Under the fitted model, holding age and sex fixed, one additional G allele corresponds to estimated disease odds 1.200 times as large, or 20.0% higher odds. This is not a 20.0 percentage-point increase in disease probability. The probability change depends on the starting odds and hence on covariate values. The estimate is an association, not evidence by itself that the allele causes disease.

Direction Depends on Allele and Outcome Coding

A coefficient’s sign has meaning only with its coding attached. If dosage is changed from the count of allele \(A\) to the count of the other allele \(a\), then, for a diploid biallelic variant, \(G_a=2-G_A\). In a model with an intercept, replacing \(G_A\) by \(G_a\) reverses the genotype coefficient while leaving fitted values unchanged. The data have not changed; the sign has changed because the contrast has been reversed.

\beta_GG_A+\beta_0 =\beta_G(2-G_a)+\beta_0 =(2\beta_G+\beta_0)-\beta_GG_a

Thus the coefficient for \(G_a\) is \(-\beta_G\). In logistic regression, the odds ratio for the other allele is the reciprocal: \(\exp(-\beta_G)=1/\exp(\beta_G)\). Outcome coding matters as well. If a binary outcome is recoded by swapping cases and controls, the logistic coefficient changes sign under the same model structure. A direction comparison is meaningful only after both the effect allele and outcome event have been aligned.

Worked Example: Resolving an Apparent Sign Disagreement

In a hypothetical discovery analysis, the effect allele is C and the estimated linear-regression coefficient for a biomarker is \(-0.30\) units per C allele. A second analysis reports \(+0.30\) units per T allele for the same variant and biomarker. Assume C and T are the two alleles and each person has two copies in total.

The second analysis counts T rather than C. If \(G_C\) and \(G_T\) are their dosages, then \(G_T=2-G_C\). Re-expressing the first model in terms of \(G_T\) changes the coefficient to \(-(-0.30)=+0.30\) units per T allele. The reported signs therefore agree after allele alignment; they do not indicate opposing associations.

Now suppose the second estimate were \(-0.30\) units per T allele. Re-expressed per C allele, it would be \(+0.30\) units per C allele, which is opposite in direction to the discovery estimate of \(-0.30\). The sign comparison must be made on a common allele and outcome coding, not by comparing the printed plus and minus signs alone.

Define the Outcome and Target Population

A phenotype name may conceal differences that matter to the estimand. “Lipid level,” for example, might mean a particular lipid fraction measured in different units, at different ages, or before versus after treatment. A binary diagnosis may use different clinical thresholds or ascertainment rules. The outcome definition, measurement scale, and relevant timing should therefore be stated precisely enough to establish whether the studies concern the same variable.

The target population is the population to which the estimand refers. It can be described by eligibility criteria and relevant features such as ancestry, age range, disease status, or recruitment setting. A population label alone may be too broad: two cohorts described with the same ancestry category could differ in age, environmental exposures, sampling, or phenotype ascertainment. Conversely, a difference in recruitment source does not automatically imply different estimands if the intended target and model quantity are genuinely the same.

Association coefficients can differ across populations for reasons that do not require a change in a causal biological effect. Allele frequencies and linkage disequilibrium—the correlation between the analyzed variant and a causal variant—may differ, so a marker can tag underlying variation differently. Environmental context, phenotype measurement, covariate distributions, and model specification can also matter. In particular, a marker association is not necessarily the same parameter as the effect of a causal variant. These possibilities make population definition substantive, not decorative.

Worked Example: Same Outcome Label, Different Estimands

Consider two hypothetical studies of a variant and “lung function.” Study A models forced expiratory volume in liters among adults aged 40–70, adjusting for age, sex, and smoking status, in a specified population. Its per-allele coefficient is \(-0.045\) liters. Study B models a standardized lung-function score among adults aged 18–80, adjusts for age and sex but not smoking, and reports \(-0.12\) standard deviations.

The coefficients cannot be compared as magnitudes directly: one is in liters and the other in standard-deviation units. The populations and adjustment sets also differ. Both estimates are negative, but even a same-direction statement should be made only after confirming that the effect allele and outcome coding agree and that the two outcomes represent sufficiently comparable traits.

If Study B instead analyzed the same lung-function measure in liters, with matching eligibility, covariates, and allele coding, its coefficient could be compared with \(-0.045\) liters per allele as an estimate on a common scale. That would make magnitude comparison interpretable, though it would not alone establish that the estimates are statistically compatible or that the underlying populations are identical.

Direction and Magnitude Are Distinct Claims

Let \(\beta_D\) and \(\beta_R\) denote the target effects in a discovery and replication setting, respectively, after harmonizing allele and outcome coding and defining the same effect scale. A direction claim concerns whether the effects have the same sign: both positive or both negative. A magnitude claim concerns the values themselves, for example whether their difference is small relative to a prespecified scientific tolerance. Direction agreement does not imply equal magnitude, and similar absolute magnitudes do not imply direction agreement.

When effects are measured on different scales, comparing their numerical sizes is generally meaningless. A coefficient of 0.2 log-odds units is not directly comparable to a coefficient of 0.2 standard deviations. A transformation can sometimes put estimates on a common scale, but the transformation and assumptions must be stated. For logistic models, comparing odds ratios requires the same allele increment and outcome event; an odds ratio per two alleles is not the same quantity as an odds ratio per allele.

Key distinction: “Same direction” means the target effects have the same sign after allele and outcome harmonization. “Same magnitude” means their effect sizes are comparable on a common, explicitly stated scale. A sign match is not a magnitude match, and neither statement is guaranteed by matching variant identifiers and phenotype labels.

Justification: Why Harmonization Is Necessary

The coding identity \(G_a=2-G_A\) shows exactly why signs cannot be compared before allele alignment. Under the additive model, substituting this identity changes only the intercept and reverses the coefficient. The fitted association and its strength are unchanged, while its sign is reversed. Therefore, an unharmonized sign comparison can label the same association as opposite in direction.

By contrast, changing the outcome units changes coefficient magnitude. If a quantitative outcome is rescaled as \(Y^*=cY\), with \(c>0\), then the linear coefficient becomes \(\beta_G^*=c\beta_G\). Its sign is preserved, but its numerical magnitude changes. If \(c<0\), the sign reverses too. Thus direction is invariant to positive rescaling of a quantitative outcome, while magnitude is not; the unit and coding remain essential to interpretation.

Diagnostics, Pitfalls, and Limits

Before describing a result as replicated, check whether the estimands are actually aligned. Record the effect allele and reference allele, how dosage was defined, which outcome value is the event for a binary trait, the outcome units or transformation, the model form and covariates, and the population to which each estimate refers. These checks diagnose ambiguity in the claim; they do not replace formal assessment of uncertainty or study design.

Comparing signs without harmonizing alleles. A sign reversal can be caused entirely by counting the alternate allele. Re-express estimates using the same effect allele first.
Treating a phenotype label as a full definition. Different thresholds, measurement protocols, or transformations can define different outcomes. State what was measured and on what scale.
Calling different effect measures equal in size. A linear coefficient and a log-odds coefficient have different units and meanings. Do not compare their numerical values as though they were on one scale.
Assuming direction agreement proves replication of the same parameter. Effects can share a sign while differing in magnitude, target population, or model interpretation.
Interpreting association as causation. A regression coefficient describes the association under its model and population. Causal interpretation requires additional assumptions and evidence.

The estimand description does not guarantee that a model is correct, that estimates are precise, or that a result will generalize to every population. Nor does it determine by itself whether a replication study has adequate power or whether a formal criterion for success has been met. Its role is more basic: it makes clear which scientific quantity those later assessments would concern.

Fix the genetic contrast.
Name the effect allele, state whether dosage counts 0–2 copies, and give the assumed genetic model.

Fix the outcome and effect scale.
Specify the measured trait or binary event, its units or coding, and the model coefficient or effect measure.

Fix the population and conditioning.
Describe the target population and the covariates defining the conditional association, where applicable.

State the replication claim precisely.
Say whether the claim concerns direction, magnitude on a common scale, or both; do not use one as a substitute for the other.

Check Your Understanding

Answer each question by identifying the estimand or the coding issue involved.

A dosage model counts allele A, and its coefficient is positive. What additional information is needed before calling the effect direction “positive” in a replication comparison?
A linear coefficient is 0.25 centimeters per allele in one study and 0.10 meters per allele in another. Convert the first coefficient to meters and state whether the magnitudes agree numerically after conversion.
A binary-trait logistic model estimates \(\hat{\beta}_G=-0.223\) per allele. Calculate the odds ratio and interpret it in context, assuming the event and effect allele are specified.
Two studies report positive coefficients for the same variant, but one uses a standardized outcome and the other uses raw units. What can and cannot be concluded from the sign agreement?
Why can an association estimate for a marker variant differ between populations even if the marker has the same identifier and both studies use an additive model?

Solutions. (1) Confirm that both analyses count the same effect allele and code the same outcome event; otherwise the coefficient signs may reverse under recoding. Also verify that the modeled contrast is comparable. (2) Since \(1\) centimeter \(=0.01\) meters, \(0.25\) centimeters \(=0.0025\) meters per allele. This is not equal to \(0.10\) meters per allele: the values are on a common scale and differ substantially. (3) \(\exp(-0.223)=0.800\), approximately. Each additional copy of the specified effect allele is associated with estimated odds 0.800 times as large—about 20.0% lower odds—conditional on the model covariates. This is not a 20.0 percentage-point probability reduction. (4) The estimates have the same reported direction only if allele and outcome coding match. Their magnitudes cannot be compared directly until they are placed on a common scale; sign agreement does not establish equality of effects. (5) Allele frequencies and linkage disequilibrium can differ, changing how well the marker tags underlying variation. Population, phenotype measurement, and model conditioning can also differ, so the same marker label does not ensure the same target association.