What Does an Effect-Heterogeneity Test Ask?
An association estimate can differ between ancestry groups for several reasons. The underlying genetic effect may differ, but estimates can also vary because of sampling error, allele-frequency differences, linkage disequilibrium, phenotype measurement, or model misspecification. A test of effect heterogeneity asks a narrower question: are the group-specific association effects equal on the scale defined by the chosen model? It does not, by itself, identify why they differ.
The scale matters. For a quantitative outcome analyzed with a linear model, the comparison may be on the outcome-mean scale per allele. For a binary outcome analyzed with logistic regression, it is on the log-odds scale per allele. The earlier material on population-specific association estimands and marginal effect changes across populations motivates stating the estimand and model before comparing effects. In what follows, ancestry groups are indexed by \(g=1,\ldots,K\), and the effect in group \(g\) is \(\beta_g\).
An ancestry-specific effect is interpretable only relative to its model, allele coding, outcome definition, adjustment set, and analysis population. The tests below address equality of the modeled effects, not whether the variant is causal or whether it has the same biological mechanism across groups.
Interaction Tests in Individual-Level Data
For two ancestry groups, let \(A_i=0\) denote the reference group and \(A_i=1\) the other group. Let \(G_i\) be the genotype dosage for the same effect allele in both groups, and let \(C_i\) denote covariates. A linear model for a quantitative outcome can be written
The genotype slope in the reference group is \(\beta_G\); in the other group it is \(\beta_G+\delta\). Thus the test of equal slopes is \(H_0:\delta=0\). For \(K\) groups, use \(K-1\) ancestry indicators and their interactions with genotype. The joint null is that all \(K-1\) interaction coefficients equal zero. Logistic regression uses the same linear predictor for the log odds, so its interaction tests equality of log-odds effects, not equality of odds ratios.
An interaction coefficient can be tested with a Wald test, a score test, or a likelihood-ratio test. For a single coefficient, the Wald statistic is \(\hat{\delta}/SE(\hat{\delta})\), compared approximately with a standard normal distribution under the null. For \(K-1\) coefficients, a joint Wald statistic is compared with a chi-square distribution with \(K-1\) degrees of freedom. These approximations require adequate information and a correctly specified model.
The model should include appropriate ancestry main effects and relevant covariates, and should account for population structure and relatedness as required by the study design. The genotype-by-ancestry term must be estimable: if a genotype has no variation in one group, that group’s slope cannot be reliably identified. The genotype coding and effect allele must also be aligned across groups. If individuals are related, or if the analysis requires a mixed model, the variance and test calibration must reflect that structure.
Worked Example: A Two-Group Interaction Test
Suppose a hypothetical study fits an adjusted linear model for a quantitative biomarker measured in units of mg/L. Genotype dosage counts copies of the same effect allele. Group 1 is the reference; group 2 is represented by \(A_i=1\). The fitted genotype coefficient is \(0.20\) mg/L per allele, and the genotype-by-group coefficient is \(\hat{\delta}=0.30\) mg/L per allele, with \(SE(\hat{\delta})=0.12\) mg/L per allele. Assume independent subjects, an adequate linear mean model, appropriate covariate adjustment, and sufficient genotype variation in both groups.
The estimated group 1 effect is \(0.20\) mg/L per allele. The estimated group 2 effect is \(0.20+0.30=0.50\) mg/L per allele. The heterogeneity null is \(H_0:\delta=0\).
The two-sided normal p-value is approximately \(0.0124\) (equivalently, the chi-square p-value with 1 degree of freedom). At a nominal 0.05 level, these data provide evidence that the adjusted effects differ on the outcome-mean scale. The result is not proof of a biological difference, and it does not establish that either effect is causal. In a genome-wide analysis, multiplicity must also be addressed before treating this as a discovery.
Cochran’s Q From Group-Specific Estimates
When each ancestry group has a separate association estimate and standard error, Cochran’s Q provides an omnibus test of whether the estimates are more dispersed than expected from sampling error under a common-effect model. In its standard form, it assumes approximately independent, unbiased, normally distributed estimates with known or accurately estimated variances. Write the estimate and its variance in group \(g\) as \(\hat{\beta}_g\) and \(s_g^2\), and define \(w_g=1/s_g^2\).
The weighted mean \(\bar{\beta}_w\) is the common value fitted under the null: precise estimates contribute more than imprecise ones. The \(K-1\) degrees of freedom arise because the \(K\) estimates are compared with one estimated common value. A large Q indicates inconsistency relative to the estimated sampling variances; it is not an estimate of the size or cause of heterogeneity. If estimates are correlated—for example, because samples overlap—the ordinary formula and reference distribution are not appropriate without accounting for that covariance.
Worked Example: Calculating Cochran’s Q
Consider hypothetical effect estimates, all in outcome units per allele, from three independent ancestry-group analyses: \(\hat{\beta}_1=0.20\), \(SE_1=0.10\); \(\hat{\beta}_2=0.50\), \(SE_2=0.20\); and \(\hat{\beta}_3=-0.10\), \(SE_3=0.15\). Assume the groups do not overlap, the estimates are approximately normal and unbiased, and their standard errors are well estimated.
The inverse-variance weights are \(w_1=1/0.10^2=100\), \(w_2=1/0.20^2=25\), and \(w_3=1/0.15^2=44.444\). The weighted mean is
Substituting into Q gives
With \(K-1=2\) degrees of freedom, the chi-square p-value is approximately \(0.0486\). This is modest evidence against a common effect under the assumptions, at a nominal 0.05 level. It does not show which group differs, nor does it establish that the apparent differences exceed a practically important threshold. The estimates and their uncertainty should be examined alongside the omnibus result.
Likelihood-Ratio Tests
A likelihood-ratio test compares two nested models fitted to the same observations under the same likelihood. The restricted model imposes the common-effect null; the unrestricted model allows ancestry-specific effects. If \(\ell_R\) and \(\ell_U\) are their maximized log-likelihoods, respectively, the statistic is \(2(\ell_U-\ell_R)\). Under regularity conditions and the null, it is approximately chi-square with degrees of freedom equal to the number of independent restrictions—typically \(K-1\) for \(K\) group effects constrained to be equal.
The test is useful when a likelihood model is already being fitted, including logistic regression. Its validity depends on correct likelihood specification, comparable models, adequate sample information, and regularity conditions. Do not compare likelihoods calculated from different sets of individuals or incompatible response definitions. A boundary parameter or severe sparsity can invalidate the ordinary chi-square approximation.
Worked Example: Likelihood-Ratio Test in a Logistic Model
In a hypothetical binary-outcome study, researchers fit a logistic regression adjusted for the same covariates in both models. The restricted model gives every ancestry group one common genotype log-odds coefficient. The unrestricted model gives three groups separate genotype coefficients. Both models use the same individuals and likelihood. Their maximized log-likelihoods are \(\ell_R=-210.0\) and \(\ell_U=-205.8\). Assume the models are correctly specified, the parameters are identifiable, and sample sizes are sufficient for the asymptotic approximation.
There are \(3-1=2\) equality restrictions, so the reference distribution is chi-square with 2 degrees of freedom. The p-value is approximately \(0.0150\). At a nominal 0.05 level, the data provide evidence that at least one group’s genotype log-odds effect differs from the others. The result neither names the differing group nor implies a difference in risk ratios or absolute risks.
Choosing a Test, Checking Performance, and Avoiding Overinterpretation
Interaction tests use individual-level data and can incorporate covariates, ancestry indicators, and an appropriate outcome model directly. Cochran’s Q uses group-specific estimates and standard errors, making it useful when only summary estimates are available; its ordinary form relies on independence and compatible effect scales. The likelihood-ratio test is a way to compare nested likelihood models and is not limited to a particular interaction parameterization. When the same correctly specified likelihood model is available, it tests the same equality constraints as the corresponding interaction model, though finite-sample results can differ from Wald or score tests.
Power depends on more than the total sample size. A group with few participants, low allele frequency, or poor genotype variation may have a large standard error, making its effect difficult to estimate. Unequal group sizes can also limit precision. An omnibus test spends degrees of freedom on several possible contrasts; it may have less power than a prespecified one-degree-of-freedom contrast when the expected pattern is known. Conversely, selecting a contrast after seeing the estimates requires accounting for that selection.
The tests can fail or mislead when group-specific models are not comparable, allele coding is inconsistent, ancestry or relatedness is inadequately handled, or model assumptions are poor. For Q, overlapping samples can induce covariance among estimates. For interaction and likelihood-ratio tests, sparse genotype-by-group cells can cause unstable estimates or unreliable asymptotic p-values. Diagnostics include checking group-specific allele counts and standard errors, convergence and coefficient stability, covariate balance, and whether the assumed outcome model fits adequately. A large p-value is not evidence that effects are identical; low power can leave meaningful differences undetected.
A frequent mistake is to interpret a significant association in one group and a nonsignificant association in another as proof that the effects differ. The difference between “significant” and “not significant” is not itself a test of the difference. The correction is to test the interaction or another explicit equality hypothesis, then report the effect estimates and uncertainty in each group. Likewise, a heterogeneity p-value does not establish causation, explain the source of the difference, or quantify its importance.
These tests answer an equality question for specified effects. They do not combine the effects into an overall association estimate, which is a separate inferential task. Whatever test is used, report the model scale, group-specific estimates, uncertainty, test statistic, degrees of freedom, p-value, and any multiplicity procedure relevant to the analysis.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a two-group linear model with genotype-by-ancestry coefficient \(\hat{\delta}=-0.24\) and standard error \(0.08\), calculate the Wald statistic and state the null being tested.
- Why does the standard Cochran’s Q test have \(K-1\), rather than \(K\), degrees of freedom?
- Two ancestry-specific estimates come from overlapping participants. What assumption of the ordinary Q formula is threatened?
- A variant has \(p=0.01\) in group 1 and \(p=0.20\) in group 2. Does this establish effect heterogeneity? Explain.
Solutions. 1. \(z=-0.24/0.08=-3.00\), so \(z^2=9.00\). The null is that the genotype slopes are equal across the two groups, equivalently \(\delta=0\); the two-sided normal p-value is about \(0.0027\). 2. The \(K\) estimates are compared with one estimated common weighted mean, using one degree of freedom, leaving \(K-1\) independent deviations. 3. Independence of the group-specific estimates is threatened. Their covariance must be accounted for; the ordinary Q reference distribution may not be valid. 4. No. Separate p-values test association within each group, not equality between effects. Test an explicit genotype-by-ancestry interaction or other equality hypothesis and compare the estimated effects with their uncertainty.