Heterogeneity In Pharmacogenomic Meta-Analysis

Why Heterogeneity Matters

A pharmacogenomic association can be replicated in several cohorts without having exactly the same estimated effect in each. Differences may reflect sampling error, but they can also arise because the cohorts studied different doses, defined response differently, included different ancestry groups, or followed different prescribing practices. Meta-analysis must therefore address two questions: what overall effect is supported by the estimates, and how much do the underlying cohort effects vary?

The analysis below assumes each cohort reports an estimate of the same type of effect—for example, a log odds ratio for an adverse drug reaction per specified genotype contrast—and its standard error. Effects and outcomes must be oriented consistently: an estimate above zero should represent the same direction of association in every cohort. Estimates with different units or incompatible phenotype definitions cannot be made comparable simply by averaging them.

Common-Effect Pooling and Cochran’s Q

A common-effect model assumes every cohort shares one true effect, \(\theta\), and that differences among estimates arise from sampling error. Assuming independent, approximately normally distributed estimates with valid standard errors, the inverse-variance weight for cohort \(i\) is \(w_i=1/s_i^2\). The pooled estimate is the weighted mean, and its standard error follows from the sum of the weights.

Cochran’s \(Q\) measures weighted dispersion around that pooled estimate. Under the common-effect model, independent normal estimates, and correctly specified standard errors, \(Q\) approximately follows a chi-square distribution with \(k-1\) degrees of freedom. Its upper-tail p-value assesses whether the observed dispersion is unusually large under those assumptions. A non-small p-value does not establish that all true effects are identical; with few or imprecise cohorts, the test may have limited power.

The degrees of freedom arise because the \(k\) weighted deviations are constrained by the fitted weighted mean: their weighted sum is zero. In the normal common-effect model, the residual variation after estimating one mean therefore has \(k-1\) independent components, giving the chi-square reference distribution for \(Q\).

Quantifying Dispersion With I²

The statistic \(I^2\) expresses the fraction of the observed weighted dispersion that is estimated to exceed sampling-error variation. It is commonly reported as a percentage. It is a descriptive measure, not the percentage of patients, genes, or causal effect explained by a particular moderator. It can be imprecise, and its value depends partly on the precision of the cohort estimates.

In the preceding example, \(I^2=(6.142-3)/6.142\times100\%=51.2\%\). This indicates estimated excess dispersion relative to sampling error, despite a non-small \(Q\)-test p-value. The two summaries answer different questions: the p-value tests a model-based null, whereas \(I^2\) describes the estimated proportion of dispersion beyond sampling error. Neither identifies why cohorts differ.

Random-Effects Pooling

A random-effects model allows cohort-specific true effects to vary around a mean effect. Write \(\theta_i=\mu+u_i\), where \(u_i\) has mean zero and variance \(\tau^2\); the sampling model is \(y_i\mid\theta_i\) with standard error \(s_i\). The model assumes independent cohorts and, in its usual form, normally distributed between-cohort effects. Here \(\mu\) is the mean of the distribution of cohort effects, and \(\tau^2\) is their between-cohort variance.

Because the true effects vary, random-effects weights include both sampling variance and estimated between-cohort variance. Consequently, less precise cohorts often receive relatively more weight than under common-effect pooling. The pooled estimate targets the mean of the modeled distribution of cohort effects, not a single effect guaranteed to apply to every setting.

One conventional method-of-moments estimator is the DerSimonian–Laird estimator. It estimates excess dispersion from \(Q\), truncating at zero because a variance cannot be negative. Define \(C=\sum_iw_i-\sum_iw_i^2/\sum_iw_i\).

This is one estimator, not a universal requirement: other approaches, including restricted maximum likelihood, are also used. The random-effects confidence interval based on the displayed standard error is a normal-approximation interval for the mean effect and treats the estimated variance in a simplified way. With few cohorts, uncertainty in \(\hat{\tau}^2\) can make such intervals too narrow; small-sample adjustments such as Hartung–Knapp methods may be appropriate. A confidence interval for the mean effect is also not a prediction interval for the true effect in a new cohort.

Investigating Dose and Other Cohort Differences

Heterogeneity is a diagnostic signal, not an explanation. Before fitting moderators, inspect whether the cohort estimates actually refer to comparable effects. Pharmacogenomic differences that deserve attention include:

• Dose and regimen: cohorts may differ in starting dose, titration, duration, adherence, or whether dose was genotype-guided. A genotype association with response can change across exposure levels.
• Phenotype definition: response thresholds, follow-up windows, adverse-event adjudication, and handling of competing events may differ. A harmonized label does not guarantee identical measurement.
• Ancestry: allele frequencies, linkage disequilibrium, environmental context, and care patterns can differ across ancestry groups. A cohort-level ancestry label alone may not capture individual-level effect modification.
• Prescribing and clinical context: co-medications, indication, dose selection, monitoring, and treatment changes may affect both exposure and observed outcome. In observational data, confounding by indication and related prescribing processes also matter.

Useful checks include reviewing cohort protocols and effect coding, tabulating estimates by clinically meaningful features, and repeating analyses after a defensible harmonization of genotype contrast and outcome. Subgroup analyses or meta-regression can describe whether an observed cohort characteristic tracks with effect estimates. They do not by themselves show that the characteristic caused the differences: moderators are often correlated, measured only at cohort level, and represented by few cohorts.

For a continuous cohort characteristic \(x_i\), such as a standardized summary of dose intensity, a meta-regression models the cohort estimates as

Here \(\beta_1\) describes the change in the mean cohort effect per unit of \(x_i\), conditional on the model. This is an ecological association across cohorts, not an individual-level dose-response estimate. In particular, a positive slope cannot establish that increasing an individual patient’s dose would change that patient’s genotype effect.

Common Mistakes and Practical Interpretation

• Treating a non-significant Q test as proof of homogeneity. The test can have low power when there are few cohorts. Report the estimates, their precision, and a heterogeneity summary rather than declaring effects identical.
• Reading I² as a causal percentage. An \(I^2\) of 50% does not mean that a measured factor caused half the variation. It summarizes excess dispersion relative to sampling error and does not identify its source.
• Switching to random effects as a cure for incompatibility. Random-effects weights accommodate variation in underlying effects; they do not repair inconsistent outcome definitions, effect directions, or genotype contrasts.
• Interpreting meta-regression causally. A cohort-level relationship may be confounded by other cohort characteristics and cannot be assumed to describe individual patients.
• Reporting only a pooled result. Include cohort estimates and standard errors, the model used, the heterogeneity method and estimate, and a contextual interpretation. If effects vary substantially, the pooled mean may not summarize any particular clinical setting well.

The choice between common-effect and random-effects pooling should follow the scientific question and assumptions, not whichever model produces a preferred p-value. When heterogeneity is present, investigate plausible differences in dose, phenotype, ancestry, and prescribing, and state what the available cohort-level data can and cannot distinguish.