Interpreting I-Squared in Genetic Meta-Analysis

What I-Squared Summarizes

A genetic meta-analysis combines effect estimates from multiple cohorts. Cochran’s Q summarizes how far those estimates lie from their inverse-variance-weighted mean, relative to their estimated sampling errors. I-squared, often written $I^2$, is a related summary: it compares the excess in Q above its expected value under a common-effect model with the observed Q.

This comparison gives a relative measure of heterogeneity. It does not tell us how many cohorts have different effects, how large the differences are in the outcome’s units, or whether any particular cohort is an outlier. To interpret it, examine the cohort estimates and their standard errors as well as the I-squared value.

Definition: For $k$ independent cohort estimates, let $Q$ be Cochran’s heterogeneity statistic and let $df=k-1$. The conventional I-squared estimate is $$ I^2 = \max\left(0,\frac{Q-df}{Q}\right)\times 100\%. $$ If $Q=0$, define $I^2=0\%$. I-squared is reported as a percentage.

The Setting and the Calculation

Suppose cohort $i$ provides an effect estimate $\hat{\beta}_i$ with standard error $SE_i$, for $i=1,\ldots,k$. Before calculating Q, the estimates must refer to the same effect allele and be on the same phenotype scale. Assume, for this standard calculation, that the cohort estimates are independent, their standard errors are appropriate, and their sampling distributions are approximately normal. Under the common-effect model used to assess heterogeneity, each cohort estimates the same underlying effect.

The inverse-variance fixed-effect mean and Q are

w_i=\frac{1}{SE_i^2},\qquad \hat{\beta}_{FE}=\frac{\sum_{i=1}^{k}w_i\hat{\beta}_i}{\sum_{i=1}^{k}w_i},\qquad Q=\sum_{i=1}^{k}w_i(\hat{\beta}_i-\hat{\beta}_{FE})^2.

The weights $w_i$ give greater influence to more precise estimates. With $k$ independent cohorts, Q has $k-1$ degrees of freedom. If all cohorts share one true effect and the normal approximation holds, Q is approximately distributed as a chi-squared random variable with $k-1$ degrees of freedom. Under that model its expected value is $df$. I-squared uses this expected value as a reference for how much of the observed Q exceeds what sampling variation alone would typically produce.

Interpretation: I-squared is a conventional estimate of the percentage of observed variation in effect estimates, as summarized by Q, that is beyond the amount expected from sampling error under the common-effect model. It is a relative, precision-dependent summary, not a direct measurement of the magnitude or biological importance of effect differences.

Worked Example: Modest Differences With Limited Precision

Suppose four hypothetical cohorts estimate the same harmonized genetic effect, measured in standard-deviation units of the phenotype per effect allele. Their estimates are $0.10$, $0.11$, $0.09$, and $0.12$, each with $SE=0.05$. Assume independent cohorts and appropriate standard errors.

Calculate the weights and fixed-effect mean. Each weight is $1/0.05^2=400$. Because the weights are equal, the fixed-effect mean is the ordinary average:

\hat{\beta}_{FE}=\frac{0.10+0.11+0.09+0.12}{4}=0.105.

Calculate Q. The deviations from the mean are $-0.005$, $0.005$, $-0.015$, and $0.015$. Thus,

Q=400\left[(-0.005)^2+(0.005)^2+(-0.015)^2+(0.015)^2\right] =400(0.0005)=0.20.

There are $df=4-1=3$ degrees of freedom. Since $Q<df$, the excess $Q-df$ is negative, and the conventional estimate is truncated at zero:

I^2=\max\left(0,\frac{0.20-3}{0.20}\right)\times100\%=0\%.

The reported I-squared is $0\%$. This does not establish that the four underlying effects are exactly identical. It says that this Q does not exceed its common-effect reference value. The observed differences are small relative to the standard errors in this example.

Why Precision Changes the Interpretation

I-squared is not a property of the spread of the cohort estimates alone. It also depends on how precisely those estimates are measured. With more precise estimates, a given discrepancy in effect units contributes more to Q because its inverse-variance weight is larger. With less precise estimates, the same discrepancy contributes less. Therefore, two meta-analyses with similarly sized differences between cohort estimates can have different I-squared values.

Worked Example: The Same Estimates With Greater Precision

Now suppose the four hypothetical cohorts report the same estimates as in the previous example: $0.10$, $0.11$, $0.09$, and $0.12$ standard-deviation units per effect allele. Their standard errors are now $0.01$, rather than $0.05$. Continue to assume independent cohorts and correctly estimated standard errors.

The estimates have the same mean, $0.105$, and the same sum of squared deviations, $0.0005$. Each weight is now $1/0.01^2=10{,}000$, so

Q=10{,}000(0.0005)=5.00,\qquad df=4-1=3.

The resulting I-squared is

I^2=\frac{5.00-3}{5.00}\times100\%=40.0\%.

The observed effect estimates are unchanged, but their smaller standard errors make their differences more notable relative to sampling error. I-squared rises from $0\%$ in the previous example to $40.0\%$. Neither value, by itself, describes the absolute size of the differences; that remains visible in the effect estimates, which span $0.03$ standard-deviation units per allele.

A Short Justification for the Formula

Under the common-effect model, Q has $df=k-1$ degrees of freedom and expected value $df$. The amount by which the observed Q exceeds this reference is $Q-df$. Dividing that excess by Q gives a relative excess. This motivates the conventional expression when $Q>df$:

\frac{\text{excess above the reference}}{\text{observed }Q} =\frac{Q-df}{Q} =1-\frac{df}{Q}.

The ratio is multiplied by $100\%$ to report a percentage. If $Q\le df$, the ratio is zero or negative. A negative percentage would not be a meaningful report of heterogeneity, so the conventional estimator uses zero instead. This truncation is a rule for reporting the estimate; it is not proof that all cohorts have identical true effects.

Reading a Larger I-Squared in Context

A larger I-squared indicates that Q is large relative to its degrees-of-freedom reference. It suggests that differences among the observed estimates are more substantial relative to their sampling errors than would be expected under the common-effect model. It does not by itself identify the source of those differences. A larger value also does not necessarily imply a large difference in the outcome’s units: very precise cohorts can make modest absolute differences stand out.

Consider the following hypothetical five-cohort analysis. Suppose the harmonized effects are measured in standard-deviation units per effect allele, all standard errors are $0.10$, and the estimates are $0.08$, $0.20$, $0.30$, $0.40$, and $0.52$. The equal weights are $100$, and the fixed-effect mean is $0.30$. The cohort estimates deviate from that mean by $-0.22$, $-0.10$, $0$, $0.10$, and $0.22$. These data meet the stated independence, scale, and standard-error assumptions for the illustrative calculation.

Worked Example: Interpreting a Higher I-Squared

Using the five hypothetical cohort estimates above, calculate Q from the weighted squared deviations:

Q=100\left[(-0.22)^2+(-0.10)^2+0^2+(0.10)^2+(0.22)^2\right] =100(0.1168)=11.68.

Here $df=5-1=4$, so

I^2=\frac{11.68-4}{11.68}\times100\% =\frac{7.68}{11.68}\times100\% \approx65.8\%.

The I-squared is approximately $65.8\%$. The estimates range from $0.08$ to $0.52$ standard-deviation units per allele, and the cohort standard errors are $0.10$. The value summarizes the observed dispersion relative to sampling error; it does not mean that $65.8\%$ of these five cohorts have a different effect. There is no count of cohorts in the formula. The pattern and scale of the estimates must be inspected directly.

What I-Squared Does Not Say

The percentage is not a literal partition of the biological effects into “heterogeneous” and “non-heterogeneous” portions. It is a function of Q and its degrees of freedom, and therefore depends on the observed estimates, their standard errors, and the number of cohorts. In particular, it is not the proportion of cohorts whose true effects differ from the others. A single cohort with an unusual estimate can contribute substantially to Q, while several differing effects may not produce a large I-squared if the estimates are imprecise.

A value of zero should be read as “no positive excess over the reference after truncation,” not “no heterogeneity.” Conversely, a higher value does not quantify the absolute spread in effect units, establish a biological explanation, or show that a pooled estimate is clinically important. Always report or inspect cohort-level effect estimates and standard errors alongside I-squared.

I-squared is also not a substitute for assessing whether the calculation’s assumptions are credible. Misaligned alleles or incompatible phenotype scales make the estimates unsuitable for this comparison. Dependence between cohort estimates, such as from overlapping participants, also violates the independence assumption behind the standard Q reference distribution. In such settings, the usual interpretation of I-squared may not apply without an analysis that accounts for the dependence.

Common Mistakes

Wrong: “An I-squared of $40\%$ means $40\%$ of cohorts have different effects.” Correction: I-squared is calculated from Q and $df$; it does not count cohorts or classify their true effects.
Wrong: “An I-squared of $0\%$ proves there is no heterogeneity.” Correction: It means the conventional estimate has no positive excess above the reference. Differences may remain, and the analysis may have limited precision.
Wrong: “The same effect-estimate spread must give the same I-squared.” Correction: Q weights discrepancies by inverse variance. The standard errors affect I-squared even when the estimates are unchanged.
Wrong: “A high I-squared means the effects differ by a large amount.” Correction: It is relative to sampling error. Inspect the effect estimates and their units to assess the absolute differences.

Key takeaway: Calculate I-squared as the nonnegative excess of Q over $df$, divided by Q. Interpret it as a relative summary of heterogeneity in the context of cohort precision and the actual effect estimates—not as a percentage of cohorts or effects that differ.

Check Your Understanding

Use the definitions and examples above to answer these questions.

A meta-analysis has $Q=8$ and $df=5$. Calculate I-squared, showing the substitution.
What does an I-squared of $0\%$ mean, and what does it not establish?
Why can the same set of effect estimates produce different I-squared values if their standard errors change?
Explain why an I-squared of $60\%$ does not mean that $60\%$ of cohorts have different true effects.
Name two assumptions needed for the usual interpretation of Q and I-squared in a cohort meta-analysis.