Deriving Cochran’s Q for Effect Heterogeneity

Why Measure Effect Heterogeneity?

A fixed-effect meta-analysis combines estimates from multiple cohorts by weighting each estimate according to its precision. The pooled estimate summarizes the evidence under a common-effect model, in which all cohorts estimate the same underlying genetic effect. Cochran’s Q measures how far the cohort estimates lie from that pooled value, taking their precision into account.

This is useful because a pooled association can conceal differences among cohorts. Yet a raw range of cohort estimates is not enough to establish that their effects differ: noisy estimates can vary considerably by chance, while small differences between precise estimates may be informative. Q compares the weighted variation actually observed with the variation expected from sampling error under a common effect.

Definition: Suppose $k$ independent cohorts provide estimates $\hat{\beta}_i$ of the same genetic effect, with standard errors $SE_i$, for $i=1,\ldots,k$. Define the inverse-variance weight $w_i=1/SE_i^2$ and the fixed-effect pooled estimate $$ \hat{\beta}_{FE}=\frac{\sum_{i=1}^{k} w_i\hat{\beta}_i}{\sum_{i=1}^{k}w_i}. $$ Cochran’s $Q$ statistic is the weighted sum of squared deviations from that estimate: $$ Q=\sum_{i=1}^{k}w_i(\hat{\beta}_i-\hat{\beta}_{FE})^2. $$

The effect estimates must refer to the same variant, effect allele, phenotype, and effect scale for these deviations to be meaningful. For example, each $\hat{\beta}_i$ might be a log odds ratio per copy of the same effect allele. The standard error must describe the sampling uncertainty of its corresponding estimate. Only cohorts with valid estimates for the variant contribute to $k$; a missing estimate is not a zero effect. If only one cohort contributes, $Q=0$ by construction and there is no between-cohort heterogeneity test to perform.

Deriving the Null Distribution

The heterogeneity null hypothesis is that all cohorts share a common underlying effect. It does not require that the common effect equal zero. To derive the reference distribution, first suppose the estimates are independent and normally distributed with known sampling variances: $$ \hat{\beta}_i \sim N(\beta, SE_i^2), \qquad i=1,\ldots,k. $$ Here $\beta$ is the shared effect under the null. The inverse-variance pooled estimate is the weighted mean of these estimates.

Standardize each cohort estimate by its standard error. The resulting vector has independent standard normal components under the null. The statistic $Q$ is the squared length of the part of this standardized vector that remains after removing its component in the direction corresponding to the weighted pooled mean. The following derivation makes that projection, and its degrees of freedom, explicit.

Standardize the cohort estimates.
Let $Z_i=(\hat{\beta}_i-\beta)/SE_i$. Under the stated null model, the $Z_i$ are independent standard normal variables. Write their vector as $Z$.

Express the pooled residuals in standardized coordinates.
Let $a_i=\sqrt{w_i}/\sqrt{\sum_j w_j}$, so that $\sum_i a_i^2=1$. The weighted pooled estimate removes the component of $Z$ in the unit direction $a$. Consequently, $Q=Z^\mathsf{T}(I-aa^\mathsf{T})Z$, where $I$ is the $k$-dimensional identity matrix.

Count the remaining independent directions.
The matrix $I-aa^\mathsf{T}$ is a projection: it removes one direction from a $k$-dimensional vector and has rank $k-1$. An orthogonal change of coordinates preserves independent standard normal components. Thus $Q$ is the sum of squares of $k-1$ independent standard normal variables.

Theorem: Under the independent normal common-effect model with known variances, $Q\sim\chi^2_{k-1}$. In cohort-level genetic association analyses, standard errors are estimated and effect estimates are often only approximately normal, so this chi-square reference distribution is generally an approximation. The test uses $df=k-1$, where $k$ is the number of cohorts contributing estimates for the variant.

The rank is $k-1$, rather than $k$, because the cohort effects are compared with a pooled value estimated from those same cohorts. Estimating the common pooled effect uses one degree of freedom. As a further check, direct expansion gives a convenient equivalent calculation:

Q=\sum_{i=1}^{k}w_i\hat{\beta}_i^2- \frac{\left(\sum_{i=1}^{k}w_i\hat{\beta}_i\right)^2}{\sum_{i=1}^{k}w_i}.

To verify this identity, expand the definition of $Q$, then substitute the weighted mean: $$ \sum_i w_i(\hat{\beta}_i-\hat{\beta}_{FE})^2 =\sum_i w_i\hat{\beta}_i^2-2\hat{\beta}_{FE}\sum_iw_i\hat{\beta}_i +\hat{\beta}_{FE}^2\sum_iw_i =\sum_i w_i\hat{\beta}_i^2-\frac{(\sum_iw_i\hat{\beta}_i)^2}{\sum_iw_i}. $$ The last expression can be useful for checking arithmetic, but the sum of squared deviations is usually easier to interpret.

Calculating Q From Cohort Results

Worked Example: Three Cohorts With Similar-Precision Estimates

Hypothetical data: Three independent cohorts estimate the same allele effect on a common scale. Their estimates are $0.10$, $0.20$, and $0.30$, with standard errors $0.10$, $0.10$, and $0.20$, respectively. Calculate $Q$, its degrees of freedom, and its p-value.

The weights are $100$, $100$, and $25$. Therefore, $$ \hat{\beta}_{FE} =\frac{100(0.10)+100(0.20)+25(0.30)}{100+100+25} =\frac{37.5}{225}=0.1667. $$

The weighted squared deviations are $$ Q=100(0.10-0.1667)^2+100(0.20-0.1667)^2 +25(0.30-0.1667)^2 \approx 0.4444+0.1111+0.4444=1.000. $$

There are $k=3$ contributing cohorts, so $df=3-1=2$. The upper-tail probability is $P(\chi^2_2\geq1.000)\approx0.6065$. Under the test assumptions, this is not unusual if the cohorts share a common effect. The result does not prove their effects are identical; it indicates that these data provide little evidence against the common-effect null.

The p-value is the probability, assuming the common-effect null model and its sampling assumptions, of obtaining a $Q$ at least as large as the observed value. It is an upper-tail test because larger deviations from the pooled estimate count as stronger evidence against a common effect. A small p-value indicates that the observed spread is difficult to explain by sampling error alone under that model. It does not identify which cohort differs, explain why, or establish that the pooled genetic effect is nonzero.

Worked Example: A Small Heterogeneity P-Value

Hypothetical data: Three independent cohorts report effects of $0.05$, $0.25$, and $-0.05$, each with standard error $0.05$, on the same genetic effect scale. Calculate and interpret $Q$.

Each weight is $1/0.05^2=400$. The fixed-effect estimate is $$ \hat{\beta}_{FE} =\frac{400(0.05)+400(0.25)+400(-0.05)}{1200} =\frac{100}{1200}=0.08333. $$

Thus, $$ Q=400\left[(0.05-0.08333)^2+(0.25-0.08333)^2+(-0.05-0.08333)^2\right] \approx400(0.04667)=18.67. $$ With $df=2$, the chi-square upper-tail p-value is approximately $0.0000886$. Under the common-effect model and the stated assumptions, such a large weighted dispersion would be unusual. These results provide evidence of effect heterogeneity across the cohorts; they do not by themselves determine its source or show that any particular cohort estimate is the cause.

Worked Example: Unequal Standard Errors and the Shortcut Formula

Hypothetical data: Three independent cohorts estimate a log odds ratio per copy of the same effect allele. The estimates and standard errors are $0.12$ and $0.06$, $0.20$ and $0.10$, and $-0.02$ and $0.04$. Use the shortcut formula to calculate $Q$.

The weights are $277.78$, $100$, and $625$, respectively. Their sum is $1002.78$, and the weighted sum of effects is $$ 277.78(0.12)+100(0.20)+625(-0.02) \approx 33.33+20.00-12.50=40.83. $$ Therefore $\hat{\beta}_{FE}\approx40.83/1002.78=0.04072$. For the shortcut formula, the weighted sum of squared estimates is $$ 277.78(0.12^2)+100(0.20^2)+625((-0.02)^2) \approx4.00+4.00+0.25=8.25. $$

Substitution gives $$ Q\approx8.25-\frac{40.83^2}{1002.78}\approx8.25-1.663=6.587. $$ There are two degrees of freedom. The upper-tail p-value is approximately $0.0371$, which is evidence against a common effect at the 0.05 level under the test assumptions. The effects and their standard errors are both on the log odds ratio scale; the Q statistic itself is dimensionless because each squared deviation is multiplied by an inverse variance.

Assumptions and Interpretation

The chi-square calibration depends on more than having several effect estimates. The cohorts should be independent, the estimates should target the same effect, and their standard errors should adequately represent their sampling uncertainty. In genetic association work, the common-effect null and approximate normality of cohort estimates are modeling assumptions. If cohorts overlap, ordinary independence does not hold and the usual reference distribution may not be appropriate. Differences in allele alignment, phenotype definition, or effect scale can also create apparent disagreement that is not the biological or statistical heterogeneity the test is intended to assess.

Q is sensitive to both the size of cohort differences and their precision. A fixed difference between two estimates contributes more when their standard errors are smaller. Conversely, substantial differences among imprecise estimates may not yield a large Q. The test can also have limited power when few cohorts contribute. A non-small p-value therefore should not be read as proof of homogeneity.

Common Mistakes

Testing the wrong null. A small Q p-value is evidence against equal cohort effects, not evidence that the pooled effect differs from zero. The heterogeneity null permits a shared nonzero effect.
Using unweighted deviations. Summing $(\hat{\beta}_i-\hat{\beta}_{FE})^2$ without weights ignores differences in precision. Q weights each squared deviation by $1/SE_i^2$.
Using the wrong degrees of freedom. For $k$ contributing cohorts, use $k-1$, because the common pooled estimate is fitted from those cohorts.
Treating a missing cohort estimate as zero. A cohort with no usable estimate does not contribute to the calculation. Use the number of cohorts with valid, comparable estimates.
Reading a large p-value as proof of equal effects. It means the observed Q is not unusually large under the null model; limited precision or few cohorts can make heterogeneity difficult to detect.

Key takeaway: Cochran’s $Q=\sum_i w_i(\hat{\beta}_i-\hat{\beta}_{FE})^2$ measures precision-weighted dispersion around the fixed-effect estimate. Under a common-effect model and suitable sampling assumptions, it is compared with a chi-square distribution with $k-1$ degrees of freedom. Its p-value addresses heterogeneity, not whether the pooled genetic association is nonzero.

Check Your Understanding

Use the definitions and assumptions above to answer these questions.

Why does the null reference distribution for $Q$ have $k-1$, rather than $k$, degrees of freedom?
Two cohorts estimate the same nonzero genetic effect with high precision. Is that nonzero shared effect, by itself, evidence of heterogeneity? Explain.
If a variant has valid estimates in four cohorts and is missing in a fifth, what degrees of freedom are used for the ordinary Q test?
What does a small upper-tail Q p-value indicate, and what does it not establish?
Why can two cohort estimates with the same absolute difference make different contributions to Q when their standard errors differ?