Cochran’s Q for MR Heterogeneity

Why Test for Heterogeneity in Mendelian Randomization?

With several genetic variants as instruments, each variant can produce a ratio estimate of the exposure’s effect on the outcome. If the variants are valid instruments and a common causal effect is a reasonable description, these estimates should differ mainly because of sampling error. In practice, they may be more dispersed than that explanation predicts. Cochran’s \(Q\) summarizes this dispersion around the inverse-variance weighted (IVW) estimate.

A large \(Q\) can flag disagreement among variant-specific estimates, but it does not identify why they disagree. Horizontal pleiotropy is one possible explanation, not the only one. The statistic is therefore a diagnostic of excess heterogeneity under a specified model, rather than a test that proves or rules out instrument validity.

Ratio Estimates and the Q Statistic

For variant \(j\), let \(\hat\beta_{Xj}\) be its estimated association with the exposure and \(\hat\beta_{Yj}\) its estimated association with the outcome, with alleles harmonized so both associations refer to the same effect allele. The variant-specific ratio estimate divides the outcome association by the exposure association.

Definition: For \(K\) genetic variants with nonzero \(\hat\beta_{Xj}\), the ratio estimate for variant \(j\) is \(\hat\theta_j=\hat\beta_{Yj}/\hat\beta_{Xj}\). Under the usual first-order approximation, which treats the exposure association estimate as sufficiently precise, its standard error is \(SE(\hat\theta_j)\approx SE(\hat\beta_{Yj})/|\hat\beta_{Xj}|\). The IVW estimate is the inverse-variance weighted mean of the ratio estimates, and Cochran’s \(Q\) is their weighted sum of squared deviations from that mean.

Writing \(w_j=1/SE(\hat\theta_j)^2\), the IVW estimate and Q statistic are

\hat\theta_{\mathrm{IVW}} = \frac{\sum_{j=1}^{K} w_j\hat\theta_j}{\sum_{j=1}^{K}w_j}, \qquad Q = \sum_{j=1}^{K}w_j(\hat\theta_j-\hat\theta_{\mathrm{IVW}})^2.

Under the first-order approximation, \(w_j\approx\hat\beta_{Xj}^2/SE(\hat\beta_{Yj})^2\). Substitution shows that the ratio-scale expression is equivalent to the familiar residual form of IVW Q:

Q \approx \sum_{j=1}^{K} \frac{(\hat\beta_{Yj}-\hat\theta_{\mathrm{IVW}}\hat\beta_{Xj})^2} {SE(\hat\beta_{Yj})^2}.

The ratio form makes the interpretation direct: Q is large when weighted squared deviations of the variant estimates from their common weighted mean are large. The residual form emphasizes that the IVW model fits a line through the origin to the outcome associations as a function of the exposure associations.

Reference Distribution and Assumptions

The usual reference test compares the observed statistic with a chi-square distribution with \(K-1\) degrees of freedom. One degree of freedom is used to estimate the common IVW effect from the \(K\) variant estimates. Under the null model, Q is approximately distributed as \(\chi^2_{K-1}\).

Model and assumptions: The conventional Q reference test assumes a common underlying causal slope across the variants, independent variants (or otherwise independent ratio-estimation errors), approximately normal association-estimation errors, and sufficiently precise exposure associations for the first-order standard errors to be adequate. Under the null of no excess dispersion around the common-effect IVW model, \(Q\) is approximately \(\chi^2_{K-1}\). The test is approximate, particularly when instruments are weak or the assumptions about dependence and standard errors do not hold.

A short projection argument explains both the statistic and its degrees of freedom. Suppose the true outcome association for each variant follows a common slope \(\theta\), and let the standardized outcome-association errors be independent standard normal variables. Fitting the slope removes one direction of variation from the \(K\)-dimensional error vector. The weighted residuals are the projection of that vector onto the remaining \(K-1\) dimensional subspace. Their squared length is Q; the squared length of a standard normal projection onto a subspace of rank \(K-1\) has a chi-square distribution with \(K-1\) degrees of freedom. This yields the reference distribution when the stated approximation is reasonable.

The null here is not “there is no causal effect.” It concerns dispersion around a common-effect model. The IVW estimate may be nonzero whether or not Q is large. Conversely, a small Q does not establish that the common estimate is unbiased: a pattern of pleiotropy that shifts all variants similarly may leave little residual dispersion.

Worked Calculations

Worked Example: Three Similar Estimates and a Nonsignificant Q

Suppose three independent, harmonized variants have the following hypothetical summary associations. The exposure associations are treated as precise enough for the first-order standard errors.

Variant	\(\hat\beta_{Xj}\)	\(\hat\beta_{Yj}\)	\(SE(\hat\beta_{Yj})\)	\(\hat\theta_j\)	\(SE(\hat\theta_j)\)
1	0.10	0.020	0.010	0.20	0.10
2	0.20	0.060	0.020	0.30	0.10
3	0.15	0.075	0.015	0.50	0.10

For example, variant 1 has ratio estimate \(0.020/0.10=0.20\) and first-order standard error \(0.010/0.10=0.10\). Each ratio estimate has weight \(w_j=1/0.10^2=100\), so

\hat\theta_{\mathrm{IVW}} = \frac{100(0.20)+100(0.30)+100(0.50)}{300} = 0.3333.

The heterogeneity statistic is

Q = 100(0.20-0.3333)^2+ 100(0.30-0.3333)^2+ 100(0.50-0.3333)^2 \approx 4.667.

There are \(K-1=2\) degrees of freedom. For a chi-square variable with 2 degrees of freedom, the upper-tail probability is \(P(\chi^2_2\ge q)=\exp(-q/2)\). Thus \(p=\exp(-4.667/2)\approx0.0970\). At a 0.05 significance level, this is not statistically significant evidence of excess heterogeneity. It does not prove that the instruments are valid or that their effects are identical; it says this amount of dispersion is not unusual under the stated reference model.

Worked Example: More Dispersion and a Small Q-Test P-Value

Consider another hypothetical set of three independent variants. The exposure associations are \(0.10\), \(0.20\), and \(0.15\); the outcome associations are \(0.010\), \(0.060\), and \(0.075\); and their outcome standard errors are \(0.005\), \(0.010\), and \(0.0075\), respectively. The first-order ratio estimates are \(0.10\), \(0.30\), and \(0.50\), each with standard error \(0.05\). These values use the same first-order approximation as above.

Each weight is \(1/0.05^2=400\), giving

\hat\theta_{\mathrm{IVW}} = \frac{400(0.10)+400(0.30)+400(0.50)}{1200} = 0.30.

The weighted squared deviations are

$$ Q = 400(0.10-0.30)^2+ 400(0.30-0.30)^2+ 400(0.50-0.30)^2 = 16+0+16 = 32. $$

With 2 degrees of freedom, \(p=\exp(-32/2)=\exp(-16)\approx1.13\times10^{-7}\). This is strong evidence against the common-effect model’s expected level of dispersion, given the assumptions used for the test. It is not, by itself, evidence that a particular variant is pleiotropic, nor does it distinguish pleiotropy from other sources of heterogeneity.

Worked Example: Unequal Precision Changes the Weighted Mean

Suppose three hypothetical ratio estimates are \(0.10\), \(0.40\), and \(0.30\), with standard errors \(0.10\), \(0.20\), and \(0.10\). The corresponding weights are \(100\), \(25\), and \(100\). The middle estimate receives less weight because it is less precise.

The weighted mean is

\hat\theta_{\mathrm{IVW}} = \frac{100(0.10)+25(0.40)+100(0.30)}{100+25+100} = \frac{50}{225} = 0.2222.

Using this mean, the Q statistic is

\begin{aligned} Q &= 100(0.10-0.2222)^2+ 25(0.40-0.2222)^2+ 100(0.30-0.2222)^2\\ &\approx 1.4938+0.7901+0.6049 =2.8889. \end{aligned}

With \(K-1=2\) degrees of freedom, \(p=\exp(-2.8889/2)\approx0.236\). The statistic is dimensionless because each squared deviation is divided by its variance through the weight. The estimates themselves retain the units of the outcome per unit of exposure.

What Heterogeneity Can and Cannot Tell You

A small Q-test p-value indicates that the ratio estimates are more dispersed around the IVW estimate than expected under the common-effect model and its assumptions. This may be compatible with horizontal pleiotropy, in which a variant affects the outcome through a pathway other than the exposure. But Q does not name a pathway, identify which variant is responsible, or establish that pleiotropy is the cause.

Other explanations include chance variation, a genuinely different causal effect across the mechanisms represented by the variants, imprecise association estimates, or violations of the test’s assumptions. For example, correlated variants can make the independent-error reference distribution inappropriate. Weak exposure associations can also make first-order ratio standard errors and the usual calibration unreliable. The interpretation should therefore be tied to the data structure and estimation assumptions, not just to whether the p-value crosses a threshold.

Q can also have limited power when there are few variants or when individual ratio estimates are imprecise. More importantly, heterogeneity is not required for invalid instruments. If every variant’s direct effect shifts its outcome association in the same proportional way, all ratio estimates could remain similar even though the exclusion restriction is violated. Thus, a nonsignificant Q is not a certification of validity.

Common Pitfall

Wrong reasoning: “The Q-test p-value is below 0.05, so the test proves horizontal pleiotropy and tells us the IVW estimate is wrong.”

Correction: A small p-value is evidence against the assumed common-effect model’s predicted dispersion, conditional on the test’s assumptions. Pleiotropy is one possible explanation, but Q does not identify a cause or determine the direction or size of bias in the IVW estimate. Conversely, a large p-value does not prove the absence of pleiotropy. Report Q, its degrees of freedom, and the p-value, then describe the result as evidence for or against excess heterogeneity rather than as a definitive validity verdict.

Key takeaway: Cochran’s Q is the inverse-variance weighted sum of squared deviations of variant-specific MR estimates from their IVW mean. Its usual reference test uses \(K-1\) degrees of freedom under a common-effect, independent-error approximation. It diagnoses excess dispersion, not its source.

Check Your Understanding

Answer each question by stating the relevant assumption or calculation, not only a conclusion.

Why does the usual Q test have \(K-1\), rather than \(K\), degrees of freedom?
If three independent ratio estimates are \(0.2\), \(0.3\), and \(0.5\), each with standard error \(0.1\), calculate their IVW mean and Q statistic.
What does a small Q-test p-value say about the common-effect model, and what does it not establish about pleiotropy?
Why can a nonsignificant Q test not prove that all instruments satisfy the exclusion restriction?
Name two conditions under which the usual chi-square reference test may be poorly calibrated.