What MR Diagnostics Can And Cannot Establish

Why MR Diagnostics Need Careful Interpretation

Mendelian randomization (MR) uses genetic variants as instrumental variables to investigate the effect of an exposure on an outcome. Diagnostics can reveal weaknesses in the observed inputs: for example, a variant may be only weakly associated with the exposure, alleles may be misaligned across data sources, or variant-specific estimates may differ substantially. These checks are useful, but they do not by themselves establish that the variants satisfy all the assumptions needed for a causal interpretation.

The distinction matters because some features can be examined in the data while others concern how a genetic variant relates to unmeasured causes or to pathways to the outcome. The first group can be assessed directly or indirectly using available summaries. The second group cannot generally be verified just by inspecting the same summary data used to estimate the MR effect.

Definition: In MR, an instrumental variable must be relevant to the exposure, independent of causes of the outcome that could confound the exposure–outcome relationship, and affect the outcome only through the exposure (the exclusion restriction). Diagnostics can provide evidence about relevance and data alignment, and can flag patterns that warrant investigation. They cannot, on their own, prove independence or the exclusion restriction.

Instrument-Strength Summaries

For variant \(j\), let \(\hat{\beta}_{Xj}\) be its estimated association with the exposure and \(SE(\hat{\beta}_{Xj})\) its standard error. When the association is approximately normal, the single-variant strength summary is often written

F_j \approx \left(\frac{\hat{\beta}_{Xj}}{SE(\hat{\beta}_{Xj})}\right)^2.

This is the squared association statistic for the variant–exposure estimate. A larger value indicates stronger statistical evidence that the variant is associated with the exposure, and is therefore relevant evidence for the relevance assumption. In a one-variant setting, this statistic is often used as an approximate first-stage F statistic. The exact first-stage F statistic in an individual-level analysis depends on the specified first-stage model and its degrees of freedom.

A threshold such as \(F_j > 10\) is sometimes used as a rough warning rule for weak-instrument concerns, not as a boundary between valid and invalid instruments. A variant above the threshold may still violate independence or exclusion. Conversely, a variant below it is not thereby shown to be invalid; its association may simply be estimated imprecisely. Strength summaries are also affected by how variants were selected and by the sample used to estimate their exposure associations. Selection on unusually large observed associations can make those estimates too large on average, the concern known as winner’s curse.

Worked Example: Interpreting a Single-Variant F Statistic

Suppose a hypothetical variant has an estimated association of \(0.080\) standard deviations of exposure per effect allele, with standard error \(0.016\) standard deviations per allele. The corresponding outcome association is \(0.012\) standard deviations per allele, with standard error \(0.006\).

The exposure-association statistic is \(0.080/0.016=5.00\), so

F_j \approx \left(\frac{0.080}{0.016}\right)^2 =5.00^2=25.0.

This is evidence that the variant is associated with the exposure in the data, and it is above the commonly used rough threshold of 10. It does not show that the variant is independent of confounders or that it affects the outcome only through the exposure. The strength calculation uses the exposure association; it cannot test those other conditions.

With multiple variants, do not treat a collection of SNP-level values as a complete validity assessment. A mean F statistic may summarize average strength, but an average can conceal particularly weak variants. Strength also does not guarantee that the variants are independent of one another, that their alleles were harmonized correctly, or that their outcome associations are free of pathways outside the exposure. The interpretation depends on the design and on what the reported statistic actually summarizes.

Allele Checks and Harmonization

Summary-data MR compares an exposure association and an outcome association for the same genetic contrast. The effect alleles must therefore refer to the same allele. If the exposure estimate is per copy of one allele and the outcome estimate is per copy of the other allele at a biallelic variant, one estimate must be re-expressed with the opposite sign before forming a ratio.

An allele check uses information such as the effect allele, other allele, allele frequency, and strand convention in each data source. These are observable features of the supplied data and can reveal an apparent mismatch. However, the check may be inconclusive when the available allele information is incomplete or ambiguous. In particular, A/T and C/G variants are palindromic: their alleles are complements of one another. For such a variant, allele labels alone may not tell whether the two data sources use the same orientation. Allele frequencies can help resolve the orientation when they are available and sufficiently informative; otherwise, the variant may need to be excluded rather than guessed.

Worked Example: Correcting an Opposite-Allele Outcome Estimate

Consider a hypothetical C/T variant. In the exposure data, the effect allele is T, and the estimated association is \(0.040\) exposure standard deviations per T allele. In the outcome data, the reported effect allele is C, and the estimated association is \(0.008\) outcome standard deviations per C allele. The reported T-allele frequencies are 0.32 in the exposure data and 0.31 in the outcome data.

The frequencies are close, which supports—but does not prove—that the same T allele has been identified in both data sources. Because C and T are opposite alleles at this variant, the outcome estimate per T allele is the negative of the estimate per C allele:

\hat{\beta}_{Y,T}=-0.008.

The harmonized ratio estimate is therefore

\hat{\theta}_T =\frac{\hat{\beta}_{Y,T}}{\hat{\beta}_{X,T}} =\frac{-0.008}{0.040} =-0.200.

This is an estimate of \(-0.200\) outcome standard deviations per one-standard-deviation increase in exposure, under the MR assumptions. If the outcome estimate were mistakenly used with its original C-allele sign, the ratio would be \(+0.008/0.040=+0.200\), reversing the direction. The frequency check is a useful alignment check, not evidence that the exclusion restriction holds.

Comparing Variant-Specific Estimates

For a harmonized variant \(j\), the Wald ratio is \(\hat{\theta}_j=\hat{\beta}_{Yj}/\hat{\beta}_{Xj}\), provided the exposure association is not zero. Its value and uncertainty can be inspected alongside those from other variants. Similar estimates can be reassuring about the internal pattern of the data; markedly different estimates may flag an issue such as a coding error, an unusual variant, or different variant-specific causal effects.

A comparison must account for uncertainty. Ratios based on small or imprecise exposure associations can be especially unstable. In a two-sample setting where exposure- and outcome-association estimates are independent, a first-order delta-method approximation to the ratio’s standard error is

SE(\hat{\theta}_j) \approx \sqrt{ \frac{SE(\hat{\beta}_{Yj})^2}{\hat{\beta}_{Xj}^{\,2}} + \frac{\hat{\beta}_{Yj}^{\,2}SE(\hat{\beta}_{Xj})^2}{\hat{\beta}_{Xj}^{\,4}} }.

This approximation uses the estimated exposure association in the denominator and treats the ratio as locally well behaved around the estimates. It can be unreliable when the exposure association is very weak or close to zero. If exposure and outcome estimates are not independent, the covariance term cannot simply be ignored.

A Short Justification for the Ratio Standard Error

Write \(g(a,b)=b/a\), where \(a=\hat{\beta}_{Xj}\) and \(b=\hat{\beta}_{Yj}\). The first-order delta method approximates the variance of a function of estimated quantities by applying its gradient to their covariance matrix. Here,

\frac{\partial g}{\partial a}=-\frac{b}{a^2}, \qquad \frac{\partial g}{\partial b}=\frac{1}{a}.

If the two estimates are independent, their covariance is zero. The approximate variance is therefore the sum of each estimate’s variance multiplied by the square of the corresponding derivative:

\operatorname{Var}(\hat{\theta}_j) \approx \frac{\hat{\beta}_{Yj}^{\,2}}{\hat{\beta}_{Xj}^{\,4}} SE(\hat{\beta}_{Xj})^2 + \frac{1}{\hat{\beta}_{Xj}^{\,2}} SE(\hat{\beta}_{Yj})^2.

Taking the square root gives the stated standard error. This calculation describes uncertainty under the approximation and independence condition; it does not diagnose whether the variant is a valid instrument.

Worked Example: Inspecting Three Variant-Specific Estimates

Suppose three hypothetical, harmonized variants have the following associations. The exposure and outcome are both measured in standard-deviation units, and the estimates come from non-overlapping samples.

Variant	\(\hat{\beta}_{Xj}\)	\(SE(\hat{\beta}_{Xj})\)	\(\hat{\beta}_{Yj}\)	\(SE(\hat{\beta}_{Yj})\)
1	0.100	0.010	0.0150	0.006
2	0.080	0.020	0.0100	0.005
3	0.050	0.010	0.0125	0.006

The ratio estimates are

\hat{\theta}_1=0.0150/0.100=0.150,\quad \hat{\theta}_2=0.0100/0.080=0.125,\quad \hat{\theta}_3=0.0125/0.050=0.250.

For variant 1, the approximate standard error is

SE(\hat{\theta}_1) \approx \sqrt{(0.006/0.100)^2+ (0.0150\times0.010/0.100^2)^2} =\sqrt{0.003600+0.000225} =0.06185.

Applying the same formula to variants 2 and 3 gives approximately \(0.06988\) and \(0.1300\), respectively. Approximate 95% intervals, using estimate \(\pm 1.96SE\), are \(0.0288\) to \(0.271\), \(-0.0120\) to \(0.262\), and \(-0.0048\) to \(0.505\). These intervals overlap substantially. The point estimates are not identical, but their uncertainty is also substantial, particularly for variant 3.

This inspection does not show a clear contradiction among the estimates, but neither does it establish that all three variants are valid or that a common causal effect is correct. A formal test of between-variant heterogeneity is a separate analysis. Even a reassuring comparison cannot rule out shared bias affecting all variants.

What the Checks Can and Cannot Establish

The practical value of diagnostics is to identify evidence that should change the analysis or prompt investigation—not to certify a causal conclusion. A useful distinction is whether a claim concerns an observable feature of the analyzed data or an assumption about causal structure that is not identified from those data alone.

Diagnostic or assumption	What can be assessed	What it cannot establish by itself
Instrument strength	Whether the estimated variant–exposure association is precise and sufficiently strong for the chosen summary	Independence from confounders or the exclusion restriction
Allele alignment	Whether effect-allele labels, frequencies, and signs appear consistent across data sources	That every allele was identified correctly if information is ambiguous, or that the variant is causally valid
Consistency of ratio estimates	Whether estimates differ visibly relative to their uncertainty	That all variants satisfy the assumptions; shared bias can produce similar estimates
Independence	Whether known ancestry, relatedness, or design concerns merit investigation	That the variant is independent of all relevant causes of the outcome
Exclusion restriction	Whether external biological knowledge raises plausible alternative pathways	That no pathway from the variant to the outcome bypasses the exposure

The independence assumption can be threatened by population structure, assortative mating, or other relationships between genetic variation and causes of the outcome. Observed ancestry variables and study-design information can help reveal risks, but absence of an observed association with measured covariates does not show independence from every relevant cause. Likewise, exclusion can be challenged by evidence of a direct or alternative pathway, but a lack of such evidence does not prove that no such pathway exists.

Common Mistakes

Wrong reasoning: “The F statistic exceeds 10, so the instrument is valid.” Correction: A strength statistic informs the relevance question; it does not establish independence or exclusion.
Wrong reasoning: “The estimates agree, so horizontal pleiotropy is absent.” Correction: Agreement may be compatible with validity, but variants can share bias or pleiotropic effects that produce similar estimates.
Wrong reasoning: “The effect-allele labels look similar, so the signs need no checking.” Correction: Confirm the same allele contrast in both sources, use frequency information where informative, and avoid guessing when orientation is ambiguous.
Wrong reasoning: “A diagnostic found no problem, so the assumption has been proved.” Correction: A diagnostic’s failure to flag a problem is not proof that the underlying assumption holds.

Key takeaway: Strength summaries, allele checks, and variant-specific comparisons are evidence about the data and potential sources of error. They can identify concerns and guide cautious interpretation, but they cannot verify the independence assumption or the exclusion restriction. A causal interpretation still depends on those assumptions and on the design and biological context.

Check Your Understanding

For each question, distinguish what a diagnostic can show from what remains an assumption.

A variant has \(\hat{\beta}_{X}=0.060\) and \(SE(\hat{\beta}_{X})=0.015\). Calculate its approximate F statistic and state what that value does—and does not—tell you.
In one data source an outcome estimate is reported per copy of allele A, while the exposure estimate is per copy of the opposite allele. What must happen to the outcome estimate before calculating a ratio?
Why can close agreement between several variant-specific estimates not prove that the exclusion restriction holds?
What additional difficulty arises when a variant is palindromic and allele-frequency information is unavailable or uninformative?
Explain why a high F statistic is not evidence that a variant is independent of all causes of the outcome.