Influence Diagnostics For Mendelian Randomization

Why Examine the Influence of Individual Variants?

A Mendelian randomization estimate combines information from genetic variants used as instruments. That combination can look precise while depending strongly on one variant: removing it may substantially change the estimate or its uncertainty. Influence diagnostics make this dependence visible by repeatedly recalculating an estimate after omitting one variant at a time.

A leave-one-variant-out analysis is a sensitivity analysis, not a test that certifies instruments as valid. A large change identifies a variant whose inclusion matters for the chosen estimator and data. It does not, by itself, tell us whether the variant is affected by pleiotropy, has a measurement problem, or simply carries unusually much information.

Setup and Definitions

Suppose summary data are available for $J$ genetic variants. For variant $j$, let $\hat\beta_{Xj}$ be its estimated association with the exposure and $\hat\beta_{Yj}$ its estimated association with the outcome. Let $SE(\hat\beta_{Yj})$ be the standard error of the outcome association, and set $w_j=1/SE(\hat\beta_{Yj})^2$. Assume that the variants have been harmonized so the exposure and outcome associations refer to the same effect allele.

The inverse-variance weighted (IVW) estimate fits a weighted regression of outcome associations on exposure associations through the origin. In this fixed-effect summary-data formulation, the weights are $w_j$. The standard calculation treats the exposure associations as fixed and assumes independent variant-specific outcome-association errors with mean zero and variances $1/w_j$. Its causal interpretation additionally requires the usual instrumental-variable assumptions, including no relevant horizontal pleiotropy, and adequate instrument strength. Correlation between variants, weak-instrument effects, or violations of the instrumental-variable assumptions can undermine that interpretation.

Definition: The IVW estimate and its usual standard error are $$ \hat\theta_{\mathrm{IVW}}= \frac{\sum_{j=1}^{J} w_j\hat\beta_{Xj}\hat\beta_{Yj}} {\sum_{j=1}^{J} w_j\hat\beta_{Xj}^{\,2}}, \qquad SE(\hat\theta_{\mathrm{IVW}})= \frac{1}{\sqrt{\sum_{j=1}^{J} w_j\hat\beta_{Xj}^{\,2}}}. $$ The leave-one-variant-out estimate $\hat\theta_{-j}$ is the same calculation using all variants except $j$, provided the remaining variants have positive total weight for estimating the slope.

When $\hat\beta_{Xj}\ne0$, the Wald ratio is $r_j=\hat\beta_{Yj}/\hat\beta_{Xj}$, and its IVW weight is $q_j=w_j\hat\beta_{Xj}^{\,2}$. Thus the equivalent ratio expression is $$ \hat\theta_{\mathrm{IVW}}= \frac{\sum_{j\in J_+}q_jr_j}{\sum_{j\in J_+}q_j}, \qquad J_+=\{j:\hat\beta_{Xj}\ne0\}. $$ A variant with $\hat\beta_{Xj}=0$ contributes zero to both sums in the regression formula and is omitted from this ratio expression: its Wald ratio is undefined, so it must not be written as a zero-weight ratio term.

Two useful summaries are the signed deletion change $\Delta_j=\hat\theta_{-j}-\hat\theta_{\mathrm{IVW}}$ and the standardized deletion change $I_j=\Delta_j/SE(\hat\theta_{\mathrm{IVW}})$. The first is in the units of the causal estimate; the second expresses the change in units of the full estimate's standard error. Neither has a universal cutoff that determines whether a variant is invalid.

What the Deletion Change Measures

For nonzero exposure associations, the IVW estimate is a weighted average of the ratios, with weights $q_j$. Write $Q=\sum_j q_j$ and $Q_{-j}=Q-q_j$. If $Q_{-j}>0$, the estimate with variant $j$ removed is the weighted average of all the other ratios. The relationship between the full and deleted estimates can be derived directly:

\hat\theta_{\mathrm{IVW}} =\frac{Q_{-j}\hat\theta_{-j}+q_jr_j}{Q} \quad\Longrightarrow\quad \hat\theta_{\mathrm{IVW}}-\hat\theta_{-j} =\frac{q_j}{Q}\left(r_j-\hat\theta_{-j}\right) =\frac{q_j}{Q-q_j}\left(r_j-\hat\theta_{\mathrm{IVW}}\right).

This identity explains two components of influence. A variant can matter because it has a large weight $q_j$, because its ratio differs from the other variants, or both. A ratio far from the pooled estimate need not be influential if its weight is tiny. Conversely, a well-aligned variant can influence the estimate because it carries a large share of the total weight. The identity describes the fixed-effect IVW calculation; influence for another estimator must be defined using that estimator's own deletion procedure.

Worked Example: A Variant Changes the IVW Estimate

Consider four hypothetical variants. Their exposure associations are all $0.10$. The first three have outcome associations $0.05$ and outcome standard errors $0.10$. The fourth has outcome association $0.20$ and outcome standard error $0.057735$. Assume the variants are independent and the fixed-effect IVW assumptions above are being used for this calculation.

For each of the first three variants, $r_j=0.05/0.10=0.50$ and $q_j=(1/0.10^2)(0.10^2)=1$. For the fourth, $r_4=0.20/0.10=2.00$, and $q_4=(1/0.057735^2)(0.10^2)$, approximately $3.00$. Therefore,

\hat\theta_{\mathrm{IVW}} =\frac{1(0.50)+1(0.50)+1(0.50)+3(2.00)}{1+1+1+3} =\frac{7.50}{6} =1.25.

Removing variant 4 leaves three ratios of $0.50$, so $\hat\theta_{-4}=0.50$. Its deletion change is $\Delta_4=0.50-1.25=-0.75$. The full standard error is $1/\sqrt{6}=0.4082$, giving $I_4=-0.75/0.4082=-1.837$. The estimate is notably sensitive to this variant; the calculation does not establish why the ratio differs or whether the variant is invalid.

Calculating and Reading Leave-One-Out Results

A standard diagnostic is to calculate the full estimate, remove each variant in turn, and compare the resulting estimates. Useful displays include a table of the deleted estimates, a plot of those estimates with their standard errors or confidence intervals, and a plot of the deletion changes. Always identify the estimator and the scale being shown. If an estimate is near zero, a percentage change can be unstable or misleading; the signed change in the estimate's own units is often clearer.

Fit the full estimate.
Record the estimate, its standard error, the variants included, and the assumptions used to calculate it.

Delete one variant at a time.
Recalculate the same estimator after omitting each variant. Check that the remaining instruments still permit an estimate.

Compare changes and contributions.
Examine $\Delta_j$, $I_j$, and, for IVW, the variant's share $q_j/Q$. Look for estimates that change materially in magnitude, direction, or uncertainty.

Investigate, do not automatically discard.
Check data alignment, strength, and the scientific plausibility of the influential variant. Report the full and deletion results rather than silently selecting a preferred estimate.

Worked Example: A Zero Exposure Association

Suppose three hypothetical variants have $(\hat\beta_{Xj},\hat\beta_{Yj},SE(\hat\beta_{Yj}))$ values $(0.10,0.04,0.10)$, $(0.20,0.12,0.20)$, and $(0,0.03,0.15)$. For the first variant, $w_1=100$ and $w_1\hat\beta_{X1}^2=1$. For the second, $w_2=25$ and $w_2\hat\beta_{X2}^2=1$. The third has $w_3=1/0.15^2$, but its exposure association is zero, so its contribution to the IVW numerator and denominator is zero.

The first two Wald ratios are $0.04/0.10=0.40$ and $0.12/0.20=0.60$. The third ratio is undefined and is not used. Hence,

\hat\theta_{\mathrm{IVW}} =\frac{100(0.10)(0.04)+25(0.20)(0.12)} {100(0.10^2)+25(0.20^2)} =\frac{1.00}{2.00} =0.50.

Deleting either of the first two variants leaves the other variant's ratio, so the corresponding estimates are $0.60$ and $0.40$. Deleting the zero-association variant leaves the estimate at $0.50$. Its absence from the ratio calculation is not a special correction: it follows directly from its zero contribution in the regression formula. In practice, an association estimated as exactly zero also signals that the variant contributes no IVW slope information under this calculation.

Worked Example: Distinguishing Weight From Disagreement

For a third hypothetical IVW analysis, suppose three variants have ratio weights $q_1=4$, $q_2=1$, $q_3=1$, and ratios $r_1=0.25$, $r_2=0.75$, and $r_3=0.75$. These weights could arise from independent variants with nonzero exposure associations; assume the fixed-effect calculation is appropriate. The full estimate is

\hat\theta_{\mathrm{IVW}} =\frac{4(0.25)+1(0.75)+1(0.75)}{6} =\frac{2.50}{6} =0.4167.

Deleting variant 1 gives $\hat\theta_{-1}=(0.75+0.75)/2=0.7500$, a change of $0.3333$. Deleting variant 2 gives $\hat\theta_{-2}=(4(0.25)+0.75)/5=0.3500$; deleting variant 3 gives the same estimate. The full standard error is $1/\sqrt{6}=0.4082$, so the standardized change for deleting variant 1 is $0.3333/0.4082=0.8165$, while the changes for deleting variants 2 and 3 are $-0.06667/0.4082=-0.1633$. Variant 1 is influential primarily because it carries two-thirds of the total weight and pulls the estimate toward its lower ratio. Its ratio is not an extreme outlier relative to the other two; influence can arise from leverage as well as unusual disagreement.

Assumptions, Diagnostics, and Failure Modes

The usual IVW standard error and the weighted-average interpretation rely on the stated fixed-effect model: the exposure associations are treated as fixed, outcome-association errors are independent across variants, and their variances are represented by the reported standard errors. These conditions concern the calculation and its uncertainty. A causal interpretation needs more: instruments must satisfy the relevant independence and exclusion restrictions, and weak-instrument and selection effects must not invalidate the inference. The earlier material on Winner's Curse in Instrument Selection is relevant when exposure associations were selected for strength using the same or overlapping data.

Influence analysis can reveal sensitivity, but it cannot repair those assumptions. If variants are correlated, treating them as independent can misstate both the estimate's uncertainty and the apparent amount of independent information. A variant with a strong exposure association may dominate the weighted fit; a poorly estimated exposure association can also make ratio-based summaries unstable. Review harmonization and the variant-level associations before interpreting a large deletion change.

Leave-one-out analysis can also miss joint dependence. Several variants may share a source of bias, so no single deletion changes the result much even though the entire estimate is biased. Conversely, a single deletion may alter the point estimate because it removes a large amount of genuine information, not because it exposes a violation. If deleting a variant leaves too little total weight, the resulting estimate may be undefined or too imprecise to be useful; do not treat a missing deletion estimate as evidence of stability.

The standard error after deletion should be recalculated from the remaining variants, not copied from the full analysis. A point estimate can stay similar while its uncertainty grows substantially. Also, a change in statistical significance is not itself an influence measure: it can occur because the standard error changes even when the point estimate barely moves. Report the estimates and uncertainty together, and interpret differences in the scientific units of the outcome per unit of exposure.

How This Diagnostic Relates to Alternatives

Leave-one-out analysis is transparent and broadly applicable: it asks what the selected estimator would produce without each variant. It is not a competing causal estimator with its own automatic protection against invalid instruments. The weighted median, weighted mode, MR-Egger, and other robust approaches make different assumptions and target different forms of sensitivity. Agreement across methods can be reassuring only to the extent that their distinct assumptions are plausible; disagreement is a prompt to investigate, not a rule for choosing whichever result is preferred.

Variant-level outlier procedures use formal models or tests to identify unusual contributions, while leave-one-out analysis reports the effect of each deletion directly. Those approaches answer related but nonidentical questions. In particular, do not turn a visually large deletion change into a formal declaration of pleiotropy without a justified statistical procedure and appropriate attention to multiplicity and model assumptions.

Common Mistake

Pitfall: “The estimate changes when variant $j$ is removed, so variant $j$ must be invalid; I will exclude it.” A deletion change shows dependence of the estimate on that variant, not the reason for that dependence. The correction is to report the full and leave-one-out estimates, investigate data quality and assumptions, and use additional evidence or an appropriate method before drawing conclusions about validity.

A stable set of leave-one-out estimates means the result is not evidently dependent on any single included variant under the chosen estimator and analysis. It does not prove that all instruments are valid, rule out shared bias, or establish causation. An unstable result means the causal conclusion deserves caution and further investigation; it does not identify the correct estimate on its own.

Check Your Understanding

Try each problem, then compare your reasoning with the solutions below.

An IVW analysis has ratio weights $q=(2,1,1)$ and ratios $r=(0.4,0.8,0.8)$. Find the full estimate and the estimate after deleting the first variant.
Why must a variant with $\hat\beta_{Xj}=0$ be omitted from the Wald-ratio weighted-average expression, even though it contributes zero to the IVW regression sums?
A leave-one-out estimate changes little after any single deletion. Does that establish that all instruments are valid? Explain.
The full estimate is $0.30$ with standard error $0.10$; deleting a variant gives $0.45$. Calculate the signed change and standardized deletion change, and interpret them.

Solutions. (1) The full estimate is $\{2(0.4)+1(0.8)+1(0.8)\}/4=2.4/4=0.60$. Deleting the first variant gives $(0.8+0.8)/2=0.80$. (2) Its ratio divides by zero and is undefined. Its regression contributions are zero, so omit it from both ratio-expression sums rather than writing an undefined ratio multiplied by zero. (3) No. Single deletions assess dependence on individual variants; shared pleiotropy or other bias affecting several variants can persist under every deletion. (4) The signed change is $0.45-0.30=0.15$ outcome units per exposure unit. The standardized change is $0.15/0.10=1.5$ full-estimate standard errors. This indicates sensitivity worth investigating, not proof that the deleted variant is invalid.