Why a Sensitivity-Analysis Workshop?
Mendelian randomization (MR) estimates can change when the analysis uses different assumptions about genetic instruments. Agreement across methods can be reassuring, but does not establish that every assumption holds. Disagreement is useful evidence to investigate: it may reflect pleiotropy, a poorly estimated association, heterogeneity, or the fact that estimators target their assumptions differently.
This workshop follows one hypothetical association dataset through inverse-variance weighted (IVW) estimation, MR-Egger regression, the weighted median, and heterogeneity and outlier diagnostics. The goal is not to identify one universally best estimator. It is to see what each calculation says, what it assumes, and what conclusion the combined evidence supports.
Dataset and Assumptions
Suppose four independent genetic variants have been harmonized to the same effect allele. For variant \(j\), \(\hat{\beta}_{Xj}\) is its estimated association with the exposure, in standard deviations of exposure per allele; \(\hat{\beta}_{Yj}\) is its estimated association with the outcome, in standard deviations of outcome per allele. The outcome-association standard error is \(SE(\hat{\beta}_{Yj})\). All values below are hypothetical. To keep the calculations transparent, take the variants as uncorrelated, assume the exposure and outcome association estimates come from non-overlapping samples, and treat the exposure-association estimates as sufficiently precise for these calculations.
| Variant | \(\hat{\beta}_{Xj}\) | \(\hat{\beta}_{Yj}\) | \(SE(\hat{\beta}_{Yj})\) |
|---|---|---|---|
| 1 | 0.100 | 0.04327 | 0.010 |
| 2 | 0.200 | 0.03345 | 0.010 |
| 3 | 0.300 | 0.09000 | 0.010 |
| 4 | 0.400 | 0.12995 | 0.010 |
A causal interpretation requires more than arithmetic. The variants must be relevant to the exposure; they must be independent of confounders of the exposure–outcome relationship; and they must affect the outcome only through the exposure, as required by the exclusion restriction. Robust estimators alter how some violations are handled, but none makes all these concerns disappear. The calculations below also rely on the stated independence and precision simplifications; correlated variants or appreciable uncertainty in \(\hat{\beta}_{Xj}\) require methods that account for them.
Worked Example: Fixed-Effect IVW and Heterogeneity
Worked Example: Fixed-Effect IVW and Heterogeneity
With independent variants and the stated approximation that uncertainty in the exposure associations is negligible, fixed-effect IVW estimates the common slope in a weighted regression of \(\hat{\beta}_{Yj}\) on \(\hat{\beta}_{Xj}\) through the origin. The weights are \(w_j=1/SE(\hat{\beta}_{Yj})^2\). Here every outcome standard error is 0.010, so the weights are equal and cancel from the slope calculation.
IVW estimate. The formula and substitution are
The numerator is approximately \(0.09000\), and the denominator is \(0.3000\). Thus the estimated slope is about 0.300 standard deviations of outcome per standard deviation of exposure. Under the fixed-effect model, its standard error is \(1/\sqrt{\sum_j w_j\hat{\beta}_{Xj}^{\,2}}=0.01/\sqrt{0.3000}\approx0.01826\). A model-based 95% interval is \(0.3000\pm1.96(0.01826)\), or approximately 0.2642 to 0.3358. This interval describes uncertainty under the stated model; it does not account for unmodelled pleiotropy.
Heterogeneity. Cochran’s \(Q\) measures the weighted residual variation around the fitted IVW slope:
Using the displayed associations, the residuals are approximately \(0.01327,-0.02655,0,0.00995\). Since each weight is \(1/0.010^2=10{,}000\),
There are four variants and one fitted slope, so \(df=4-1=3\). Under the no-heterogeneity reference model, the chi-squared tail probability is approximately 0.02034. This is evidence of more between-variant residual variation than that model predicts. It is not the probability that the common-effect model is true, and it does not identify the cause of the variation. Rounded table entries can produce a very slightly different \(Q\); the displayed \(Q\) and p-value use the underlying values before rounding.
A short justification explains why \(Q\) has three degrees of freedom here. Under the fixed-effect model, the four outcome associations are compared with predictions from one slope estimated using those same associations. Estimating that one parameter uses one degree of freedom, leaving \(4-1=3\) independent residual contrasts. With independent, normally distributed association errors of known standard errors and a correctly specified common slope, the weighted sum of squared residuals has a chi-squared reference distribution with those degrees of freedom. Pleiotropy or other model violations can invalidate that reference interpretation.
Worked Example: MR-Egger and the Weighted Median
Worked Example: MR-Egger and the Weighted Median
MR-Egger. MR-Egger fits \(\hat{\beta}_{Yj}=\alpha+\beta_{\mathrm{Egger}}\hat{\beta}_{Xj}+\varepsilon_j\), using inverse-variance weights. The intercept \(\alpha\) represents average directional pleiotropic association under the model; the slope is the MR-Egger causal estimate. Because the outcome standard errors are equal here, this is ordinary least squares on the four displayed points.
The mean exposure association is \(\bar{x}=0.250\), and \(S_{xx}=\sum_j(\hat{\beta}_{Xj}-\bar{x})^2=0.0500\). The fitted slope is
The mean outcome association is approximately \(0.07417\); therefore the intercept is \(\hat{\alpha}=\bar{y}-\hat{\beta}_{\mathrm{Egger}}\bar{x}\approx0.07417-0.3166(0.250)=-0.00498\). In this simplified regression, the slope standard error is \(0.010/\sqrt{0.0500}\approx0.04472\), giving a model-based 95% interval of about 0.2290 to 0.4042. The intercept is small relative to its standard error, approximately \(0.01225\); that does not show that directional pleiotropy is absent.
MR-Egger’s causal interpretation depends especially on the InSIDE assumption: instrument strength must be independent of the instruments’ direct effects on the outcome. Its regression also relies on the treatment of exposure-association uncertainty; appreciable uncertainty can attenuate the slope unless appropriately addressed. An intercept test has limited power, particularly with few variants, so a non-significant result is not a validation of the exclusion restriction.
Weighted median. With equal outcome standard errors, ratio-estimate precision weights are proportional to \(\hat{\beta}_{Xj}^{\,2}\). The ratio estimates are approximately 0.4327, 0.1673, 0.3000, and 0.3249 for variants 1 through 4. In increasing order they are 0.1673, 0.3000, 0.3249, and 0.4327, with corresponding weights proportional to 0.0400, 0.0900, 0.1600, and 0.0100. The total weight is 0.3000; cumulative weight passes half of that total at 0.3249. The weighted-median estimate is therefore approximately 0.3249.
Its robustness requires that more than half of the total weight come from valid instruments. The point estimate alone cannot verify that condition. In this example, variant 4 alone carries \(0.1600/0.3000=53.3\%\) of the weight, so the weighted-median result is particularly sensitive to whether that variant is valid. This illustrates why a robustness label is conditional on assumptions, not a guarantee.
Worked Example: Reading an Outlier Diagnostic
Worked Example: Reading an Outlier Diagnostic
A useful first diagnostic is each variant’s contribution to \(Q\), \(w_j(\hat{\beta}_{Yj}-\hat{\beta}_{\mathrm{IVW}}\hat{\beta}_{Xj})^2\). Using the residuals from the IVW fit and \(w_j=10{,}000\), the contributions are approximately 1.761, 7.049, 0, and 0.990. They sum to \(Q\approx9.800\). Variant 2 contributes the most; its standardized residual is approximately \(-0.02655/0.010=-2.655\).
This makes variant 2 a candidate for closer inspection, not a proven invalid instrument. A large residual can arise from a direct pleiotropic effect, a data or harmonization issue, chance variation, or an inadequate common-slope model. A formal outlier procedure must account for the fact that the slope was estimated from the same data and, if many variants are screened, for multiple testing. It is not justified to delete the largest contributor automatically and report the remaining estimate as causal.
The point estimates now tell a mixed but interpretable story: IVW is about 0.3000, MR-Egger is about 0.3166, and the weighted median is about 0.3249. Their proximity is not proof of validity, while the heterogeneity result and the largest \(Q\) contribution warrant scrutiny. The methods also rely on different assumptions, so their differences should be investigated rather than resolved by choosing the estimate that looks most appealing.
How to Report the Sensitivity Analysis
A useful report separates the numerical findings from the causal interpretation. State the effect scale, the number and independence of variants, the estimator and its key assumptions, and uncertainty where it is available. Report heterogeneity and explain what it indicates without treating its p-value as a pleiotropy test for a specific mechanism. Describe influential variants as candidates for investigation and report any justified sensitivity analysis transparently; do not conceal analyses that disagree.
For this hypothetical dataset, a measured conclusion would be: “The fixed-effect IVW estimate was 0.300 standard deviations of outcome per standard deviation of exposure (model-based 95% interval 0.264 to 0.336). MR-Egger and the weighted-median point estimates were 0.317 and 0.325, respectively. Cochran’s \(Q=9.800\) on 3 degrees of freedom (\(p\approx0.02034\)) indicates heterogeneity under the fixed-effect reference model, with variant 2 contributing most. These results are compatible with a positive effect under their respective assumptions, but heterogeneity and instrument validity require further assessment.” The wording reports the estimates without claiming that their agreement proves causality.
Common Pitfall: Picking the Estimate That “Wins”
Wrong reasoning: “The weighted median is called robust, so I will use it and ignore the heterogeneity.” This is not justified: the weighted median requires more than half the total weight to come from valid instruments, and the name does not establish that condition in these data.
Correction: Compare each estimate with its assumptions, examine heterogeneity and influential variants, and investigate plausible data or biological explanations. Do not choose an estimator solely because it gives a preferred result, or delete an outlier solely because it changes the estimate. Sensitivity analysis shows how conclusions depend on assumptions; it cannot, by itself, certify those assumptions.
Check Your Understanding
Use the worked dataset and explanations to answer these questions.
- Why does the fixed-effect IVW estimate use one fewer degree of freedom than the number of variants when calculating \(Q\)?
- What does \(Q=9.800\) with 3 degrees of freedom suggest, and what does it not establish?
- What is the difference between the MR-Egger intercept and its slope?
- Why does the weighted-median estimate of approximately 0.3249 not, by itself, establish a robust causal effect?
- What does variant 2’s large contribution to \(Q\) justify doing, and what does it not justify doing automatically?