Likelihood-Based Sensitivity Curves for Treatment Effects

Why trace an estimate across sensitivity values?

In a selection model, the assumed relationship between an outcome and the chance that it is observed can affect the treatment-effect estimate. When the response-effect parameter is uncertain, reporting only the estimate at one chosen value can hide that dependence. A sensitivity curve makes it visible: it traces the treatment-effect estimate, and the likelihood support around it, as the response-effect parameter is held fixed at a sequence of plausible values.

This is a conditional analysis over a grid, not a claim that the data have selected the most scientifically credible grid value. Its purpose is to show how conclusions depend on the selection assumption. The central tasks are to define the likelihood slice precisely, optimize it reliably, and keep within-slice uncertainty separate from comparisons across different fixed values.

Model and definition of a sensitivity curve

For a concrete setup, let \(A_i\) indicate treatment (\(A_i=1\)) or control (\(A_i=0\)), let \(Y_i\) be a continuous outcome, and let \(R_i=1\) indicate that \(Y_i\) is observed. Consider the selection model

Here \(\gamma\) is the response-effect parameter: it describes how the log odds of observation change with the outcome, conditional on treatment. In this model, increasing \(Y_i\) by one outcome unit multiplies the odds of observation by \(e^\gamma\). The treatment-effect estimand is \(\tau=\mu_1-\mu_0\), in outcome units. This example uses normal outcome distributions and a logistic response model; a different fitted selection model requires its own corresponding likelihood and estimand.

For observed data \((A_i,R_i,R_iY_i)\), a person's likelihood contribution is \(f(Y_i\mid A_i)\pi(Y_i,A_i)\) when \(R_i=1\), and \(\int f(y\mid A_i)\{1-\pi(y,A_i)\}\,dy\) when \(R_i=0\), where \(\pi(y,a)=P(R_i=1\mid Y_i=y,A_i=a)\). The integral accounts for the unobserved outcome rather than treating it as known. At a fixed \(\gamma\), maximize the resulting observed-data log likelihood over the remaining parameters.

The maximization over \(\lambda\) is repeated at each treatment-effect value when constructing \(\ell_p(\gamma,\tau)\); it is not enough to hold nuisance estimates at their values from the unrestricted fit. Similarly, each fixed \(\gamma\) requires its own fit. The estimates are therefore conditional on the corresponding response-effect assumptions.

Why the likelihood-ratio curve has this form

At a given \(\gamma\), the best-fitting treatment effect has log likelihood \(\ell_p(\gamma,\hat{\tau}(\gamma))\). A candidate \(\tau\) cannot fit better than that maximum. The difference in log likelihood is nonnegative, and multiplying by two gives the usual likelihood-ratio discrepancy:

The inequality follows directly from the definition of \(\hat{\tau}(\gamma)\): it maximizes the profile log likelihood over \(\tau\). If regular likelihood theory is appropriate, a one-parameter likelihood-ratio confidence set uses values with \(D_\gamma(\tau)\leq 3.84\), the approximate 95th percentile of a chi-squared distribution with one degree of freedom. This is a within-slice result. Boundary or other nonstandard features can invalidate that calibration, so the appropriate calibration must be checked rather than assumed.

A local quadratic approximation also explains the connection to curvature. Near a well-behaved maximum, write \(\ell_p(\gamma,\tau)\approx \ell_p(\gamma,\hat{\tau})-\frac{1}{2}I_p(\gamma)(\tau-\hat{\tau})^2\), where \(I_p(\gamma)>0\) is the profile curvature. Then \(D_\gamma(\tau)\approx I_p(\gamma)(\tau-\hat{\tau})^2\). A flatter likelihood has smaller curvature and gives weaker support over a wider range of treatment effects. This approximation is local; it should not be mistaken for an exact description of a skewed or irregular likelihood.

Reading estimates and support together

A sensitivity analysis should display more than a sequence of point estimates. For each \(\gamma\), report or plot \(\hat{\tau}(\gamma)\), a within-slice likelihood-support summary such as a likelihood-ratio interval, and enough diagnostics to judge whether the fit converged to a reliable maximum. The response-effect grid and the treatment-effect units should be explicit. A plot can use one panel for the estimate curve and another for the profile likelihood or interval limits; this avoids suggesting that the estimate alone captures all uncertainty.

Comparing sensitivity values without conflating them

A likelihood-ratio curve within a fixed \(\gamma\) compares candidate treatment effects under the same response-effect assumption. It does not, by itself, compare the plausibility of different \(\gamma\) values. If the response-effect parameter is deliberately treated as a sensitivity parameter, the analyst should not select its value merely because it gives a preferred estimate or the largest reported conditional likelihood. In a model where \(\gamma\) is instead estimated as an ordinary parameter, cross-value likelihood comparisons address a different inferential question and may be unreliable when the likelihood is weakly identified or irregular.

The main alternatives serve different purposes. A single fixed-\(\gamma\) analysis gives an answer conditional on one assumption but conceals variation across assumptions. A sensitivity curve exposes that variation. A Wald interval summarizes local curvature and can be misleading when the profile is asymmetric, flat, or ridge-like; a profile-likelihood interval uses the shape of the likelihood more directly, subject to its calibration conditions. Parametric bootstrap inference can assess finite-sample behavior under a specified model and fixed sensitivity value, but it does not remove uncertainty about which sensitivity value is credible. These tools complement, rather than replace, a transparent choice of a plausible grid.

Diagnostics, limitations, and common pitfalls

A curve is only as trustworthy as the fits used to produce it. At every grid value, check convergence, gradients or scores, Hessian behavior, and whether alternative starting values lead to the same maximum. Inspect the profile likelihood for flat regions, asymmetry, multiple peaks, or abrupt changes between nearby grid values. Repeating fits with denser grid points can reveal apparent jumps caused by inadequate optimization or coarse spacing. Numerical integration error must also be controlled when the observed-data likelihood contains integrals over missing outcomes.

A changing interval width can reflect changing profile curvature, not just movement of the point estimate. An interval that suddenly expands, a maximum that shifts markedly under small changes in starting values, or a discontinuity in the estimate curve is a warning to investigate the optimization and likelihood shape before interpreting the result. Sensitivity curves also remain conditional on the outcome and response models, covariate specification, and chosen grid; they cannot establish that those assumptions are correct.

• Pitfall: treating \(\gamma=0\) as automatically correct. It means no outcome dependence in the specified response model, not that missingness is harmless under every model or that this value is scientifically established. Include it when useful, but justify the broader grid.
• Pitfall: calling each interval an unconditional confidence interval. The intervals describe uncertainty within a fixed-\(\gamma\) slice. They do not account for uncertainty about which \(\gamma\) value is appropriate.
• Pitfall: comparing the maxima of separate slices as if they selected the sensitivity value. A sensitivity grid is commonly an assumption analysis, not an estimation rule for \(\gamma\). State the conditional interpretation instead.
• Pitfall: interpreting a smooth-looking curve as evidence of a stable fit. Smoothness is not a convergence diagnostic. Check the likelihood and optimization at each grid point.