Reporting Subject-Level Influence in MNAR Analyses

Why Reporting Details Matter

A local influence plot can identify directions in which a fitted selection model is sensitive to small changes. But a plot alone does not say what was perturbed, how large a unit perturbation represents, or whether the sensitive direction changes the scientific target. A reproducible report must make those choices explicit. This is particularly important in missing-not-at-random (MNAR) analyses: local influence diagnoses sensitivity of a fitted model to specified perturbations; it does not establish that the MNAR assumptions are correct.

The earlier material on local influence establishes how to calculate directional curvature, adjust a diagnostic to a target parameter, and check numerical calculations. Here the emphasis is how to communicate those results so another analyst can understand and reproduce the subject-level analysis without mistaking a diagnostic for a model comparison or a causal conclusion.

What a Reproducible Report Specifies

State the selection-model likelihood, the fitted parameter vector, and the perturbation scheme. Let \(\ell(\theta;\omega)\) be the observed-data log likelihood under the selection model, where \(\theta\) contains the model parameters and \(\omega\) indexes a chosen perturbation. The unperturbed fit is at \(\omega_0\), and \(\hat{\theta}\) maximizes \(\ell(\theta;\omega_0)\). For subject-level perturbations, say exactly which subject-specific observed-data contribution or model component is modified. A statement such as “we perturbed subject \(i\)” is not enough: for example, the perturbation might rescale that subject’s contribution to the log likelihood or alter a specified selection probability.

Also report the perturbation’s baseline, units, and scaling. If a subject’s observed outcome is shifted, give the shift in outcome units or in standard-deviation units. If a selection probability is perturbed, state the scale on which it changes. A unit change on a logit scale is not the same as a unit change in probability. The perturbations should be defined so the baseline value reproduces the original fitted model.

For a coordinate perturbation that changes only subject \(i\), the corresponding coordinate curvature is obtained by taking \(d\) to be the unit vector for that coordinate. Report whether directions are unit length, and how coordinates were scaled. Curvature depends on the perturbation scheme and its units; it is not an intrinsic property of a subject. When a joint direction is important, report its components as well as its curvature. High curvature for a joint direction need not mean that any one subject alone produces the same sensitivity.

Worked Example: Reporting Subject-Level Curvature

Connect Curvature to the Scientific Target

Likelihood-displacement curvature summarizes changes in the fit as a whole. It need not rank subjects the same way as sensitivity of the scientific target. Let \(q(\theta)\) be a scalar target, such as a treatment-effect contrast, and let \(q_\theta\) be its gradient with respect to \(\theta\) at the fitted value. Differentiating the likelihood score equation gives the first-order parameter change in direction \(d\):

This follows because the fitted score is zero at each perturbed optimum. Differentiating that equation gives \(H(\partial\hat{\theta}/\partial\omega)d+\Delta d=0\). Since \(H=-J\), rearranging yields the displayed expression. This is a local, first-order approximation: it describes the fitted target’s rate of change at the baseline, not necessarily its change under a large perturbation.

For subject-level reporting, give the fitted target and its standard error, then report the target’s first-order change for a clearly defined perturbation. A change divided by the target’s standard error can help communicate scale, but it is a sensitivity measure, not a test statistic or a new confidence interval. If the perturbation affects a nuisance parameter but barely changes the target, high overall curvature and low target sensitivity can coexist.

Numerical Checks and a Worked Report

A report should describe checks that support the calculation, not merely state that software returned a curvature. At a minimum, check that the optimizer converged at the baseline and at representative perturbed fits; that the fitted score is sufficiently close to zero; and that the observed information is invertible and numerically stable for the calculation being reported. If the information matrix is nearly singular, curvature and derivative estimates may be highly sensitive to numerical error.

Compare analytical derivatives with finite-difference approximations at more than one small step size when feasible. The perturbation should be small enough for a local approximation but not so small that rounding error dominates. Check that the baseline perturbation reproduces the original likelihood and estimates. If results change materially with reasonable step sizes, optimization tolerances, or parameter scaling, report that instability rather than selecting the most convenient calculation.

How to Present Findings and Limitations

A concise report can present the perturbation and its scale, the curvature measure and ranking rule, the subjects or directions with the largest values, the target estimate and its local sensitivity, and the numerical checks. Give enough precision to reproduce rankings, but avoid implying that tiny differences in curvature are meaningful. When several directions have similar high curvature, report that fact rather than presenting an unstable single “top” direction as definitive. If records have different perturbation scales, explain how comparability was achieved.

Local influence complements, rather than replaces, other diagnostics. A deletion diagnostic asks what happens after omitting a record, a finite change that may move the fit beyond the local neighborhood. Local curvature instead studies the response to infinitesimal changes under a defined perturbation scheme. Neither alone provides a general ranking that is independent of the model and question. Global fit summaries address different questions and do not substitute for a subject-level sensitivity report.

Several failure modes deserve explicit attention. A poorly scaled perturbation can make one subject appear influential merely because its coordinate represents a larger change. A nearly singular information matrix can magnify numerical noise. A local approximation can be misleading when the likelihood is irregular or the fitted parameters move substantially under modest changes. Finally, MNAR conclusions depend on assumptions about unobserved outcomes and selection; a local influence analysis cannot verify those assumptions from observed data alone. These limitations should be stated alongside, not hidden behind, a curvature plot.