Local Influence in a Small Surgical Trial

Why Local Influence Matters in a Small Trial

In a small surgical trial, a few participants can have considerable leverage over an estimate of the treatment effect or the assumed association between outcome and dropout. That possibility is especially important in a selection model: a participant with an observed outcome contributes information about both the outcome distribution and the probability of observation, while a participant with a missing outcome contributes through the model’s implied probability of dropout. Local influence asks how much fitted estimates would move under a small change to one participant’s contribution.

This is a diagnostic, not a new estimator or a test of whether a participant should be excluded. The aim is to identify which records deserve careful review, and whether their influence is mainly tied to an unusual outcome, an unusual dropout pattern, or the model’s assumptions. The calculations are conditional on the specified selection model and its identifying restrictions.

A Selection Model and Its Participant Contributions

Consider a hypothetical randomized surgical trial. Let \(Y_i\) be a participant’s 12-week pain improvement, in points, with larger values representing greater improvement. Let \(R_i=1\) if that outcome is observed and \(R_i=0\) if it is missing. Let \(A_i\) indicate randomized treatment, and let \(X_i\) be baseline pain severity. A simple selection model specifies an outcome distribution and a response probability conditional on the outcome:

Y_i\mid A_i,X_i \sim N(\beta_0+\tau A_i+\beta_1X_i,\sigma^2), \qquad P(R_i=1\mid Y_i,A_i,X_i) = \operatorname{logit}^{-1}(\alpha_0+\alpha_A A_i+\gamma Y_i+\alpha_X X_i).

Here \(\tau\) is the treatment contrast in mean improvement, and \(\gamma\) describes how the chance that the outcome is observed changes with the outcome itself, conditional on treatment and baseline severity. A nonzero \(\gamma\) allows missingness to depend on an unobserved outcome, which is the feature that makes this a selection model for potentially MNAR data.

Definition: Write \(\theta=(\beta_0,\tau,\beta_1,\sigma,\alpha_0,\alpha_A,\gamma,\alpha_X)\) for the model parameters. The observed-data likelihood contribution for participant \(i\) is \(f_\theta(y_i\mid A_i,X_i)\pi_\theta(y_i,A_i,X_i)\) when \(R_i=1\), and \(\int f_\theta(y\mid A_i,X_i)\{1-\pi_\theta(y,A_i,X_i)\}\,dy\) when \(R_i=0\), where \(f_\theta\) is the outcome density and \(\pi_\theta\) is the response probability above.

Assume participants are independent conditional on the model covariates, the outcome and response models are correctly specified for the diagnostic being conducted, and the likelihood has a sufficiently regular local maximum. The response probabilities must also be meaningful over the covariate and outcome ranges under consideration. In particular, an extreme fitted probability near zero or one can make the fit and its local approximations unstable. As with other MNAR selection models, the observed data alone generally do not identify every outcome-dependence parameter without additional restrictions or sensitivity assumptions. Influence calculations do not remove that limitation.

Let \(\ell_i(\theta)\) be the log of the observed-data likelihood contribution for participant \(i\), using the appropriate observed or integrated expression, and let \(\ell(\theta)=\sum_{i=1}^n\ell_i(\theta)\). At the fitted estimate \(\hat\theta\), define the participant score \(s_i=\partial\ell_i(\theta)/\partial\theta\) evaluated at \(\hat\theta\), and the observed information \(J=-\partial^2\ell(\theta)/\partial\theta\partial\theta^\mathsf{T}\) evaluated there. Under regularity conditions, \(J\) is positive definite. The fitted score contributions sum to zero, \(\sum_i s_i=0\), up to numerical tolerance.

Deriving the Local Perturbation

Give participant \(i\)’s log-likelihood contribution a weight \(w_i\), leaving the other weights at one. The weighted log likelihood is \(\ell_w(\theta)=\sum_jw_j\ell_j(\theta)\), and its fitted estimate \(\hat\theta(w)\) solves the weighted score equation. At the unperturbed fit, all \(w_j=1\). Differentiate that equation with respect to \(w_i\):

0=\frac{\partial}{\partial w_i} \left\{\sum_j w_j\frac{\partial\ell_j(\hat\theta(w))}{\partial\theta}\right\} =s_i-J\frac{\partial\hat\theta(w)}{\partial w_i}.

The last equality uses the definition \(J=-\partial^2\ell/\partial\theta\partial\theta^\mathsf{T}\). Rearranging gives the first-order change in the estimate for a small weight change \(\varepsilon\):

\frac{\partial\hat\theta(w)}{\partial w_i}\bigg|_{w=1} =J^{-1}s_i, \qquad \hat\theta(w_i=1+\varepsilon)-\hat\theta \ \approx\ \varepsilon J^{-1}s_i.

Thus, a large score alone is not the whole story: the information matrix determines how strongly the fitted parameters respond in that direction. A useful overall, dimensionless diagnostic for a unit-sized weight perturbation is \(C_i=s_i^\mathsf{T}J^{-1}s_i\). It measures squared displacement in the local information metric. To inspect a particular coordinate, such as \(\gamma\), use the corresponding coordinate of \(J^{-1}s_i\), and compare its magnitude with the estimated standard error \(\sqrt{(J^{-1})_{\gamma\gamma}}\). These are local approximations; deleting a participant changes a weight by \(-1\), which may be too large for the approximation to be accurate.

Worked Example: A Participant’s Local Effect on the Treatment and Selection Estimates

Suppose a hypothetical trial fit has two parameters of immediate interest, \(\theta=(\tau,\gamma)\). For one participant, the fitted score components are \(s_i=(0.20,-0.60)^\mathsf{T}\), and the corresponding information matrix for this simplified two-parameter calculation is \(J=\begin{pmatrix}4&0\\0&9\end{pmatrix}\). These are stipulated outputs from the hypothetical model fit, not raw trial data. The fitted standard error for \(\gamma\) in this simplified calculation is \(\sqrt{1/9}=0.3333\).

The local change per unit increase in the participant’s likelihood weight is \(J^{-1}s_i=(0.20/4,-0.60/9)^\mathsf{T}=(0.0500,-0.0667)^\mathsf{T}\). For a small increase of \(0.1\) in the weight, the first-order estimate changes are therefore \(\Delta\tau\approx0.1(0.0500)=0.00500\) and \(\Delta\gamma\approx0.1(-0.0667)=-0.00667\).

The overall diagnostic is \(C_i=s_i^\mathsf{T}J^{-1}s_i=0.20(0.0500)+(-0.60)(-0.0667)=0.0100+0.0400=0.0500\). For \(\gamma\) alone, the unit-weight change is about \(0.0667/0.3333=0.200\) fitted standard errors. That is a modest local displacement in this illustration. It is not a p-value and does not say that the participant is erroneous or should be removed.

Distinguishing Outcome Influence from Dropout Influence

A selection-model contribution depends on both the observed outcome and the response indicator. Consequently, influential records do not all have the same explanation. An observed participant with an extreme pain-improvement value may have a large score because that value is surprising under the fitted outcome distribution, because it is surprising under the fitted response model, or both. A participant with a missing outcome has no observed value to be an outcome residual; their contribution is the integrated probability of being missing across possible outcomes. Their influence can instead reflect an unusual combination of treatment, baseline covariates, and dropout status under the fitted model.

Worked Example: Comparing an Extreme Observed Outcome with Dropout

Imagine two records from a hypothetical 18-participant surgical trial, with scores evaluated at the same fitted model and parameters ordered as \((\tau,\gamma)\). Participant A has an observed improvement of 18 points, unusually high relative to the fitted outcome distribution, and a score \(s_A=(0.40,-1.80)^\mathsf{T}\). Participant B has a missing 12-week outcome after being assigned to the treatment arm; their integrated missing-outcome likelihood contribution yields \(s_B=(0.10,-1.20)^\mathsf{T}\). These score values are hypothetical fitted diagnostics.

Using the same illustrative information matrix \(J=\begin{pmatrix}4&0\\0&9\end{pmatrix}\), participant A’s displacement per unit weight is \(J^{-1}s_A=(0.40/4,-1.80/9)^\mathsf{T}=(0.100,-0.200)^\mathsf{T}\). Their overall influence measure is \(C_A=0.40(0.100)+(-1.80)(-0.200)=0.0400+0.360=0.400\).

For participant B, \(J^{-1}s_B=(0.10/4,-1.20/9)^\mathsf{T}=(0.0250,-0.133)^\mathsf{T}\), and \(C_B=0.10(0.0250)+(-1.20)(-0.133)=0.00250+0.160=0.163\), to three significant figures. In this illustrative calculation, A has the larger overall local influence, and both records’ score directions would locally pull \(\gamma\) downward if their weights were increased.

The scores flag records for investigation; they do not by themselves prove why those records are influential. For A, compare the outcome with the fitted outcome distribution and examine the response-model contribution. For B, review the observed dropout information and fitted response probabilities for participants with comparable covariates. A large dropout-related score is not evidence that the missing outcome took a particular value: that value was integrated out.

A practical review therefore pairs influence calculations with record-level checks. Confirm the recorded treatment, baseline covariates, outcome units, visit timing, and missingness reason against the trial data definitions. Then inspect whether high influence is concentrated among observed extreme outcomes, among dropouts in one treatment group, or among participants with covariate profiles that are sparsely represented. The score decomposition into outcome and response-model components, when available, helps make this assessment more specific.

Case Deletion, Diagnostics, and Alternatives

Case deletion is a direct companion to local influence. For participant \(i\), set \(w_i=0\) and refit the model. The first-order approximation is \(\hat\theta_{(-i)}-\hat\theta\approx-J^{-1}s_i\), because the weight change is \(-1\). Compare that approximation with the actual refit, particularly for the treatment contrast \(\tau\) and selection parameter \(\gamma\). A substantial discrepancy indicates that the deletion is too large for the local approximation, that the likelihood is strongly nonlinear nearby, or that the fit has moved into a different region.

Worked Example: Checking a Case-Deletion Approximation

For a participant in a hypothetical fit, suppose \(J^{-1}s_i=(0.0500,-0.0667)^\mathsf{T}\) for \((\tau,\gamma)\), as in the first calculation. Removing that participant changes the weight by \(-1\), so the local approximation is \(\Delta\tau\approx-0.0500\) pain-improvement points and \(\Delta\gamma\approx+0.0667\).

Suppose the full refit after deletion gives \(\Delta\tau=-0.047\) points and \(\Delta\gamma=+0.072\). The discrepancies are \(|-0.047-(-0.0500)|=0.00300\) points and \(|0.072-0.0667|=0.00533\) in the selection-model coefficient. They are small relative to the displayed changes in this hypothetical example, so the local approximation describes the deletion reasonably well. If instead the refit gave \(\Delta\gamma=+0.40\), the local calculation would clearly understate the deletion effect; the refit, not the derivative, would describe the actual case-deletion result.

This comparison checks approximation quality, not whether the model assumptions are true. Both fits still rely on the specified outcome and response models and on whatever restrictions were used to identify the selection parameter.

Common diagnostics include the case-specific scores, \(C_i\), the estimated changes in \(\tau\) and \(\gamma\), and actual leave-one-participant-out refits for the most influential records. If computation permits, perturbations can also be targeted: for example, vary the contribution of observed outcomes separately from the dropout contributions, or examine whether changing one treatment group’s dropout contributions has a disproportionate effect. Such targeted perturbations answer specific diagnostic questions; they should be defined transparently and not selected after seeing which one produces a preferred treatment estimate.

Approach	What it reveals	Main limitation
Local likelihood-weight perturbation	Direction and first-order size of estimate changes near the fitted model	May poorly approximate a full deletion or a large perturbation
Case deletion and refitting	Actual change when one participant’s full likelihood contribution is removed	Can be computationally demanding and may expose separation or convergence problems
Outcome residual or response-probability review	Whether unusual observed values or fitted dropout patterns merit data checks	Does not measure the joint effect on the selection-model estimates by itself
Alternative MNAR assumptions	How conclusions change under different assumptions about unobserved outcomes	Addresses assumption sensitivity, not the influence of a single participant alone

Results should be treated cautiously if \(J\) is nearly singular, if standard errors are very large, if fitted response probabilities are extreme, or if deleting a participant causes convergence failure. These can indicate sparse data, weakly determined parameters, or model instability. A large influence measure is also scale- and parameterization-dependent unless the comparison uses a stated metric; report changes in the original parameter units alongside any standardized diagnostic.

Common Pitfall and Interpretation

A common mistake is to label the participant with the most extreme observed outcome as the influential case, and then assume that dropping that record is a valid correction. That reasoning confuses extremeness with influence and ignores dropout contributions. An extreme outcome may have little effect if it is well accommodated by the model or if other records provide strong information. A participant with a missing outcome may be influential through the response likelihood even though no outcome residual can be calculated. The correction is to evaluate the fitted likelihood contributions, inspect their sources, and compare local results with actual refits before drawing a conclusion.

Key takeaway: Local influence identifies how sensitive a fitted selection-model estimate is to small changes in participant contributions. It can direct data checks and model diagnostics, but it cannot establish that a record is invalid, identify a missing outcome, or resolve the assumptions needed for an MNAR analysis.

Check Your Understanding

Answer each question, then compare with the brief solutions.

In a selection model, why can a participant with a missing outcome still influence the fitted selection parameter?
If \(J^{-1}s_i=(0.08,-0.12)^\mathsf{T}\) for \((\tau,\gamma)\), what is the first-order change in the estimates when that participant’s weight is reduced by \(0.25\)?
Why should a local deletion approximation be compared with an actual case-deletion refit?
Does a large \(C_i\) show that a participant’s record is erroneous? Explain.
Name one pattern that could suggest influence is related to dropout rather than an unusual observed outcome.

Solutions. (1) The missing-outcome likelihood contribution integrates the joint outcome and dropout model over possible outcomes, so the observed dropout indicator and covariates contribute information. (2) Multiply by \(-0.25\): \(\Delta\tau\approx-0.25(0.08)=-0.020\), and \(\Delta\gamma\approx-0.25(-0.12)=+0.030\). (3) A full deletion is a finite change, while the derivative is only a local approximation; nonlinearity or instability can make them differ. (4) No. It indicates sensitivity of the fitted estimates to that likelihood contribution and motivates review, not a verdict about data quality. (5) Examples include influence concentrated among dropouts in one treatment group or among dropouts with sparsely represented baseline covariates.