Influence Diagnostics With Outcome-Dependent Dropout

Why Outcome-Dependent Dropout Changes Influence Calculations

When the probability of dropout depends on the outcome, a subject who drops out still contributes information to the likelihood. That contribution is not the selection probability evaluated at an invented or imputed outcome: it averages over the outcome values compatible with the outcome model. This integration affects both the fitted likelihood and its curvature, so it can also affect which subjects or perturbations appear influential.

This tutorial studies a scalar outcome and a probit selection model. The model is deliberately simple enough to show the integration and influence derivatives explicitly. The same distinction between observed and integrated contributions matters in more elaborate selection models, where numerical integration may be required.

Model: Let $Y\sim N(\mu,\sigma^2)$, where $\sigma>0$, and let $R=1$ indicate that $Y$ is observed and $R=0$ indicate dropout. Conditional on $Y=y$, suppose $P(R=1\mid Y=y)=\Phi(\alpha+\gamma y)$, where $\Phi$ is the standard normal cumulative distribution function and $\alpha,\gamma$ are selection-model parameters. In the examples, $\sigma,\alpha,\gamma$ are fixed and $\mu$ is the parameter being examined.

The model assumes the stated normal outcome distribution and probit selection mechanism are correctly specified for the analysis. It also assumes each subject contributes the appropriate observed-data likelihood term: a density and selection probability when $R=1$, and a probability integrated over the unobserved outcome when $R=0$. Influence calculations below assume a differentiable log likelihood, an interior fitted value of $\mu$, and a nonzero likelihood curvature there. These conditions make the local derivative meaningful; they do not establish that the model fits the data well.

Observed Likelihood Contributions

For an observed outcome $y$, the joint contribution for observing both $Y=y$ and $R=1$ is the outcome density multiplied by the selection probability. For a dropout, the outcome is not observed, so its contribution is the integral of the outcome density times the conditional dropout probability.

L(R=1,Y=y)=\frac{1}{\sigma}\phi\!\left(\frac{y-\mu}{\sigma}\right)\Phi(\alpha+\gamma y), \qquad L(R=0)=\int_{-\infty}^{\infty} \frac{1}{\sigma}\phi\!\left(\frac{y-\mu}{\sigma}\right) \Phi(-\alpha-\gamma y)\,dy .

Here $\phi$ is the standard normal density. The dropout integral has a closed form in this model. To see why, let $Z$ be an independent standard normal variable. Conditional dropout can be represented as the event $Z\leq-\alpha-\gamma Y$. Thus the marginal dropout probability is $P(Z+\gamma Y\leq-\alpha)$. Since $Z+\gamma Y$ is normal with mean $\gamma\mu$ and variance $1+\gamma^2\sigma^2$, standardizing gives the result below.

Integrated dropout contribution: For the normal outcome and probit selection model, $$ L(R=0)= \Phi\!\left(\frac{-\alpha-\gamma\mu} {\sqrt{1+\gamma^2\sigma^2}}\right). $$ The unobserved outcome is integrated out; this contribution is not the conditional dropout probability at a single chosen value of $Y$.

Worked Example: Evaluating Both Types of Contribution

Consider hypothetical parameter values $\mu=0$, $\sigma=1$, $\alpha=-0.2$, and $\gamma=0.8$. First consider an observed value $y=0.5$. Its selection predictor is $-0.2+0.8(0.5)=0.2$, so its likelihood contribution is

L(R=1,Y=0.5)=\phi(0.5;0,1)\Phi(0.2) \approx 0.3521(0.5793)=0.2039.

The likelihood is a density contribution, so it is not itself a probability for an exact continuous outcome. For a dropout, the integrated contribution is

L(R=0)= \Phi\!\left(\frac{-(-0.2)-0.8(0)}{\sqrt{1+0.8^2}}\right) =\Phi(0.1562)\approx 0.5621.

The two terms differ because the observed subject contributes the selection probability at the recorded outcome, whereas the dropout contributes an average over the outcome distribution. The integrated value also depends on $\mu$, even though no outcome was recorded for that subject.

Case-Weight Influence and the Integrated Score

Write the observed-data log likelihood as $\ell(\mu)=\sum_i\ell_i(\mu)$, with one subject-level contribution per subject. For an observed subject, $\ell_i$ is the logarithm of the observed contribution; for a dropout, it is the logarithm of the integrated contribution. Define the subject score $s_i=\partial\ell_i/\partial\mu$, the total score $U=\sum_i s_i$, and the observed Hessian $H=\partial^2\ell/\partial\mu^2$, all evaluated at the fitted value when used for a local diagnostic.

For an observed outcome $y_i$, with the selection parameters fixed, the selection factor does not depend on $\mu$. Its score and second derivative are therefore $s_i=(y_i-\mu)/\sigma^2$ and $-1/\sigma^2$. A dropout has a different score because its likelihood contribution is integrated. Let $s=\sqrt{1+\gamma^2\sigma^2}$, $z=(-\alpha-\gamma\mu)/s$, and $\lambda(z)=\phi(z)/\Phi(z)$. Differentiating the log of the integrated contribution gives

s_{\mathrm{drop}}=-\frac{\gamma}{s}\lambda(z), \qquad \frac{\partial^2\ell_{\mathrm{drop}}}{\partial\mu^2} =-\frac{\gamma^2}{s^2}\lambda(z)\{z+\lambda(z)\}.

These expressions show why omitting the dropout term, or treating a dropout as if its outcome were known, changes the score and curvature used for influence. The contribution of every subject must be derived from that subject’s observed-data likelihood.

To perturb subject weights, use $\ell(\mu;\varepsilon)=\sum_i(1+\varepsilon d_i)\ell_i(\mu)$, where $d_i$ specifies the direction of the weight change. Let $\hat\mu(\varepsilon)$ solve the perturbed score equation. Differentiating that equation at $\varepsilon=0$ yields the local response.

\left.\frac{d\hat\mu(\varepsilon)}{d\varepsilon}\right|_{\varepsilon=0} =-H^{-1}\sum_i d_i s_i .

In particular, if only subject $i$'s weight increases, then $d_i=1$ and all other $d_j=0$, so the response is $-H^{-1}s_i$. A positive response means the fitted $\mu$ increases locally as the specified weight increases; it is not a finite case-deletion result. The formula requires $H\neq0$ and a differentiable score equation near the fitted value.

Worked Example: Comparing a Case-Weight Direction

Use the hypothetical model with $\sigma=1$, $\alpha=-0.2$, $\gamma=0.8$, and a fitted value $\hat\mu=0.2$. Suppose the observed outcomes are $0.3000$ and $0.5861$, and one other subject has dropped out. The observed scores are $0.3000-0.2=0.1000$ and $0.5861-0.2=0.3861$. For the dropout, $s=\sqrt{1.64}=1.2806$ and $z=(-\alpha-\gamma\mu)/s=(-(-0.2)-0.8(0.2))/1.2806=0.04/1.2806=0.03123$. Hence $\lambda(z)\approx0.7781$, and

s_{\mathrm{drop}}=-\frac{0.8}{1.2806}(0.7781)\approx-0.4861.

The total score is approximately $0.1000+0.3861-0.4861=0.0000$, consistent, to rounding, with the stated fitted value. The observed subjects each contribute $-1$ to $H$. The dropout contributes

-\frac{0.8^2}{1.64}\lambda(z)\{z+\lambda(z)\} \approx -0.2458, \qquad H\approx-2.2458.

Increasing the weight of the observed subject with outcome $0.5861$ gives the local response

-\frac{0.3861}{-2.2458}\approx 0.1719.

Thus the fitted mean responds upward by about $0.1719$ outcome units per unit increase in that subject’s weight, locally at the fitted model. Increasing the dropout’s weight instead gives $-(-0.4861)/(-2.2458)\approx-0.2165$ outcome units per unit of weight change. The dropout affects this direction through its integrated score, not through a hypothetical completed outcome.

Perturbing an Observed Outcome

An outcome perturbation changes a recorded value while retaining the same selection model. For a specified observed subject $i$, write $y_i(\varepsilon)=y_i+\varepsilon a_i$, where $a_i$ is the perturbation direction in outcome units per unit of $\varepsilon$. The local response follows by differentiating the score equation with respect to $\varepsilon$:

\left.\frac{d\hat\mu}{d\varepsilon}\right|_{\varepsilon=0} =-H^{-1}\frac{\partial U}{\partial\varepsilon}.

In this scalar model, the observed subject’s score is $(y_i-\mu)/\sigma^2$, so $\partial U/\partial\varepsilon=a_i/\sigma^2$ when only that outcome changes. The selection factor $\Phi(\alpha+\gamma y_i)$ changes the subject’s likelihood value, but with $\alpha,\gamma$ fixed it has no $\mu$-score contribution. The dropout score and curvature remain in $U$ and $H$; they are not removed just because a different subject’s observed outcome is perturbed.

Worked Example: A Small Outcome Perturbation

Continue with the fitted example above, so $H\approx-2.2458$. Perturb the observed outcome $0.3000$ by $y_1(\varepsilon)=0.3000+0.1\varepsilon$. Here $a_1=0.1$ outcome units per unit of $\varepsilon$, and $\sigma=1$, so $\partial U/\partial\varepsilon=0.1$. The local derivative is

\left.\frac{d\hat\mu}{d\varepsilon}\right|_0 =-\frac{0.1}{-2.2458}\approx0.04453.

For $\varepsilon=0.01$, the outcome moves by $0.001$ units. The first-order approximation predicts a fitted-mean change of $0.04453(0.01)=0.0004453$ outcome units. This is a local approximation, not an exact refit. In particular, the integrated dropout contribution still supplies about $-0.2458$ of the curvature in the calculation.

Diagnostics, Alternatives, and Limits

A useful influence calculation is conditional on the model and on the chosen perturbation. It should be checked against the likelihood and its numerical implementation, especially when the dropout contribution is evaluated by numerical integration rather than the closed form available here.

Check the observed-data contribution. Confirm that observed outcomes use the density times $\Phi(\alpha+\gamma y)$, while dropouts use an integral or its analytically equivalent expression. Substituting a guessed missing value into the observed contribution is not an observed-data likelihood calculation.
Check derivatives. Compare analytic scores and Hessians with finite differences of the same log likelihood. Large discrepancies can indicate a sign error, a missing integrated term, or inaccurate integration.
Check local validity. If $H$ is close to zero, the inverse curvature can produce a very large response. Such a value may indicate weak local identification or a poor quadratic approximation, not necessarily a uniquely influential subject.
Check scale and direction. A case-weight change is dimensionless, while an outcome perturbation has outcome units. Their response magnitudes cannot be compared without specifying the perturbation sizes and units.

Case deletion, case weighting, and outcome perturbation answer related but different questions. Exact case deletion refits without a subject and can reveal departures from the local approximation, but it is more computationally demanding. A case-weight derivative describes a local change in the fitted estimator as a subject’s likelihood contribution is reweighted. An outcome perturbation asks how the fit responds to changing a recorded value. None of these, by itself, assesses whether the assumed outcome distribution or selection equation is appropriate.

A common mistake is to evaluate a dropout probability at the mean outcome and treat it as the dropout likelihood contribution. In general, $\Phi(-\alpha-\gamma\mu)$ is not the same as the integrated probability $\Phi\{(-\alpha-\gamma\mu)/\sqrt{1+\gamma^2\sigma^2}\}$. The extra denominator accounts for uncertainty in the outcome being integrated out. Another mistake is to apply a single-subject weight formula to a direction that changes several weights: the correct numerator is then the sum $\sum_i d_i s_i$. These diagnostics concern the local calculation; they do not resolve the broader sensitivity of conclusions to alternative MNAR assumptions.

Key takeaway: Under outcome-dependent dropout, a dropout contributes an integrated observed-data likelihood term. Its score and curvature must be included in case-weight and outcome-perturbation calculations, alongside the contributions from subjects with observed outcomes.

Check Your Understanding

Answer each question using the model and definitions above. Brief solutions follow the questions.

For $\mu=0,\sigma=1,\alpha=-0.2,\gamma=0.8$, what is the integrated dropout contribution, to four decimal places?
Why is evaluating the conditional dropout probability at $Y=\mu$ not generally equivalent to integrating over $Y$?
If a perturbation changes two subject weights by directions $d_1$ and $d_2$, what score term appears in the local response?
With $H=-2$, what is the local response to a single-subject weight increase when that subject’s score is $0.3$?
In the scalar model with fixed selection parameters, what is the derivative of the observed subject’s score with respect to its outcome?

Solutions. (1) The integrated contribution is $\Phi(0.2/\sqrt{1.64})=\Phi(0.1562)\approx0.5621$. (2) The conditional probability is nonlinear in $Y$; integration averages that probability over the outcome distribution and gives the denominator $\sqrt{1+\gamma^2\sigma^2}$. (3) The score term is $d_1s_1+d_2s_2$, so the response is $-H^{-1}(d_1s_1+d_2s_2)$, provided the relevant derivatives exist and $H\neq0$. (4) The response is $-(-2)^{-1}(0.3)=0.15$ outcome units per unit weight change. (5) Since the score is $(y-\mu)/\sigma^2$, its outcome derivative is $1/\sigma^2$; it equals $1$ when $\sigma=1$.