Deriving Scores with Missing Outcomes

Why Missing Outcomes Change the Score

In a selection model, the probability that an outcome is observed may depend on the outcome itself. Consequently, a subject whose outcome is missing still contributes information about the model parameters: the observed dropout pattern is part of the likelihood. But the complete-data score for that subject depends on an outcome we did not see. The relevant score contribution is obtained by averaging that complete-data score over the model-based distribution of the missing outcome, conditional on what the subject’s record does show.

This tutorial derives that relationship and applies it to outcome and response parameters. It concerns score contributions, not observed information or the curvature of the likelihood.

Likelihood and Score with Incomplete Outcomes

For subject $i$, let $Y_i$ denote the complete outcome vector and $R_i$ the response pattern. Write $Y_{i,\mathrm{obs}}$ for the observed components and $Y_{i,\mathrm{mis}}$ for the missing components. The observed record is $O_i=(Y_{i,\mathrm{obs}},R_i)$. Let the joint complete-data model have density or mass function $p_\lambda(y_i,r_i)=f_\theta(y_i)g_\psi(r_i\mid y_i)$, where $\lambda$ collects all unknown parameters. The outcome parameters are denoted by $\theta$, and the response parameters by $\psi$; the two parameter sets may be distinct or may share components.

Definition: The complete-data score is $U_{c,i}(\lambda;y_i,r_i)=\partial\log p_\lambda(y_i,r_i)/\partial\lambda$. The observed-data score is the derivative of the log likelihood contribution for the record actually observed. It is not, in general, the complete-data score evaluated at a filled-in outcome.

For a continuous missing component, the individual observed-data likelihood contribution is

L_i(\lambda) = \int p_\lambda(y_{i,\mathrm{obs}},y_{i,\mathrm{mis}},r_i)\, d y_{i,\mathrm{mis}}.

For discrete missing outcomes, the integral is replaced by a sum. When all outcomes are observed, there is no integration and the observed-data score equals the complete-data score at the observed values. When some outcomes are missing, the likelihood integrates over their possible values while retaining the observed response pattern.

The Conditional-Expectation Score Identity

Assume the complete-data model has a common support that does not depend on $\lambda$, the likelihood is differentiable in $\lambda$, and differentiation can be passed under the integral. Also assume the relevant derivatives are integrable, so the integral of the derivative exists. These regularity conditions allow the derivative of the observed likelihood to be calculated from the complete-data likelihood.

Theorem: Under these conditions, the observed-data score for subject $i$ is the conditional expectation of the complete-data score given the observed record: $$ U_{o,i}(\lambda;O_i) = \frac{\partial\log L_i(\lambda)}{\partial\lambda} = E_\lambda\!\left[ U_{c,i}(\lambda;Y_i,R_i)\mid O_i \right]. $$ The conditional distribution in this expectation is induced by the joint model for outcomes and response patterns.

To justify the identity, differentiate the observed likelihood contribution:

\begin{aligned} \frac{\partial L_i(\lambda)}{\partial\lambda} &= \int \frac{\partial p_\lambda(y_{i,\mathrm{obs}},y_{i,\mathrm{mis}},r_i)} {\partial\lambda} \,d y_{i,\mathrm{mis}}\\ &= \int p_\lambda(y_{i,\mathrm{obs}},y_{i,\mathrm{mis}},r_i) U_{c,i}(\lambda;y_i,r_i)\,d y_{i,\mathrm{mis}}. \end{aligned}

Dividing by $L_i(\lambda)$ gives the derivative of its logarithm. The ratio of the joint density in the integrand to $L_i(\lambda)$ is precisely the conditional density of the missing outcomes given the observed record. Therefore the ratio is the conditional expectation in the theorem. For independent subjects, the full observed log likelihood is the sum of the individual log likelihood contributions, and the full score is the sum of their observed-data scores.

When the joint model factors as $f_\theta(y_i)g_\psi(r_i\mid y_i)$, the complete-data score has an outcome component and a response component:

U_{c,i} = \frac{\partial\log f_\theta(Y_i)}{\partial\lambda} + \frac{\partial\log g_\psi(R_i\mid Y_i)}{\partial\lambda}.

For disjoint parameter sets, the first term contributes to the $\theta$ score and the second to the $\psi$ score. The response score may still involve the unobserved outcome because response probabilities can depend on it. Both components are averaged over the same conditional distribution of missing outcomes given the entire observed record.

Worked Score Calculations

Worked Example: A Missing Binary Outcome and a Logistic Selection Model

Suppose $Y$ is binary with $P_\theta(Y=1)=p=0.4$. Given $Y=y$, let $R=1$ indicate that the outcome is observed, with $\operatorname{logit}P_\psi(R=1\mid Y=y)=\alpha+\beta y$. Consider a subject with $R=0$. Calculate the observed score contributions for $\alpha$, $\beta$, and the outcome logit $\theta=\operatorname{logit}(p)$, at $\alpha=-1$ and $\beta=1$.

For this model, the complete-data selection score is $(R-\pi(Y))(1,Y)$, where $\pi(y)=P(R=1\mid Y=y)$. The complete-data outcome score for $\theta$ is $Y-p$. Since $R=0$, the selection score for a known $Y=y$ would be $-\pi(y)(1,y)$. Here $Y$ is unknown, so first calculate its conditional distribution given dropout.

The two response probabilities are $\pi(0)=\operatorname{expit}(-1)=0.2689$ and $\pi(1)=\operatorname{expit}(0)=0.5000$. Thus

$$ P(R=0) =(1-0.4)(1-0.2689)+0.4(1-0.5000) =0.6386. $$

Bayes’ rule gives $P(Y=1\mid R=0)=0.4(0.5)/0.6386=0.3132$, and hence $P(Y=0\mid R=0)=0.6868$. The response-parameter scores are conditional expectations:

\begin{aligned} U_{o,\alpha} &=-\{0.6868(0.2689)+0.3132(0.5000)\} =-0.3413,\\ U_{o,\beta} &=0.6868(0)+0.3132\{-0.5000(1)\} =-0.1566. \end{aligned}

The outcome score is $U_{o,\theta}=E(Y-p\mid R=0)=0.3132-0.4=-0.0868$. These are score contributions for one dropout record at the stated parameter values; their signs do not, by themselves, indicate whether an estimate should increase or decrease after considering the other subjects’ contributions.

Worked Example: A Fully Observed Outcome

Let $Y$ have a normal outcome model with mean $\mu$ and known variance $\sigma^2=0.25$. The response model is logistic, with $\operatorname{logit}P(R=1\mid Y=y)=\alpha+\beta y$. A subject has $Y=1.4$ observed, and evaluate the scores at $\mu=1$, $\alpha=-1$, and $\beta=0.4$.

Because $Y$ is observed, there is no conditional averaging over it. The normal outcome score for $\mu$ is $(y-\mu)/\sigma^2$, so $U_{o,\mu}=(1.4-1)/0.25=1.600$ per unit of $Y$.

The response linear predictor is $\eta=-1+0.4(1.4)=-0.4400$, giving $\pi(1.4)=\operatorname{expit}(-0.44)=0.3917$. Since $R=1$, the logistic response score is $(1-\pi)(1,y)$. Therefore

U_{o,\alpha}=1-0.3917=0.6083, \qquad U_{o,\beta}=(1-0.3917)(1.4)=0.8516.

The $\beta$ score is per unit of the outcome because $\beta$ multiplies $Y$ in the log odds. This example is a useful check on the identity: when the record contains the outcome, the conditional distribution of $Y$ given the record is concentrated at its observed value, so the conditional expectation reduces to the complete-data score.

Worked Example: A Monotone Dropout Pattern

Suppose $Y_1$ is observed and $Y_2$ is observed only if the subject remains in the study at the second visit. Conditional on $Y_1=0$, let $P(Y_2=1)=q=0.3$. The probability of observing visit 2 is $\pi(y_2)=\operatorname{expit}(-1+y_2)$. For a subject with $Y_1=0$ who drops out before visit 2, calculate the response coefficient score for the coefficient of $Y_2$, at $\beta=1$, and the outcome logit score for $Y_2$.

The observed record includes $Y_1=0$ and dropout at visit 2, but not $Y_2$. As in the first example, $\pi(0)=0.2689$ and $\pi(1)=0.5000$. The probability of this dropout pattern, conditional on $Y_1=0$, is

P(R_2=0\mid Y_1=0) =0.7(1-0.2689)+0.3(1-0.5000) =0.6617.

The conditional probability of $Y_2=1$ for this record is $0.3(0.5)/0.6617=0.2266$; the probability of $Y_2=0$ is $0.7734$. The complete-data response score for $\beta$, the coefficient of $Y_2$, is $-\pi(Y_2)Y_2$ when $R_2=0$. Thus the observed score is $0.2266(-0.5)+0.7734(0)=-0.1133$. If the conditional outcome model is parameterized by $\operatorname{logit}(q)$, its complete-data score is $Y_2-q$. The observed score is $0.2266-0.3=-0.0734$. Both use the distribution of $Y_2$ conditional on the observed history and the dropout pattern, not merely its distribution conditional on the history before dropout.

Using the Identity in Practice

For a selection model with several outcomes or visits, the same calculation applies to the whole missing vector. If dropout is monotone, the response-pattern likelihood may be written as a product of visit-specific response or dropout probabilities conditional on the relevant history. The complete-data response score is then the derivative of the log of that product. For a subject who drops out, terms involving unobserved future outcomes remain inside the conditional expectation; terms involving fully observed history can be evaluated directly.

Write the complete-data model and score.
Include both the outcome density and the response-pattern probability. Differentiate their joint log likelihood with respect to the parameters of interest.

Form the missing-outcome conditional distribution.
Condition on the observed outcomes and the response pattern under the joint model. The response mechanism matters when it depends on missing outcomes.

Average the complete-data score.
Compute its conditional expectation, by summation, integration, quadrature, or Monte Carlo approximation as appropriate.

Sum subject contributions.
Add the observed-data scores across independent subjects to obtain the full score. At an interior maximum-likelihood estimate, the total score is zero.

The conditional distribution in Step 2 is proportional to the joint model evaluated at the observed pattern: $p_\lambda(y_{\mathrm{obs}},y_{\mathrm{mis}},r)$. Normalizing this expression over the missing outcomes gives the required conditional distribution. Using only $f_\theta(y_{\mathrm{mis}}\mid y_{\mathrm{obs}})$ is generally wrong when $g_\psi(r\mid y)$ depends on the missing outcomes.

For a low-dimensional missing vector, direct quadrature may be accurate and convenient. Discrete outcomes can be summed exactly when the state space is manageable. Monte Carlo methods can approximate the conditional expectation in more complex settings, but the approximation must represent the conditional distribution given the response pattern. Checking that conditional weights sum to one and comparing numerical scores with finite differences of the observed log likelihood are useful diagnostics.

Common Mistakes, Alternatives, and Limits

Plugging in a single imputed value. The score is an average over the conditional distribution of missing outcomes, not generally the score at a conditional mean or at one imputed draw. Nonlinear score terms make those quantities differ.
Ignoring the response pattern when conditioning. If response depends on the outcome, conditioning only on observed outcomes omits part of the information in the record. Use the joint model for outcomes and response.
Using the wrong logistic score sign. For a logistic response probability $\pi(y)$, the score for a linear predictor coefficient is $(R-\pi(y))$ times its covariate. It is negative $\pi(y)$ for a dropout record $R=0$, and $1-\pi(y)$ for an observed record $R=1$.
Treating a score as a test or an estimate. A single individual score need not be zero. The zero-score equation applies to the sum across subjects at an interior likelihood maximum, and does not by itself guarantee a unique or well-identified solution.

Direct differentiation of the observed-data likelihood is the main alternative way to obtain the score; it should agree with the conditional-expectation calculation. The identity can be computationally useful when complete-data derivatives are simple but the observed likelihood requires integration. EM algorithms also use conditional expectations of complete-data quantities, but the EM objective is an expected complete-data log likelihood evaluated using a distribution from the current iterate. Its maximization step is not, in general, the same operation as setting the observed score to zero at the new parameter value.

The identity is exact under its regularity conditions, but it does not remove the need to specify and evaluate the selection model. Numerical integration can be inaccurate, especially when the conditional distribution is concentrated in a narrow region or has multiple separated regions of appreciable mass. Moreover, a correctly computed score does not establish that the parameters are well identified or that the selection model is substantively appropriate. Those are separate modeling and inferential questions.

Key takeaway: For incomplete outcomes, differentiate the observed likelihood by averaging the complete-data score over the model-based distribution of missing outcomes conditional on the full observed record, including the response pattern.

Check Your Understanding

Answer each question, then compare with the short solutions below.

State the conditional-expectation identity for an observed-data score and name the conditioning information.
For a logistic response model with $\operatorname{logit}\pi(y)=\alpha+\beta y$, what is the complete-data score for $\beta$ when $R=0$ and $Y=y$ is known?
Why is $f_\theta(Y_{\mathrm{mis}}\mid Y_{\mathrm{obs}})$ generally not the correct distribution for averaging a score in a selection model?
If all outcomes are observed, what does the conditional-expectation identity reduce to?
Name one numerical check for a computed observed-data score.

Solutions. (1) $U_o=E_\lambda(U_c\mid O)$, where $O$ includes the observed outcomes and response pattern. (2) The score is $(R-\pi(y))y=-\pi(y)y$. (3) The response pattern can depend on missing outcomes, so its likelihood contribution changes their conditional distribution; use the joint model conditional on the full observed record. (4) The conditional distribution of the complete outcome vector is concentrated at the observed values, so the observed score equals the complete-data score there. (5) Compare it with a finite-difference derivative of the observed log likelihood, or check that conditional integration weights normalize to one.