Louis' Identity for Observed Information in Missing-Data Models

Why observed information needs a correction

In a selection model, the likelihood for an observed record can involve an integral over an outcome that was not recorded. The observed score is therefore not obtained by inserting a guessed value for that outcome. It averages the complete-data score over the model-based conditional distribution of the missing outcome, given what was observed. The observed information also depends on how much that complete-data score varies across the possible missing outcomes.

Louis’ identity makes this relationship explicit. It is useful when complete-data derivatives are manageable but the observed-data likelihood contains latent outcomes, as in selection models. The identity can be applied after evaluating or approximating the relevant conditional expectations; it does not remove the need to calculate the distribution of the missing outcomes correctly.

Definition: Let $O$ denote the observed data, $Z$ the latent or missing data, and $X=(O,Z)$ the complete data. Write the complete-data log likelihood as $\ell_c(\theta;O,Z)$, where $\theta$ is a parameter vector. The complete-data score is $S_c(\theta)=\nabla_\theta\ell_c(\theta;O,Z)$, and the complete-data observed information is $I_c(\theta)=-\nabla_\theta^2\ell_c(\theta;O,Z)$. The observed-data log likelihood is $\ell_o(\theta;O)=\log f_o(O;\theta)$. Its observed score is $\nabla_\theta\ell_o$, and its observed information is $I_o(\theta)=-\nabla_\theta^2\ell_o(\theta;O)$.

Louis’ identity and its assumptions

The identity concerns derivatives of the likelihood for a fixed observed record. Assume that the complete-data density or mass function is differentiable in $\theta$, that the observed likelihood is obtained by integrating or summing over the latent data, and that differentiation can be interchanged with that integration or summation. In applications, these regularity conditions require suitable integrability and a parameter-independent support, or other conditions that justify the differentiation. The conditional expectations below are taken under the model at the parameter value being evaluated.

Theorem: Under these conditions, with all conditional expectations and covariances evaluated under the complete-data model conditional on $O$, $$ \nabla_\theta\ell_o(\theta;O)=E\{S_c(\theta)\mid O\}, \qquad I_o(\theta)=E\{I_c(\theta)\mid O\} -\operatorname{Cov}\{S_c(\theta)\mid O\}. $$ For a vector score, $\operatorname{Cov}(S_c\mid O)=E[(S_c-E[S_c\mid O])(S_c-E[S_c\mid O])^T\mid O]$.

The information identity contains two terms. The first averages complete-data information across possible latent values. The second subtracts the conditional covariance of the complete-data score. That covariance is positive semidefinite, so uncertainty about the latent data reduces the averaged complete-data information in the matrix ordering. The resulting observed information for a particular record need not be positive: the second term can exceed the first in some direction.

Derivation

For fixed $O$, write the observed-data likelihood as an integral of the complete-data likelihood over $Z$. The same argument applies to a sum when $Z$ is discrete. The conditional density $f(z\mid O;\theta)$ is formed from the joint complete-data model, not from an arbitrary distribution for the missing outcome.

f_o(O;\theta)=\int f_c(O,z;\theta)\,dz,\qquad \nabla_\theta f_c=f_c S_c.

Differentiating the observed likelihood and dividing by $f_o(O;\theta)$ gives the score identity. Differentiating once more uses $\nabla_\theta^2 f_c=f_c\{H_c+S_cS_c^T\}$, where $H_c=\nabla_\theta^2\ell_c$. Subtracting the outer product of the observed score from the second derivative of the likelihood gives the Hessian of its logarithm.

\begin{aligned} \nabla_\theta\ell_o &=\frac{\int f_c S_c\,dz}{f_o} =E(S_c\mid O),\\ \nabla_\theta^2\ell_o &=E(H_c+S_cS_c^T\mid O) -E(S_c\mid O)E(S_c\mid O)^T\\ &=E(H_c\mid O)+\operatorname{Cov}(S_c\mid O). \end{aligned}

Since $I_c=-H_c$ and $I_o=-\nabla_\theta^2\ell_o$, changing signs in the final line yields Louis’ information identity. The covariance term is not an optional adjustment: it accounts for the curvature contributed by uncertainty about which complete-data score would have been observed.

Worked calculations in a selection model

Consider a binary outcome $Y\in\{0,1\}$ with $P(Y=1)=p$, parameterized by $\theta=\operatorname{logit}(p)$. Let $R=1$ mean the outcome is observed and $R=0$ mean it is missing. Suppose the response probabilities conditional on $Y$ are specified and known for these calculations. Write $r_y=P(R=0\mid Y=y)$. The complete-data likelihood includes both the outcome model and the response mechanism. Because the response mechanism is fixed here, its log likelihood has no $\theta$-derivative; the complete-data score for $\theta$ is $Y-p$, and its complete-data information is $p(1-p)$.

Worked Example: One Missing Binary Outcome

Suppose $p=0.4$, $r_0=0.8$, and $r_1=0.2$. For a record with $R=0$, calculate the observed score and observed information for $\theta$.

The observed likelihood contribution is the probability of nonresponse, summing over the two possible outcomes: $q=P(R=0)=(1-p)r_0+pr_1=(0.6)(0.8)+(0.4)(0.2)=0.56$. Bayes’ rule gives the conditional probability that the missing outcome is 1: $w=P(Y=1\mid R=0)=pr_1/q=(0.4)(0.2)/0.56=1/7$.

The complete-data score is $Y-p$. Its conditional mean, and hence the observed score, is $E(Y-p\mid R=0)=1/7-0.4=-0.2571$. As a direct check, $q=0.8-0.6p$, so $dq/d\theta=(dq/dp)(dp/d\theta)=(-0.6)(0.24)=-0.144$, and $d\log(q)/d\theta=-0.144/0.56=-0.2571$.

The conditional expected complete-data information is $p(1-p)=0.24$. The conditional variance of the score equals the conditional variance of $Y$: $\operatorname{Var}(Y-p\mid R=0)=w(1-w)=(1/7)(6/7)=6/49=0.122449$. Thus

I_o=0.2400-\frac{6}{49}=0.117551\approx 0.1176.

The observed information for this record is about $0.1176$ on the logit parameter scale. The calculation uses the unrounded conditional variance; subtracting a rounded variance would give a correspondingly rounded result.

Worked Example: A Record With Negative Observed Information

Let $p=0.1$, $r_0=0.1$, and $r_1=0.9$. Suppose a record has $R=0$. Calculate its observed information and explain its sign.

The probability of nonresponse is $q=(0.9)(0.1)+(0.1)(0.9)=0.18$. Therefore $P(Y=1\mid R=0)=(0.1)(0.9)/0.18=0.5$. The complete-data information is $p(1-p)=(0.1)(0.9)=0.09$, while the conditional variance of the complete-data score is $0.5(1-0.5)=0.25$. Louis’ identity gives

$$ I_o=0.09-0.25=-0.16. $$

Negative observed information means the log likelihood for this realized record is locally convex, rather than locally concave, in $\theta$ at this parameter value. It does not contradict the identity or imply that the response probabilities are invalid. Indeed, the observed score is $0.5-0.1=0.4$, and direct differentiation of $\log\{0.1+0.8p\}$ with respect to $\theta$ gives a second derivative of $0.16$, so its negative is $-0.16$, as calculated.

Worked Example: Combining an Incomplete and a Complete Record

Use $p=0.4$, $r_0=0.8$, and $r_1=0.2$. In an independent sample, one record has $R=0$, and a second has $R=1,Y=1$. Find the total observed score and information for $\theta$.

For the missing-outcome record, the calculations in the first example give score $-0.257143$ and information $0.117551$. For the second record, the likelihood contribution is $pP(R=1\mid Y=1)=0.8p$. Its score is $1-p=0.6$, and its information is $p(1-p)=0.24$. Here the outcome is observed, so the conditional distribution of the latent outcome is degenerate and the score covariance term is zero.

Independent record log likelihoods add, so their scores and observed information contributions add:

S_o=-0.257143+0.6000=0.342857,\qquad I_o=0.117551+0.2400=0.357551.

The sample’s observed information is approximately $0.3576$ on the logit scale. This is the curvature of the combined observed-data log likelihood at $p=0.4$, not an expected information averaged over every possible sample.

Applying the identity and checking calculations

For a selection-model likelihood, the latent-data distribution required by Louis’ identity is the distribution of the missing outcome conditional on the observed record, including its response indicator and any observed covariates. If the response probability depends on the outcome, that dependence changes the conditional distribution. Averaging only under the outcome model generally gives the wrong score and information.

A practical calculation follows a short sequence: specify the complete-data likelihood; derive its score and Hessian; determine the conditional distribution of the missing data given the observed record; and evaluate the conditional score mean, information mean, and score covariance. For independent records, calculate each contribution and add them. If the conditional distribution is evaluated by quadrature or Monte Carlo integration, the expectations and covariance must be computed under the same conditional distribution and at the same parameter value.

When feasible, direct differentiation of the observed likelihood provides a valuable check. The observed score from Louis’ identity should match the derivative of the integrated log likelihood, and the observed information should match the negative second derivative. Disagreement can reveal a sign error, an incorrect conditional distribution, a missing response-model term in the complete-data score, or inaccurate numerical integration. For vector parameters, retain the full covariance matrix, including its off-diagonal entries; calculating only componentwise variances does not recover the information matrix.

Calculation	What to use	Frequent error
Conditional expectation	The model-based distribution of latent data given the full observed record	Averaging over missing outcomes under their marginal distribution alone
Information correction	Subtract $\operatorname{Cov}(S_c\mid O)$ from $E(I_c\mid O)$	Adding the score covariance or omitting it
Response-model derivatives	Include them in $S_c$ and $I_c$ whenever the response model depends on $\theta$	Treating a parameter-dependent response mechanism as fixed
Multiple records	Add log-likelihood, score, and information contributions under independence	Adding conditional variances across records without respecting the likelihood factorization

Common pitfall and limitations

A common mistake is to argue that the observed information must be positive because complete-data information is positive. That conclusion ignores the subtracted conditional score covariance. The second worked example shows a valid record for which the observed log likelihood is locally convex. Conversely, a positive observed information value is local curvature, not a guarantee that the likelihood is globally concave or that a numerical calculation is correct.

Louis’ identity is an exact identity under its regularity conditions, but it is not a substitute for specifying a credible selection model. If the model or conditional distribution of the latent outcomes is wrong, the resulting observed information is also wrong. Nor does the identity ensure that an integral has been evaluated accurately: numerical approximation must be checked, especially when conditional distributions are concentrated or have substantial tails. The identity provides a way to organize and verify derivatives; it does not resolve the substantive uncertainty inherent in unobserved outcomes.

Key takeaway: The observed score is the conditional mean of the complete-data score. The observed information is the conditional mean of complete-data information minus the conditional covariance of the complete-data score. Missing-outcome uncertainty therefore affects curvature as well as the score.

Check Your Understanding

Answer each question, then check the solutions below.

State Louis’ identity for a scalar parameter, using complete-data score $S_c$ and information $I_c$.
For a record with observed data $O$, should the missing outcome be averaged over its marginal distribution or its distribution conditional on $O$?
In the first worked example, what is the conditional variance term, and why is it subtracted?
Can observed information for a particular record be negative even when complete-data information is positive? Explain briefly.
If two independent records have observed information contributions $0.12$ and $0.31$, what is the total contribution, and what does it represent?

Solutions. (1) $I_o=E(I_c\mid O)-\operatorname{Var}(S_c\mid O)$. (2) Use the model-based distribution conditional on the full observed record, including the response indicator. (3) It is $6/49=0.122449$; it is subtracted because the log-likelihood Hessian contains the conditional score covariance with a positive sign, while information is the negative Hessian. (4) Yes. The conditional score variance can exceed the conditional expected complete-data information, making their difference negative; the second worked example gives $-0.16$. (5) The total is $0.12+0.31=0.43$, the observed information for the combined independent-record log likelihood at the parameter value used.