Integrating Random Effects in the Observed-Data Likelihood

Why the Random Effect Must Be Integrated

A shared-parameter model describes the outcome process and the dropout process using a common subject-specific latent effect. Conditional on that effect, the two processes may be independent; across subjects, however, the effect is not observed. The likelihood for the data that were actually observed must therefore average the joint conditional contribution over the effect’s distribution.

This averaging is more than a technical step. Subjects with different latent effects can have different outcome distributions and different dropout patterns. The observed-data likelihood combines those two sources of information about the same latent effect. In general, it cannot be obtained by separately averaging the outcome term and the dropout term and then multiplying the averages.

Definition: For subject $i$, let $b_i$ be a subject-specific random effect with density or probability mass function $f_b(b;\theta_b)$. Let $Y_i^{\mathrm{obs}}$ denote the observed outcome history and $D_i$ the observed dropout or follow-up information. Write $L_{Yi}(b;\theta_Y)$ for the conditional likelihood contribution of the observed outcomes, and $L_{Di}(b;\theta_D)$ for the conditional likelihood contribution of the dropout information. Under conditional independence of the outcome and dropout processes given $b_i$ and the included covariates, the subject-level observed-data likelihood is $$ L_i(\theta)=\int L_{Yi}(b;\theta_Y)L_{Di}(b;\theta_D)f_b(b;\theta_b)\,db, $$ where $\theta=(\theta_Y,\theta_D,\theta_b)$. For a discrete random effect, the integral is replaced by a sum.

The data in $L_{Yi}$ are only the outcomes observed for subject $i$. If an outcome after dropout is unobserved, its value is not inserted into this likelihood as though it were known. Instead, its possible values have already been accounted for by using the observed-outcome likelihood contribution. The dropout factor describes the observed dropout time, event, or censoring information under the specified model.

From Conditional Independence to the Marginal Likelihood

Suppose that, conditional on $b_i=b$, the observed outcome data and dropout data have likelihood contributions $L_{Yi}(b)$ and $L_{Di}(b)$. Conditional independence gives their joint contribution as the product $L_{Yi}(b)L_{Di}(b)$. Since $b_i$ is unobserved, the joint contribution to the observed-data likelihood is its average under the random-effect distribution:

L_i(\theta) =\operatorname{E}_{b_i}\!\left[L_{Yi}(b_i;\theta_Y)L_{Di}(b_i;\theta_D)\right] =\int L_{Yi}(b;\theta_Y)L_{Di}(b;\theta_D)f_b(b;\theta_b)\,db.

If subjects are independent conditional on the model parameters and covariates, the likelihood for all subjects is the product of these integrated contributions:

L(\theta)=\prod_{i=1}^{n}L_i(\theta), \qquad \ell(\theta)=\sum_{i=1}^{n}\log L_i(\theta).

The integral is taken separately for each subject because each subject has a distinct random effect. The population parameters are shared, but the latent $b_i$ is not. The integration also shows the source of marginal dependence. In general, $ \operatorname{E}[L_{Yi}(b_i)L_{Di}(b_i)]\ne \operatorname{E}[L_{Yi}(b_i)]\operatorname{E}[L_{Di}(b_i)] $. In fact, their difference is the covariance, under the random-effect distribution, between the two conditional likelihood contributions. When both contributions tend to be relatively large for the same values of $b_i$, the joint average can exceed the product of the separate averages.

Key result: Conditional independence given a shared random effect does not generally imply marginal independence after that effect is integrated out. The observed-data likelihood averages the product of the conditional contributions, not the product of their separate averages.

Worked Example: A Two-Point Shared Effect

Consider a hypothetical subject with a binary observed outcome $Y=1$ and a binary dropout indicator $D=1$. Let $b$ take values $-1$ and $1$, each with probability $0.5$. Suppose the outcome and dropout indicators are conditionally independent given $b$, with $P(Y=1\mid b=-1)=0.2$, $P(Y=1\mid b=1)=0.8$, $P(D=1\mid b=-1)=0.1$, and $P(D=1\mid b=1)=0.6$. These probabilities specify valid Bernoulli models, and conditional independence is the stated model assumption.

For the observed pair $Y=1,D=1$, the likelihood contribution at $b=-1$ is $0.2(0.1)=0.02$; at $b=1$, it is $0.8(0.6)=0.48$. Integrating over the two-point distribution gives

$$ L_i =0.5(0.2)(0.1)+0.5(0.8)(0.6) =0.010+0.240 =0.250. $$

By contrast, the separate marginal probabilities are $P(Y=1)=0.5(0.2)+0.5(0.8)=0.50$ and $P(D=1)=0.5(0.1)+0.5(0.6)=0.35$. Multiplying them gives $0.50(0.35)=0.175$, not $0.250$. The discrepancy occurs because the value $b=1$ makes both observed events more likely, so their conditional likelihood contributions vary together across the random-effect distribution.

What the Integral Includes—and What It Does Not

The integration over $b_i$ removes an unobserved subject-specific effect from the likelihood. It is not itself an imputation of missing outcomes, nor does it assert that outcomes after dropout were observed. The outcome factor is built from the observed outcome history; the dropout factor is built from the observed dropout or censoring information. The shared effect connects these factors before the averaging occurs.

A dropout contribution must represent the information recorded by the study. For example, it may distinguish an observed dropout event from continued follow-up or right censoring, according to the chosen dropout model. The general integration formula does not dictate a particular hazard model or time scale; it requires that the appropriate conditional likelihood contribution be specified first. If the dropout process depends directly on outcome information beyond what is represented by $b_i$, the simple conditional-independence factorization requires modification.

Worked Example: Combining an Outcome History and Dropout Information

Suppose a hypothetical subject has two observed binary outcomes, $Y_1=1$ and $Y_2=0$, followed by dropout in the next interval. Let $b$ be equally likely to equal $-1$ or $1$. Conditional on $b$, assume the two outcomes are independent Bernoulli observations and the dropout indicator is independent of those outcomes. At $b=-1$, let $P(Y_1=1\mid b)=0.6$, $P(Y_2=0\mid b)=0.7$, and the probability of dropout in the specified interval be $0.2$. At $b=1$, let these three probabilities be $0.3$, $0.4$, and $0.8$, respectively. These are valid probabilities for the stated conditional model.

The conditional likelihood for the observed outcome history is the product of the two Bernoulli probabilities. Thus it is $0.6(0.7)=0.42$ at $b=-1$, and $0.3(0.4)=0.12$ at $b=1$. Including the dropout contribution and integrating gives

$$ L_i =0.5(0.42)(0.2)+0.5(0.12)(0.8) =0.042+0.048 =0.090. $$

The separately averaged outcome-history contribution is $0.5(0.42)+0.5(0.12)=0.27$, and the separately averaged dropout probability is $0.5(0.2)+0.5(0.8)=0.50$. Their product is $0.135$, which differs from the correct joint contribution $0.090$. The calculation uses only the two observed outcomes; it does not assign values to any outcomes after dropout.

Continuous Random Effects and Numerical Integration

When $b_i$ has a continuous distribution, the same likelihood expression applies, but the integral may not have a closed form. It is still conceptually an expectation under the specified distribution of the random effect. Numerical integration approximates that expectation by evaluating the product of the conditional contributions at selected values of $b$ and combining those evaluations with appropriate weights.

Worked Example: A Three-Node Approximation

For illustration, let $b\sim N(0,1)$, and suppose the conditional outcome and dropout likelihood contributions at the three standard-normal quadrature nodes $-\sqrt{3},0,\sqrt{3}$ are as follows. The corresponding normalized weights for the three-node Gauss–Hermite approximation to an expectation under $N(0,1)$ are $1/6,2/3,1/6$. Assume the conditional factorization is appropriate for these hypothetical data. At the three nodes, respectively, take $L_{Yi}(b)$ to be $0.2,0.5,0.8$, and $L_{Di}(b)$ to be $0.1,0.4,0.7$.

The approximation is applied to the product at each node:

L_i \approx \frac{1}{6}(0.2)(0.1) +\frac{2}{3}(0.5)(0.4) +\frac{1}{6}(0.8)(0.7) =0.003333+0.1333+0.09333 =0.2300.

Separately averaging the factors using the same weights gives an outcome contribution of $ \frac{1}{6}(0.2)+\frac{2}{3}(0.5)+\frac{1}{6}(0.8)=0.5000 $ and a dropout contribution of $ \frac{1}{6}(0.1)+\frac{2}{3}(0.4)+\frac{1}{6}(0.7)=0.4000 $. Their product is $0.2000$, not the joint approximation $0.2300$. The values $0.2300$ and $0.2000$ are illustrative numerical approximations, not exact integrals; accuracy depends on how well the quadrature rule represents the integrand.

Assumptions, Checks, and Alternatives

The integration formula relies on several distinct assumptions. The random-effect distribution must be specified, including its support and parameters. The outcome and dropout likelihood factors must correctly describe the observed data conditional on the effect and included covariates. The conditional-independence factorization must be justified by the model, rather than inferred merely because a shared effect has been included. Finally, the product across subjects assumes independent subjects conditional on the population parameters and their covariates.

Useful checks focus on both the model and the computation:

Verify that each conditional factor is a likelihood for the data actually observed, including the correct dropout or censoring information.
Check that conditional probabilities and densities are valid and that the random-effect distribution is normalized.
For numerical integration, increase or otherwise vary integration accuracy and compare the resulting subject-level likelihoods or log likelihood. Material changes indicate inadequate numerical approximation.
Inspect whether the integrand is concentrated in regions that receive little numerical weight. A coarse approximation may miss important contributions, particularly when the conditional likelihood changes sharply with $b$.
Check that the intended dependence structure is plausible. If conditional independence is not assumed, the joint conditional likelihood must replace the product of the two factors.

The main alternative calculation is to form separate marginal outcome and dropout likelihoods and multiply them. That is valid only when the two conditional contributions have zero covariance under the distribution of $b_i$, a special condition that does not generally hold. A different modeling approach can instead specify a joint conditional distribution directly, without factorizing it into outcome and dropout terms; it still must be integrated over any unobserved shared effects. Pattern-mixture models provide a distinct way to organize assumptions by missingness pattern, but they do not change the integration required by the shared-parameter likelihood defined here.

A common mistake is to argue, “The processes are conditionally independent, so I can integrate each one separately and multiply.” The correction is that conditional independence gives a product inside the integral: $ \operatorname{E}[L_{Yi}(b_i)L_{Di}(b_i)] $. Separate integration gives $ \operatorname{E}[L_{Yi}(b_i)]\operatorname{E}[L_{Di}(b_i)] $, which is generally a different quantity. The distinction matters because the same latent effect can make a subject’s outcomes and dropout information jointly more or less compatible with the data.

Limitation: Integrating the random effect produces an observed-data likelihood conditional on the chosen outcome model, dropout model, random-effect distribution, and dependence assumptions. The integral does not establish that those assumptions are correct, and it cannot by itself determine what would have happened to outcomes after dropout.

Check Your Understanding

Use the likelihood definition above. Each question is followed by a concise solution so you can check both the calculation and its interpretation.

A shared effect takes two equally likely values. At the first value, the outcome and dropout likelihood contributions are $0.3$ and $0.2$; at the second, they are $0.7$ and $0.8$. Calculate the integrated joint contribution.
In Question 1, calculate the product of the separately integrated contributions. Is it equal to the joint contribution?
What does conditional independence given $b_i$ permit you to factor, and where must that factorization appear in the marginal likelihood?
If an outcome after dropout is not observed, should its realized value be inserted into $L_{Yi}$? Explain.

Solutions. (1) The contribution is $0.5(0.3)(0.2)+0.5(0.7)(0.8)=0.03+0.28=0.310$. (2) The separately integrated terms are $0.5(0.3)+0.5(0.7)=0.50$ and $0.5(0.2)+0.5(0.8)=0.50$, so their product is $0.250$, not $0.310$. (3) Conditional independence permits multiplying the outcome and dropout likelihood contributions at a fixed $b_i$; that product is then integrated against $f_b$. (4) No. The outcome factor uses the observed outcome history; the unobserved realized value is not treated as observed data.