From Conditional Likelihood to Random-Effects Integral

From a Conditional Model to a Marginal Likelihood

In a generalized linear mixed model, observations within a cluster are described conditionally on that cluster’s random effect. The random effect is not observed, so the likelihood for the observed data must account for its distribution rather than treat its realized value as known. This change—from a likelihood conditional on a latent effect to one for the observed responses—is the key step in setting up likelihood-based estimation.

The immediate task is to express that step precisely. The conditional response model supplies a probability mass or density for the observed responses at a given random-effect value. The random-effects model supplies a distribution over those values. Integrating their product removes the unobserved effect and gives the marginal probability mass or density for the observed data.

For discrete responses, \(L_i(y_i\mid b_i;\beta,\phi)\) is a conditional probability mass, and the integral gives the marginal probability mass of the observed response vector. For continuous responses, the conditional contribution is a density evaluated at \(y_i\), and the integral gives the marginal density at \(y_i\). A density at a point is not the probability of observing that exact point; probabilities for continuous outcomes are obtained by integrating a density over a set of possible values.

When responses are conditionally independent within each cluster given \(b_i\), the conditional contribution factors as $$ L_i(y_i\mid b_i;\beta,\phi) = \prod_{j=1}^{n_i} f(y_{ij}\mid b_i;\beta,\phi). $$ The symbol \(f\) here stands for the appropriate conditional mass function or density. If the response model has additional conditional dependence not accounted for by \(b_i\), the cluster’s joint conditional likelihood must be used instead of this product.

If clusters are independent after accounting for their random effects, and each cluster has its own random effect drawn from the specified distribution, the observed-data likelihood is $$ L(\beta,\theta,\phi) = \prod_{i=1}^{m}L_i^{\mathrm{marg}}(\beta,\theta,\phi) = \prod_{i=1}^{m} \int_{\mathcal B_i} L_i(y_i\mid b_i;\beta,\phi) f_b(b_i;\theta)\,db_i. $$ This product relies on the stated cluster-independence structure. A shared random effect across several clusters, for example, generally requires a joint integral rather than separate clusterwise integrals.

What Is Integrated, and What Is Estimated?

The integration variable is the unobserved random-effect value \(b_i\), not the parameter \(\theta\) that describes the distribution of random effects. The fixed effects \(\beta\), random-effects parameters \(\theta\), and any conditional dispersion parameters \(\phi\) remain arguments of the marginal likelihood. Likelihood-based estimation compares the values of this function as those parameters vary.

The random effect is modeled as random in the probability model, but its integration variable is not being estimated as a fixed parameter in the marginal likelihood. Some procedures also produce cluster-specific predictions of random effects; those are distinct from forming the observed-data likelihood. For the integral here, the distribution \(f_b(b_i;\theta)\) weights the conditional response contribution across possible values of \(b_i\).

Why the Integral Has This Form

The construction follows from averaging over the latent effect’s distribution. In the discrete-response case, for a possible response vector \(y_i\), the law of total probability gives the marginal probability mass by summing the conditional mass over the random-effect distribution. For a continuous response vector, the same argument applied to a small region around \(y_i\), followed by the density limit, gives a marginal density equal to the integral of conditional densities weighted by the random-effects density. Thus the integral is not an additional modeling convention: it is the observed-data mass or density implied by the conditional model and the random-effects distribution.

A simple continuous example makes the density calculation explicit. In a hypothetical one-observation Gaussian random-intercept model, suppose \(b\sim N(0,1)\) and \(Y\mid b\sim N(b,1)\), with both variances fixed at one. The observed value is \(y=0\). The conditional density and random-effect density are $$ f(y=0\mid b)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{b^2}{2}\right), \qquad f_b(b)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{b^2}{2}\right). $$

Reading the Integrand and Checking Its Scale

The integrand is the product of two ingredients: a conditional likelihood contribution and a random-effects density. The density is normalized over the support of the random effect, so integrating the weighted conditional contribution averages it over the model’s random-effects distribution. In a continuous random-effects model, the density value itself need not be less than one; its units are inverse units of the random effect. The product with the differential \(db_i\), integrated over the support, yields a response mass or density on the response scale.

Assumptions, Diagnostics, and Alternatives

Before using the product-integral expression, verify the structure that makes it valid. The conditional response model must specify a valid joint mass or density for the observed responses given the random effects. The random-effects distribution must be a valid normalized distribution on the integration domain. The conditional independence product is appropriate only if responses are independent given the random effects and any other modeled conditioning variables. Finally, the clusterwise product across \(i\) requires the assumed independence of clusters after their random effects are accounted for.

A useful diagnostic at the setup stage is to check dimensions and dependencies before doing any numerical work. There should be one integration differential for each component of the random-effect vector. The conditional likelihood must depend on the random effects in the way specified by the model, while the random-effects density must include all parameters governing their distribution. If a covariance or scale parameter is estimated, it belongs among the likelihood’s arguments even though its associated random effects are integrated out.

Common warning signs include a conditional contribution that does not normalize over possible responses, a purported random-effects density that does not integrate to one, or a product factorization that silently assumes independence not present in the model. Another error is to substitute a single estimated or predicted random effect for the integral. That gives a conditional likelihood at one value, not the marginal likelihood defined above. Numerical methods can also produce inaccurate integrals, especially as the number of random-effect dimensions grows; recognizing that computational challenge is different from changing what the target integral means.

The conditional likelihood and the marginal likelihood answer different questions. The former describes the response contribution given a particular random-effect value; the latter describes the observed responses after accounting for the random-effects distribution. Generalized estimating equations are an alternative approach to population-averaged inference and do not form this GLMM likelihood by integrating a conditional likelihood against a random-effects density. For a GLMM, the integral is the likelihood implied by the specified conditional and random-effects models. It may be evaluated analytically in special cases or approximated numerically when a closed form is unavailable; either way, the target remains the same integral.

A common mistake is to say, “The random effect is unobserved, so I will plug in its most likely value and use the conditional likelihood there.” The correction is that a value chosen by maximizing or predicting \(b_i\) does not average over its distribution. The marginal contribution is an integral over all \(b_i\), with each conditional likelihood weighted by \(f_b(b_i;\theta)\). Replacing the integral by a single value changes the likelihood being evaluated.