Why the Subject-Level Likelihood Matters
In a population study, one individual may contribute several measurements, such as concentrations collected at multiple times after a dose. Those observations share features of the individual that are not fully described by measured covariates. A random effect represents such unobserved individual-level variation. The likelihood for that individual must account for the fact that the same random effect is present across the individual's repeated observations.
The central distinction is between conditioning on the random effect and integrating it out. Given a particular random effect, repeated observations may be independent under the model. But after the random effect is integrated out, the observations will generally be dependent. Multiplying separately averaged observation densities discards that shared source of variation and corresponds to a different model.
This definition assumes a specified conditional observation model, a specified distribution for the random effects, and conditional independence of observations given the random effect and model covariates. It also assumes that the integral exists. If the conditional observation model includes residual dependence, its joint conditional density replaces the product in the definition; conditional independence should not be imposed automatically.
For independent individuals, the population likelihood is the product of their subject-level marginal likelihoods: \(L^{\mathrm{marg}}(\theta,\psi)=\prod_{i=1}^{N}L_i^{\mathrm{marg}}(\theta,\psi)\). The product is across individuals, not generally across all observations as though each observation came from a new individual.
Deriving the Marginal Likelihood
The derivation follows from the rule for a joint density. For one individual, the joint density of the observations and random effect is the conditional density of the observations given the random effect multiplied by the random-effect density. The observation density alone is obtained by integrating that joint density over every possible random-effect value.
For a realized \(\eta_i\), write the joint conditional density of all observations. Under conditional independence, this is \(\prod_j f(y_{ij}\mid\eta_i,\theta)\).
The joint density of observations and random effect is \(\left\{\prod_j f(y_{ij}\mid\eta_i,\theta)\right\}g(\eta_i\mid\psi)\).
Integrating the joint density with respect to \(\eta_i\) gives the marginal density of the complete observation vector \(y_i\), and hence its likelihood.
The order of operations is important: first multiply the observation densities conditional on the shared \(\eta_i\), then integrate. In general, integration does not distribute over multiplication: \(\int f_1(\eta)f_2(\eta)g(\eta)d\eta\) is not equal to \(\{\int f_1(\eta)g(\eta)d\eta\}\{\int f_2(\eta)g(\eta)d\eta\}\). The latter expression treats the latent effect affecting one observation as independent of the latent effect affecting the other.
A Shared Random Effect Creates Marginal Dependence
A useful justification is the law of total covariance. If \(Y_{i1}\) and \(Y_{i2}\) are conditionally independent given \(\eta_i\), then their conditional covariance is zero. Their marginal covariance is still $$ \operatorname{Cov}(Y_{i1},Y_{i2}) =E\{\operatorname{Cov}(Y_{i1},Y_{i2}\mid\eta_i)\} +\operatorname{Cov}\{E(Y_{i1}\mid\eta_i),E(Y_{i2}\mid\eta_i)\} =\operatorname{Cov}\{E(Y_{i1}\mid\eta_i),E(Y_{i2}\mid\eta_i)\}. $$ Thus, whenever the conditional means tend to move together as the shared random effect varies, the observations are associated marginally. Conditional independence is not a claim of marginal independence.
Worked Example: Two Repeated Binary Outcomes
Suppose one individual has two binary outcomes \(Y_{i1}\) and \(Y_{i2}\). For this illustration, let the shared random intercept \(\eta_i\) equal \(1\) or \(-1\), each with probability \(0.5\). Given \(\eta_i\), the outcomes are independent Bernoulli variables with success probability \(q(\eta_i)=1/(1+\exp(-\eta_i))\). This two-point random-effect distribution is used to make the integration explicit; it is not a claim that random effects must have this distribution.
The unrounded probabilities are \(q(1)\approx0.7310586\) and \(q(-1)\approx0.2689414\). For two observed successes, the marginal likelihood contribution is the average of the conditional likelihoods:
The powers above use the unrounded logistic probabilities; the displayed rounded values are only for readability. Each observation separately has marginal success probability \(0.5q(1)+0.5q(-1)=0.5\). Multiplying those two marginal probabilities would give \(0.5\times0.5=0.25\), not \(0.3033881\). That product would describe independent draws of the random intercept across the two observations, rather than one shared intercept.
The same calculation gives \(P(Y_{i1}=1,Y_{i2}=0)=0.5\{q(1)[1-q(1)]+q(-1)[1-q(-1)]\}\approx0.196612\). By symmetry, the probability of two failures is \(0.303388\). These four probabilities sum to \(1.000000\) to the shown precision, as required.
Worked Likelihoods with Continuous Random Effects
The next examples show that the same marginalization applies when the random-effect distribution is continuous. The Gaussian example also provides a closed-form result, while the Poisson example illustrates an exact integral for counts.
Worked Example: Repeated Gaussian Measurements
Assume the model \(Y_{ij}=\mu+\eta_i+\varepsilon_{ij}\), where \(\eta_i\sim N(0,\tau^2)\), \(\varepsilon_{ij}\sim N(0,\sigma^2)\), and the residuals are independent of one another and of \(\eta_i\). Let one hypothetical individual have measurements \(y_i=(8,10)\), with \(\mu=9\), \(\tau^2=1\), and \(\sigma^2=1\). Conditional on \(\eta_i\), the two measurements are independent normal variables.
After integrating over \(\eta_i\), the observation vector is normal with mean \(\mu\mathbf{1}\) and covariance matrix \(\tau^2\mathbf{1}\mathbf{1}^{T}+\sigma^2 I\). Here that matrix is \(\begin{pmatrix}2&1\\1&2\end{pmatrix}\), with determinant \(3\). The residual vector is \((-1,1)^T\), and the quadratic form using the inverse covariance is \(2\). Therefore the marginal density at the observed vector is
The units of this joint density are inverse squared measurement units. For comparison, each single observation has marginal distribution \(N(9,2)\). Multiplying its two marginal densities at 8 and 10 gives \(\{(1/\sqrt{4\pi})\exp(-1/4)\}^2 =(1/(4\pi))\exp(-1/2)\approx0.04827\). That is not the joint density under the shared-effect model. The nonzero covariance between the repeated measurements is \(\tau^2=1\), so the product of separate marginal densities omits their dependence.
Worked Example: Repeated Poisson Counts with a Gamma Random Effect
Let \(Y_{i1}\) and \(Y_{i2}\), conditional on a shared rate multiplier \(U_i\), be independent Poisson counts with means \(U_i\) for each observation. Suppose \(U_i\sim\operatorname{Gamma}(a=2,\text{ rate }b=1)\), whose density is \(g(u)=ue^{-u}\) for \(u>0\). For the hypothetical counts \((1,2)\), conditional independence gives \(P(1,2\mid u)=e^{-2u}u^3/(1!2!)\).
Integrating this joint conditional likelihood against the gamma density yields
If instead the marginal probabilities for the two observations were multiplied, the first would be \(\int_0^\infty u^2e^{-2u}du=2!/2^3=0.25\), and the second would be \(\frac12\int_0^\infty u^3e^{-2u}du=3!/(2\cdot2^4)=0.1875\). Their product is \(0.046875\), different from the shared-effect joint probability. Again, the discrepancy reflects a different dependence structure, not a rounding issue.
What the Alternative Factorization Assumes
The product of separately marginalized observation densities has the form $$ \prod_{j=1}^{m_i}\left\{\int f(y_{ij}\mid\eta_{ij},\theta) g(\eta_{ij}\mid\psi)\,d\eta_{ij}\right\}. $$ It uses an independently integrated latent effect \(\eta_{ij}\) for each observation. In a repeated-measures setting, this is generally not the specified model with one shared \(\eta_i\). It might be appropriate if the scientific model genuinely assigns a new independent latent effect to each occasion, but that is a substantive modeling choice, not a computational shortcut.
| Approach | Likelihood construction | Interpretation |
|---|---|---|
| Conditional likelihood | Use \(L_i(\theta,\eta_i)\) at a specified \(\eta_i\). | Describes observations conditional on the individual's effect; does not by itself average over the population distribution. |
| Subject-level marginal likelihood | Integrate the complete conditional likelihood over one shared \(\eta_i\). | Describes the joint observations from an individual sampled from the random-effect distribution. |
| Product of observation-level marginal densities | Integrate each observation separately and multiply. | Generally treats latent effects as independently varying by observation and removes shared-effect dependence. |
For nonlinear conditional models, the subject-level integral may not have a closed form. Numerical integration can then be used to evaluate it, with accuracy checked over the relevant parameter range. Such computation does not change the target likelihood: the integrand remains the product of the conditional contributions for the entire individual multiplied by the density of the shared random effect.
Diagnostics, Limitations, and a Common Pitfall
Before evaluating or approximating a marginal likelihood, check the model's structure. Confirm which observations share each random effect, whether the conditional observation model really factors into independent terms, and whether the random-effect density is normalized and has the intended parameterization. With numerical integration, compare results at tighter tolerances or with a different reliable integration scheme; substantial changes indicate that the integral may not yet be evaluated accurately.
- Wrong grouping: If observations from the same individual are given separate random effects by mistake, the fitted likelihood loses the intended within-individual association.
- Unjustified conditional independence: A shared random effect does not capture every possible residual dependence. If observations remain dependent after conditioning on it, the conditional joint model must represent that dependence.
- Integration difficulty: High-dimensional or sharply concentrated random-effect distributions can make numerical evaluation challenging. An approximation is not automatically accurate just because it returns a value.
- Population interpretation: The marginal likelihood averages over the specified distribution of individual effects. If that distribution is poorly chosen, the resulting likelihood may not represent the population adequately.
The subject-level marginal likelihood is therefore the appropriate likelihood when the target is population-level inference under a shared random-effect model. Its key limitation is that the integral can be difficult to evaluate, and its validity depends on the assumed conditional model and random-effect distribution. A convenient factorization or approximation should not be allowed to obscure those assumptions.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For two repeated measurements sharing \(\eta_i\), write the subject-level marginal likelihood under conditional independence.
- Why can conditional independence hold while the observations are marginally dependent?
- In the binary example, what probability would the product of separate marginal success probabilities give for two successes, and why is it different from the shared-effect result?
- In the Gaussian example, identify the off-diagonal covariance and explain why multiplying the two marginal densities is not the joint likelihood.
Solutions. (1) The likelihood is \(\int f(y_{i1}\mid\eta_i,\theta)f(y_{i2}\mid\eta_i,\theta)g(\eta_i\mid\psi)d\eta_i\); both conditional factors use the same random effect. (2) The shared random effect varies across individuals and induces covariance between conditional means; the total-covariance expression shows that marginal covariance can be nonzero even when conditional covariance is zero. (3) Each marginal success probability is \(0.5\), so their product is \(0.25\). The shared-effect joint probability is approximately \(0.303388\), because both outcomes respond to the same latent intercept. (4) The off-diagonal covariance is \(\tau^2=1\). The separate marginal densities correspond to an independent product, whereas the correct bivariate normal density uses the covariance matrix with off-diagonal entries equal to \(1\).