Differentiating a Marginal Likelihood Integral

Why Differentiate a Marginal Likelihood Integral?

In a shared random-effects model, each subject's observed-data likelihood is obtained by integrating over unobserved random effects. Estimation often requires a derivative of that marginal likelihood with respect to a model parameter. The central question is whether that derivative can be taken inside the integral. When it can, the result gives an exact expression for the derivative of the marginal likelihood; when it cannot, omitting boundary or density terms can produce a wrong result.

Let $O_i$ denote subject $i$'s observed longitudinal and event-time data, $b_i\in\mathbb{R}^q$ their latent random effects, and $\theta$ a parameter vector. Under a shared-random-effects model, the subject contribution has the form

L_i(\theta)=\int_{\mathcal{B}}g_i(b;\theta)\,db, \qquad g_i(b;\theta)=p(O_i\mid b;\theta)f_b(b;\theta).

Here $p(O_i\mid b;\theta)$ is the conditional contribution from the observed data, and $f_b(b;\theta)$ is the random-effects density. The conditional contribution incorporates the longitudinal model and the event-time contribution, including its censoring information. If entry is delayed, its conditioning adjustment must already be included in that contribution. The sample log likelihood is $\ell(\theta)=\sum_i\log L_i(\theta)$.

Definition: Differentiating a marginal likelihood integral means finding a derivative such as $\partial L_i(\theta)/\partial\theta_r$ from its integral representation. For a fixed integration region $\mathcal{B}$, the desired identity is $$ \frac{\partial L_i(\theta)}{\partial\theta_r} = \int_{\mathcal{B}}\frac{\partial g_i(b;\theta)}{\partial\theta_r}\,db. $$ This identity is valid only when suitable regularity and integrability conditions hold.

Conditions for Moving the Derivative Inside

A standard sufficient condition is a local version of differentiation under the integral sign. Fix a parameter value $\theta_0$ and one component $\theta_r$. Suppose that, in a neighborhood of $\theta_0$, the following conditions hold:

The integration region $\mathcal{B}$ does not depend on $\theta_r$.
For almost every $b\in\mathcal{B}$, $g_i(b;\theta)$ is differentiable with respect to $\theta_r$.
There is an integrable function $H(b)$ such that $\left|\partial g_i(b;\theta)/\partial\theta_r\right|\leq H(b)$ throughout that neighborhood.
The integral defining $L_i(\theta)$ is finite there.

Then, at $\theta_0$, the derivative exists and equals the integral of the pointwise derivative. The key technical requirement is the integrable bound: it prevents the derivative's tails from becoming too large to interchange the limiting operation and integration. A bound that works only at $\theta_0$, but not in a neighborhood, is generally not enough for this standard argument.

Theorem: Under the fixed-support, pointwise differentiability, and integrable domination conditions above, $$ \left.\frac{\partial}{\partial\theta_r} \int_{\mathcal{B}}g_i(b;\theta)\,db\right|_{\theta=\theta_0} = \int_{\mathcal{B}} \left.\frac{\partial g_i(b;\theta)}{\partial\theta_r}\right|_{\theta=\theta_0}\,db. $$ For a vector parameter, apply the result component by component.

This result is commonly justified by the dominated convergence theorem applied to difference quotients. For a small nonzero $h$, define $Q_h(b)=\{g_i(b;\theta_0+h)-g_i(b;\theta_0)\}/h$. Pointwise differentiability gives convergence of $Q_h(b)$ to the pointwise derivative for almost every $b$. The mean value theorem bounds the difference quotient by a local bound on the derivative; if that bound is integrable, dominated convergence allows its limit to pass through the integral. Thus the identity is justified by control of the entire integrand, not merely by the apparent smoothness of the model formulas.

The integrand can depend on $\theta_r$ through more than one factor. In particular, if $\theta_r$ is a component of the random-effects covariance parameters, then $f_b(b;\theta)$ also varies with that parameter. Its derivative must be included. For a product $g_i=p(O_i\mid b;\theta)f_b(b;\theta)$, the ordinary product rule gives

\frac{\partial g_i}{\partial\theta_r} = \frac{\partial p(O_i\mid b;\theta)}{\partial\theta_r}f_b(b;\theta) + p(O_i\mid b;\theta)\frac{\partial f_b(b;\theta)}{\partial\theta_r}.

The assumptions concern a neighborhood where the model is defined. For example, a covariance matrix must remain positive definite in that neighborhood if the random-effects density is a nonsingular multivariate normal density. A parameter at the boundary of its allowed range may not have the required two-sided neighborhood, and ordinary interior differentiation arguments may not apply there.

Worked Examples

Worked Example: A Gaussian Integral With a Valid Derivative

Let $B\sim N(0,1)$ and define $I(\theta)=\int_{-\infty}^{\infty}e^{-\theta b}\phi(b)\,db$, where $\phi$ is the standard normal density. Differentiate $I(\theta)$ and evaluate the derivative at $\theta=0.4$.

For $\theta$ in the neighborhood $[-0.5,0.5]$, the magnitude of the pointwise derivative is $\left|b e^{-\theta b}\phi(b)\right|\leq |b|e^{0.5|b|}\phi(b)$. This bound is integrable: the normal density's quadratic exponential decay dominates the factor $e^{0.5|b|}$. The support is the fixed real line, so differentiation under the integral is justified.

Therefore,

I'(\theta)=\int_{-\infty}^{\infty}(-b)e^{-\theta b}\phi(b)\,db.

To evaluate the integral, complete the square: $-b^2/2-\theta b=-(b+\theta)^2/2+\theta^2/2$. Hence $I(\theta)=e^{\theta^2/2}$, and $I'(\theta)=\theta e^{\theta^2/2}$. At $\theta=0.4$, $I'(0.4)=0.4e^{0.08}=0.4333$. The calculation confirms the derivative obtained by moving the derivative inside the integral.

Worked Example: Why a Moving Boundary Adds a Term

For $\theta>0$, consider $J(\theta)=\int_0^\theta 1\,db$. What is its derivative, and what goes wrong if one differentiates only the integrand?

The integrand is constant, so its derivative with respect to $\theta$ is zero. But the integration interval changes with $\theta$: direct evaluation gives $J(\theta)=\theta$, so $J'(\theta)=1$. The fixed-support theorem cannot be used because its support condition fails.

The Leibniz rule for differentiable limits $a(\theta)$ and $c(\theta)$ includes both an interior term and boundary terms:

\frac{d}{d\theta}\int_{a(\theta)}^{c(\theta)}g(b;\theta)\,db = g(c(\theta);\theta)c'(\theta) - g(a(\theta);\theta)a'(\theta) + \int_{a(\theta)}^{c(\theta)} \frac{\partial g(b;\theta)}{\partial\theta}\,db.

Here $a(\theta)=0$, $c(\theta)=\theta$, and $g=1$, so the upper boundary contributes $1\cdot1$, the lower boundary contributes zero, and the interior derivative contributes zero. Thus $J'(\theta)=1$, as required. Equivalently, set $b=\theta u$ to obtain $J(\theta)=\int_0^1\theta\,du$; the transformed integrand's derivative includes the Jacobian factor's derivative.

Worked Example: Differentiating a Censored Joint-Model Contribution

Consider a hypothetical subject with one longitudinal measurement $Y=1$, measured at a time when the latent mean is $b$. Suppose $Y\mid b\sim N(b,1)$, $b\sim N(0,1)$, and the event hazard is constant over follow-up with $h(t\mid b)=\exp(\alpha b)$. The subject is right-censored at time $t=1$, so the event contribution is the conditional survival probability $\exp\{-\exp(\alpha b)\}$. Find the derivative of this subject's marginal likelihood with respect to $\alpha$ at $\alpha=0$.

The joint integrand and marginal likelihood are

g(b;\alpha)=\phi(b)\phi(1-b)\exp\{-\exp(\alpha b)\}, \qquad L(\alpha)=\int_{-\infty}^{\infty}g(b;\alpha)\,db.

The event is censored, so there is no hazard factor multiplying the survival term. Differentiating the exponent gives $\partial g(b;\alpha)/\partial\alpha=-b\exp(\alpha b)g(b;\alpha)$. The support is fixed. Moreover, $x e^{-x}\leq 1/e$ for $x>0$; taking $x=\exp(\alpha b)$ shows that the absolute derivative is bounded by $|b|\phi(b)\phi(1-b)/e$, an integrable function. Differentiation under the integral is therefore justified near $\alpha=0$.

At $\alpha=0$, $\exp(\alpha b)=1$, so \[ L'(0)=-e^{-1}\int_{-\infty}^{\infty}b\phi(b)\phi(1-b)\,db. \] The product $\phi(b)\phi(1-b)$, when normalized as a density in $b$, is the conditional density of $B$ given $Y=1$ in the model $Y=B+\varepsilon$, with independent $B,\varepsilon\sim N(0,1)$. Completing the square gives $B\mid Y=1\sim N(0.5,0.5)$. Also $Y\sim N(0,2)$, so its density at 1 is \[ \frac{1}{\sqrt{4\pi}}e^{-1/4}=0.2197. \] Consequently, \[ \int b\phi(b)\phi(1-b)\,db = 0.2197(0.5)=0.1098, \qquad L'(0)=-e^{-1}(0.1098)=-0.0404. \] For comparison, $L(0)=e^{-1}(0.2197)=0.0808$. The negative derivative is consistent with the model: near $\alpha=0$, increasing $\alpha$ reduces the survival contribution for positive $b$, and the observed measurement shifts the conditional distribution of $b$ above zero.

What to Check in Practice

A smooth-looking likelihood expression does not by itself establish that differentiation under its integral is valid. Before using the identity, check the conditions in the model you are actually fitting:

Support: Does the random-effects integration region depend on the parameter? If it does, boundary terms or a parameter-dependent change of variables may be needed.
All parameter pathways: Does the parameter change the longitudinal density, the event contribution, the random-effects density, or more than one of these?
Tail control: Can the derivative be bounded by an integrable function locally? Pay particular attention to products involving large random effects and exponential hazards.
Model boundaries: Is the parameter at a constraint boundary, or does a covariance matrix become singular? Interior differentiability may not apply at such points.
Numerical behavior: When checking a computed derivative, compare its stability across reasonable integration tolerances and, where practical, against finite differences of the marginal integral. Agreement is a diagnostic, not a substitute for the mathematical conditions.

The main alternatives address different issues. Differentiating the integrand and then integrating uses the identity established here, provided its conditions hold. Finite differences instead approximate a derivative from nearby marginal likelihood evaluations; they can be sensitive to step size and integration error. A change of variables can sometimes make support fixed, but the transformed density and Jacobian must both be differentiated. None of these choices removes the need to account for parameter dependence in the full integrand.

A common mistake is to differentiate only $p(O_i\mid b;\theta)$ while treating $f_b(b;\theta)$ as fixed. That is wrong whenever the parameter also governs the random-effects distribution. Another is to differentiate only the integrand while ignoring moving integration limits. The correction in each case is the same in spirit: write down the complete integral and identify every way in which its integrand and support depend on the parameter before differentiating.

The result has limits. A failure of the stated sufficient conditions does not prove that differentiation is impossible, but it means this justification is unavailable and another argument is needed. Heavy tails can prevent an integrable derivative bound; changing supports can introduce boundary terms; and numerical quadrature can obscure whether the exact derivative and the derivative of a numerical approximation are close. These issues are especially relevant when assessing derivatives used in optimization.

Check Your Understanding

Answer each question, then compare with the solutions below.

For fixed support, name the central domination condition that justifies moving a derivative inside an integral.
If $K(\theta)=\int_0^\theta 2b\,db$, why is integrating only the derivative of the integrand insufficient?
In a joint model, a parameter changes both the event hazard and the random-effects variance. Which factors must be differentiated?
For the censored example above, what is $L'(0)$ if the measurement is $Y=0$ instead of $Y=1$, with all other model features unchanged?

Solutions. (1) The pointwise derivative must be dominated throughout a parameter neighborhood by an integrable function of the random effects. (2) The upper integration limit depends on $\theta$, so the boundary term is $2\theta$; indeed $K(\theta)=\theta^2$ and $K'(\theta)=2\theta$, whereas the interior derivative is zero. (3) Differentiate the full product of the conditional data contribution and random-effects density, including the hazard and variance dependence. (4) At $\alpha=0$, the posterior mean of $B$ given $Y=0$ is zero by symmetry. Thus the integral of $b\phi(b)\phi(-b)$ is zero and $L'(0)=0$.

Differentiating a Marginal Likelihood Integral

What You'll Learn