Why Likelihood Monotonicity Matters
In a joint model, the longitudinal measurements and the event-time outcome are linked through latent random effects. The observed-data likelihood therefore integrates over those effects, which can make direct maximization difficult. The EM algorithm instead alternates between calculating the current posterior distribution of the latent effects and improving an expected complete-data log likelihood. A key guarantee is that an exact EM update does not lower the observed-data likelihood.
That guarantee has conditions. In particular, a likelihood-ratio identity often used in its proof requires a support condition. If the candidate complete-data model assigns positive mass where the current model assigns zero, an expectation under the current posterior omits that candidate mass. The monotonicity result still follows, but the proof must account for the omitted mass.
The density \(f\) includes the random-effects distribution and the conditional contributions from both the longitudinal and survival parts of the model. The monotonicity argument does not require a particular choice of those components. It does require a common space on which the complete-data densities can be compared, finite positive likelihoods for the updates being compared, and an exact E-step posterior. For a useful update, \(Q(\theta\mid\theta^{(r)})\) must be well-defined and finite at the current value and at the proposed value.
An exact M-step chooses \(\theta^{(r+1)}\) to maximize \(Q(\theta\mid\theta^{(r)})\), so it satisfies \(Q(\theta^{(r+1)}\mid\theta^{(r)})\geq Q(\theta^{(r)}\mid\theta^{(r)})\). A generalized EM step need not maximize \(Q\), but must still increase or leave it unchanged. The likelihood guarantee applies to either update when that auxiliary-function condition and the proof’s regularity conditions hold.
The Jensen Bound and the Support Condition
Write \(A_r=\{b:f(y,b;\theta^{(r)})>0\}\), interpreted up to sets of measure zero. The current posterior is supported on \(A_r\). On that support define \(R(b)=f(y,b;\theta)/f(y,b;\theta^{(r)})\). The likelihood ratio is not necessarily the posterior expectation of \(R\): that equality holds only if the candidate density has no mass outside \(A_r\). In general, split the candidate likelihood into its contributions on and off the current support:
The second term is nonnegative. If \(E_{q_r}[R]\) is positive and the relevant logarithmic expectations are finite, Jensen’s inequality for the concave logarithm gives \(\log E_{q_r}[R]\geq E_{q_r}[\log R]\). On the current posterior support, the logarithm of the ratio is the difference of complete-data log densities. Consequently,
This is the monotonicity bound. If the proposed update does not decrease \(Q\), the right-hand side is nonnegative, so the observed-data log likelihood cannot decrease. Where the candidate density is zero on a set of positive current posterior probability, \(Q(\theta\mid\theta^{(r)})\) is typically negative infinity; such a candidate cannot be a finite-\(Q\)-increasing update from a finite current value.
If the candidate density is zero wherever the current complete-data density is zero, then the candidate has no mass outside \(A_r\). In that special case the nonnegative extra term is zero and the likelihood-ratio equality \(L(\theta)/L(\theta^{(r)})=E_{q_r}[R]\) is valid. This support condition is sufficient for the equality, but it is not needed for the more general inequality and monotonicity bound above.
Worked Example: Candidate Mass Outside the Current Support
Suppose \(y\) is a discrete observed outcome and \(b\) takes values \(a,b,c\). For this observed outcome, the current complete-data masses at \(a,b,c\) are \(0.2,0.3,0\), respectively. The candidate masses are \(0.3,0.3,0.1\). These are valid contributions to a discrete joint distribution: the remaining probability can be assigned to other observed outcomes.
The current likelihood is \(L(\theta^{(r)})=0.2+0.3=0.5\), and the current posterior probabilities are \(q_r(a)=0.2/0.5=0.4\) and \(q_r(b)=0.3/0.5=0.6\). The candidate likelihood is \(L(\theta)=0.3+0.3+0.1=0.7\), so the likelihood ratio is \(0.7/0.5=1.4\), with log ratio \(\log(1.4)=0.3365\).
On the current support, the ratios are \(R(a)=0.3/0.2=1.5\) and \(R(b)=0.3/0.3=1\). Thus \(E_{q_r}[R]=0.4(1.5)+0.6(1)=1.2\), not \(1.4\). The omitted difference, \(1.4-1.2=0.2\), is exactly the candidate mass \(0.1\) divided by the current likelihood \(0.5\). The Jensen bound gives \(Q(\theta\mid\theta^{(r)})-Q(\theta^{(r)}\mid\theta^{(r)})=0.4\log(1.5)+0.6\log(1)=0.1622\). Indeed, \(\log(1.4)=0.3365\geq\log(1.2)=0.1823\geq0.1622\). The likelihood increased, and the correct argument retains the extra candidate mass rather than asserting a false equality.
Worked Example: Applying Jensen When the Supports Agree
Consider two latent states with current complete-data masses \(0.10\) and \(0.30\) for the observed outcome. The current likelihood is \(0.40\), so the posterior probabilities are \(0.10/0.40=0.25\) and \(0.30/0.40=0.75\). Suppose a candidate has masses \(0.20\) and \(0.36\), with no mass outside these same states.
The candidate likelihood is \(0.56\), so the likelihood ratio is \(0.56/0.40=1.4\). The complete-data ratios are \(2\) and \(1.2\). Their posterior expectation is \(0.25(2)+0.75(1.2)=1.4\), as expected under the shared-support condition. The auxiliary-function difference is \(0.25\log(2)+0.75\log(1.2)=0.3100\).
Jensen’s inequality gives \(\log(1.4)=0.3365\geq0.3100\). Thus the candidate increases \(Q\), and the observed-data log likelihood increases by \(0.3365\) on this log-ratio scale. The likelihood increase is at least the auxiliary-function increase, not necessarily equal to it.
A Joint-Model Example of an Exact E- and M-Step
A small latent-variable model makes the E-step and M-step concrete. It is a simplified illustration of the shared-random-effects logic, not a longitudinal-survival analysis. Let \(b\in\{0,1\}\) have prior probability \(1/2\) for each value. Conditional on \(b=0\), an observed binary outcome \(y=1\) has probability \(0.2\); conditional on \(b=1\), it has probability \(\theta\), where \(0\leq\theta\leq1\). The complete-data contributions for the observed \(y=1\) are therefore \(f(1,0;\theta)=0.5(0.2)=0.1\) and \(f(1,1;\theta)=0.5\theta\). These specify valid probabilities for this outcome; the complementary outcome has the remaining conditional probability.
Worked Example: Verifying a One-Parameter EM Update
Start from \(\theta^{(r)}=0.4\). The observed likelihood is \(L(\theta^{(r)})=0.1+0.5(0.4)=0.3\). The E-step posterior probability of \(b=1\) is \(0.2/0.3=2/3\), and the probability of \(b=0\) is \(1/3\).
Ignoring terms in \(Q\) that do not depend on \(\theta\), the auxiliary function is \(Q(\theta\mid0.4)=(2/3)\log(0.5\theta)+(1/3)\log(0.1)\). For \(0<\theta\leq1\), its derivative with respect to \(\theta\) is \((2/3)/\theta>0\). Hence the exact constrained M-step maximizer is \(\theta^{(r+1)}=1\), the upper boundary of the parameter space. The auxiliary-function increase is \((2/3)\log(1/0.4)=(2/3)\log(2.5)=0.6109\).
At the updated parameter, \(L(1)=0.1+0.5=0.6\). The likelihood ratio is \(0.6/0.3=2\), and its log is \(\log(2)=0.6931\). The observed log likelihood therefore increases, and \(0.6931\geq0.6109\), as the bound predicts. The calculation uses an exact posterior and an exact maximizer of \(Q\); it does not claim that an arbitrary parameter change would have this guarantee.
What the Guarantee Does—and Does Not—Say
The result is an iteration-by-iteration statement: if the E-step posterior is exact and the M-step does not decrease the corresponding \(Q\), then the observed-data likelihood does not decrease. It does not guarantee a global maximum. The likelihood may have multiple local maxima or other stationary points, so an EM sequence can settle at a suboptimal solution or progress slowly. Monotonicity alone also does not establish that the parameter estimates have converged to a desirable solution.
In joint models, the E-step usually requires integration over random effects. If those integrals are evaluated with finite numerical accuracy, the computed posterior and auxiliary function may differ from their exact values. A small apparent decrease can then reflect integration error rather than a failure of the exact theorem; conversely, a computed increase does not by itself prove the exact likelihood increased. The earlier material on numerical integration is relevant when checking this distinction. Increasing integration accuracy and evaluating the observed likelihood consistently at successive iterates are practical checks.
A useful comparison is with direct observed-likelihood optimization. Direct optimization targets \(L(\theta)\) itself and need not have EM’s stepwise monotonicity unless its own algorithm enforces it. EM instead uses a surrogate, \(Q\), whose increase gives a lower bound on the change in log likelihood. An approximate or generalized EM update retains the guarantee only if it genuinely increases the exact \(Q\); merely taking a step that appears to improve a numerically approximated \(Q\) is not sufficient.
The main assumptions and failure checks can be summarized as follows:
| Requirement | Why it matters | Practical check |
|---|---|---|
| A common dominating measure and well-defined likelihood | Complete-data densities and their support can be compared. | Confirm the latent space and density conventions are consistent across parameter values. |
| Positive, finite current likelihood and a well-defined posterior | The E-step posterior must normalize to a probability distribution. | Check that the integrated likelihood is finite and nonzero. |
| An exact E-step for the theorem as stated | The expectation in \(Q\) must use the actual current posterior. | Assess integration accuracy; a coarse approximation can distort the posterior weights. |
| A nondecreasing \(Q\) and finite relevant expectations | Jensen’s bound translates the increase in \(Q\) into a likelihood guarantee. | Verify the proposed update increases the same \(Q\) defined by the current E-step. |
The distinction is especially important when latent supports vary with parameters or when boundary values make some complete-data contributions zero. The monotonicity conclusion is robust to extra candidate mass, but the equality-based proof is not. Exact EM is therefore best understood as a lower-bound argument: a nondecreasing auxiliary function guarantees a nondecreasing likelihood under the stated regularity conditions, even when the likelihood ratio is not exactly the posterior expectation of a complete-data ratio.
Check Your Understanding
Answer each question, then compare with the solutions below.
- What is the E-step posterior \(q_r(b)\), and why must its normalizing constant be the current observed-data likelihood?
- In the changing-support example, what would go wrong if the likelihood ratio were set equal to \(E_{q_r}[R]\)?
- State the Jensen inequality step that turns a positive expected complete-data ratio into an auxiliary-function difference.
- Does exact EM monotonicity imply that the algorithm finds the global maximum? Explain.
Solutions. (1) \(q_r(b)=f(y,b;\theta^{(r)})/L(\theta^{(r)})\); integrating the numerator over \(b\) gives \(L(\theta^{(r)})\), so this choice makes \(q_r\) integrate to one. (2) It would omit candidate mass on the part of the latent space where the current density is zero; in the example, the expectation is \(1.2\) while the likelihood ratio is \(1.4\). (3) For \(R\geq0\) with the required finite expectations, \(\log E_{q_r}[R]\geq E_{q_r}[\log R]=Q(\theta\mid\theta^{(r)})-Q(\theta^{(r)}\mid\theta^{(r)})\). (4) No. Monotonicity prevents a decrease in likelihood under the exact-update conditions, but EM can converge to a local maximum or another stationary point.