Slow EM Convergence In Joint Models

Why EM Can Move Slowly

The expectation-maximization algorithm (EM) is useful when a model contains latent variables: quantities that are part of the model but are not observed directly. In a shared-random-effects joint model, subject-specific random effects are latent. They link the longitudinal marker trajectory to the event hazard, so information about them comes from both the repeated measurements and the event or censoring history. This shared structure can also make an EM iteration move only a small distance, even when it is correctly implemented.

Slow progress matters in practice because an apparently stable fit may reflect tiny iteration-to-iteration changes rather than a well-characterized solution. The central question here is how latent-effect uncertainty and survival association can make EM slow, and how likelihood increments and parameter changes together help assess progress.

Joint-Model Likelihood and the EM Update

For subject \(i\), let \(y_i\) denote the observed longitudinal measurements, \(T_i\) the observed event or censoring time, and \(\delta_i\) the event indicator, with \(\delta_i=1\) for an event and \(0\) for right censoring. Let \(b_i\) be the subject’s latent random-effects vector. A common shared-parameter specification has

Here \(m_i(t;b_i)\) is a longitudinal mean or other specified marker feature, \(\alpha\) is its association parameter in the hazard, and \(\beta,\gamma\) are regression parameters. The exact specification may differ, but the key feature is that \(b_i\) enters both data models. The observed-data likelihood integrates over the random effects:

The parameter vector \(\theta\) contains the model parameters being estimated. In the complete data, the \(b_i\)'s are treated as if observed. At iteration \(k\), the E-step computes the conditional expectation of the complete-data log likelihood given the observed data under the current estimate \(\theta^{(k)}\). The M-step chooses \(\theta^{(k+1)}\) to maximize that expected log likelihood.

Why Coupling Creates Slow Convergence

An EM iteration does not simply plug in one estimate of each subject’s random effect. It averages over the conditional distribution of \(b_i\) given that subject’s marker measurements and event history. When \(\alpha\) is appreciable, the event history can be informative about \(b_i\): a subject’s event time changes the relative support for random-effect values associated with higher or lower hazard. At the same time, changing the random-effects distribution or the longitudinal parameters changes that conditional distribution. Thus the survival association and latent-effect parameters are updated in a mutually dependent way.

If the latent effects were nearly known from the data, there would be little missing information and EM would often move quickly. If substantial uncertainty remains, the E-step averages across plausible latent-effect values. Parameter changes that would be clear if \(b_i\) were observed can then be softened by this averaging. Strong dependence among parameter directions can make this effect more pronounced: one parameter may partly compensate for another while leaving the observed likelihood nearly unchanged.

This local result assumes a differentiable likelihood and update map, a regular interior solution, and nonsingular information matrices. It describes behavior near a fixed point; it does not guarantee that an algorithm has reached the global maximum. In a joint model, the random-effects and survival-association components can contribute to directions with substantial missing information. Coupling alone does not prove that convergence will be slow: the amount of information in the observed data and the model’s parameterization also matter.

A Short Justification for the Local Rate

The rate result connects the EM update to the information lost by not observing the latent effects. Write \(M(\theta)\) for one exact EM update, so that \(\theta^{(k+1)}=M(\theta^{(k)})\). At a fixed point \(\theta^\ast\), \(M(\theta^\ast)=\theta^\ast\). Under the regularity conditions above, differentiating the first-order condition for maximizing \(Q\) gives

Thus, locally, the error vector is transformed by \(I_c^{-1}I_m\) at each iteration. In a direction with rate eigenvalue \(\lambda\), the error is approximately multiplied by \(\lambda\) per iteration. For example, if \(\lambda=0.97\), roughly 97% of the current error in that direction remains after one iteration. This explains slow progress without implying that the algorithm is malfunctioning.

Assessing Progress with Likelihood and Parameter Changes

For exact EM on the exact observed-data likelihood, each update does not decrease the likelihood. To see why, let \(q_k\) be the conditional distribution of the latent effects under \(\theta^{(k)}\). Jensen’s inequality gives

An exact M-step makes the right-hand side nonnegative by maximizing \(Q\), so \(\ell(\theta^{(k+1)})\geq\ell(\theta^{(k)})\). This is a monotonicity statement for exact EM, not a claim that each likelihood increment must be large. A small increment may mean that the algorithm is close to a solution, but it can also arise while parameters continue to change along a relatively flat direction.

Therefore examine both the observed log-likelihood increment and the parameter changes. A likelihood increment is \(\Delta\ell_k=\ell(\theta^{(k+1)})-\ell(\theta^{(k)})\). Parameter changes should be judged on meaningful scales: a raw change of 0.01 has different importance for a parameter measured in years than for one measured on a log scale. A scaled change can be formed by dividing each parameter change by a prespecified, meaningful scale and taking a norm. Such summaries describe movement; they do not, by themselves, establish convergence.

Diagnostics, Alternatives, and Limitations

A useful progress record includes the observed log likelihood at each iteration, the increment, and changes in all scientifically important parameters. It is also sensible to inspect combinations of parameters that are known to be strongly associated, such as random-effects variation and survival association. A steadily shrinking increment together with shrinking scaled parameter changes is consistent with local convergence. A tiny increment alongside persistent parameter movement can instead indicate weak identification, a flat ridge, or movement between compensating parameter values.

Check that likelihood evaluations use consistent and sufficiently accurate numerical integration settings. Otherwise, small apparent increments may reflect evaluation error rather than true progress. The earlier material on gradient checks and Hessian diagnostics provides complementary evidence: near-zero increments alone do not show that the score is small or that uncertainty estimates are reliable. Comparing fits from different starting values can also reveal whether similar likelihood values correspond to distinct parameter estimates.

EM’s advantage is that its E-step uses the conditional distribution of latent effects, avoiding direct optimization of every integral as though it were an ordinary closed-form likelihood. Its limitation is precisely that the averaging can make updates conservative. Direct likelihood maximization, including numerical optimization of the integrated likelihood, can move more rapidly when gradients and curvature are accurately computed, but it requires reliable likelihood and derivative calculations. Hybrid strategies may use EM to obtain stable initial estimates and a different optimizer for refinement. None removes weak identification: if the observed data provide little information in a parameter direction, a faster optimizer does not create information that is absent.

A frequent mistake is to treat a very small likelihood increment as proof that the parameter estimates have stabilized. The increment measures improvement in fit, not distance traveled in parameter space. Conversely, a small parameter change in one iteration does not guarantee that the fit is at a maximum; it may be a slow step. Assess both quantities, and interpret them alongside numerical accuracy and local diagnostics.