Reading Parameter Traces During SAEM

Why Read Parameter Traces?

An SAEM run produces a sequence of parameter estimates, not just a final estimate. Looking at that sequence helps reveal whether estimates appear to settle, continue moving, alternate between values, or change abruptly. This matters because a plausible-looking final estimate can conceal an unstable path, while ordinary iteration-to-iteration variation does not by itself show that the algorithm has failed.

A trace is a plot or ordered list of an estimated quantity against SAEM iteration. For interpretation, know which quantity is plotted, its units and parameterization, whether each point is recorded before or after the M-step, and where the gain schedule changes. Traces are most informative when the model, data, estimation settings, and iteration definitions remain consistent throughout the run.

For example, if \(\beta\) is a fixed effect and \(\omega^2\) is a random-effect variance, their traces are not interchangeable with traces of the standard deviation \(\omega\). Since \(\omega=\sqrt{\omega^2}\), equal changes on the two scales have different meanings. Always read the axis label and estimation output before interpreting the shape.

What a Settling Trace Does—and Does Not—Show

A trace that moves substantially early and then fluctuates within a comparatively narrow band is often described as settling or stabilizing. A trace with a persistent direction of movement may indicate drift; repeated swings between high and low values may indicate oscillation. A sharp step is an abrupt change. These are descriptions of the displayed sequence, not formal diagnoses of the cause.

SAEM updates are stochastic: latent-effect simulation contributes randomness to the sequence of estimates. Therefore, a parameter trace need not move monotonically, even when the run is behaving reasonably. Conversely, a nearly flat trace is not sufficient evidence that the estimates are accurate, that the desired solution has been reached, or that the model is appropriate. Interpret trace shape in context and alongside other run information.

Comparing Changes on a Meaningful Scale

The visual impression of movement depends on units and axis limits. A change of 0.1 may be large for a parameter typically near 0.05 and small for one typically near 100. A simple numerical companion to the plot is the iteration-to-iteration change,

When a parameter has a meaningful nonzero reference scale \(s_j\), a scaled change can be reported as \(\Delta_{j,k}/s_j\). The reference scale must be stated; it might be a scientifically relevant scale or a typical magnitude chosen for comparing changes. A relative change \(\Delta_{j,k}/|\theta_{j,k-1}|\) is inappropriate when the previous value is zero or close to zero, and can exaggerate movement in that situation.

This comparison has a simple justification. If a parameter sequence converges to a finite limit \(\theta_j^*\), then both \(\theta_{j,k}\) and \(\theta_{j,k-1}\) approach \(\theta_j^*\), so their difference approaches zero:

This is a necessary implication for a convergent sequence, not a sufficient test: small increments can occur temporarily during drift, or because the display hides changes. A trace should therefore be inspected over a sufficiently long range, with readable axes and units, rather than judged from a few final iterations alone.

Fixed Effects, Variance Components, and Objective Quantities

Fixed-effect traces show estimates of population-level model parameters, such as a typical clearance or an effect of a covariate. Interpret each in its natural units and in the model’s parameterization. A fixed effect on a log scale, for instance, is not a direct change in the original measurement scale.

Variance-component traces require extra care. Variances are nonnegative, and an estimate near zero may be on a boundary of the parameter space. A variance trace can appear compressed near zero on a broad linear axis. If the software displays a log variance or log standard deviation instead, the trace can move by equal amounts on a multiplicative rather than additive scale. State which quantity is plotted; do not infer a variance from a standard-deviation trace without squaring it.

An objective-related trace might show an observed-data log-likelihood estimate or another reported objective quantity. In an NLME model, the observed-data log-likelihood involves integrating over the latent random effects. If the displayed values are estimated using simulation, they can vary because of that estimation as well as because the parameter estimates change. SAEM’s stochastic updates also mean that one should not presume that every iteration must improve a displayed objective. Check what the software reports and how it computes the quantity before comparing values.

Parameter and objective traces answer related but different questions. Parameter traces show which estimates are changing; objective-related traces show how a reported fit criterion behaves. A relatively flat objective trace can coexist with movement in parameters, particularly when different parameter combinations give similar fit. A changing objective trace does not by itself identify which parameter caused the change.

Recognizing Abrupt Changes and Using Complementary Evidence

An abrupt change is a sharp movement relative to the trace immediately before and after it. Possible explanations include a change in the algorithm’s operating phase, a boundary-related update, an unstable numerical step, an unexpected change in inputs, or an error in how estimates were recorded. The plot identifies when the change occurred; it does not identify its cause. Check the iteration log, parameter constraints, gain schedule, software messages, and saved estimates around that iteration.

The following hypothetical example shows why a parameter trace and an objective trace should be read together, without assuming that either settles the diagnosis.

Trace plots are a quick visual diagnostic, but they have limitations. Axis truncation can make small changes look dramatic, while a wide axis can hide them. A long trace can compress early behavior; a short window can make a transient look persistent. Plot related parameters in separate, clearly labelled panels when their units differ, and record whether variance components are shown as variances, standard deviations, or transformed values.

Parameter traces complement rather than replace other evidence. A reported objective can add information about fit, while the trace of a particular parameter shows how that estimate moved. Neither should be confused with the sequence of quantities used internally by the stochastic approximation. If a parameter trace remains puzzling, inspect the run records and other model diagnostics rather than trying to resolve every question from its plot alone.

Common Mistakes

• Calling a flat segment proof of convergence. Small changes over a short window can occur during a pause or temporary plateau. Describe what is visible and inspect the broader run.
• Expecting every SAEM update to improve the objective. Stochastic updates and simulation-based objective estimates can fluctuate. First establish exactly which objective quantity is plotted and how it was evaluated.
• Comparing unlike parameter scales. A change of 0.01 has no universal interpretation. State units and use a meaningful scale when comparing movement.
• Confusing variance with standard deviation. If the trace is \(\omega\), it is not a trace of \(\omega^2\). Read the label and transform only when appropriate.
• Explaining a jump from the plot alone. A trace reveals the timing and size of a change, not its cause. Check run records and computation settings around the jump.