Why Use More Than One Starting Value?
A numerical optimizer searches for a high value of the objective function, but its path can depend on where the search begins. In joint models, the likelihood may be difficult to optimize because longitudinal and event-time components jointly inform shared parameters, random-effects covariance parameters, and baseline-hazard parameters. A fit that reports convergence from one initial value therefore does not, by itself, show that the optimizer found the best available solution.
A practical check is to fit the same model repeatedly from a set of dispersed, valid initial values, then compare the attained log likelihoods, parameter estimates, and gradients. Agreement across runs is reassuring, though it cannot prove that a global maximum has been found. Disagreement is informative: it may indicate competing local maxima, weak identification, sensitivity to numerical approximation, a boundary solution, or simply an optimization run that did not finish adequately.
The Objective Being Compared
Let \(\theta\) collect the unknown parameters of a joint model. For example, it may contain longitudinal fixed effects, survival parameters, an association parameter, and random-effects covariance parameters. For subject \(i\), let \(y_i\) denote the longitudinal data, \(T_i\) the observed event or censoring time, and \(\delta_i\) the event indicator. If \(b_i\) is the subject-specific random-effects vector, a typical likelihood contribution integrates over its distribution:
The formula describes the objective conceptually; in computation, the integral may be evaluated using a numerical approximation. For a fair multi-start comparison, every run must use the same likelihood definition and the same integration or approximation settings. Otherwise, a difference in reported log likelihood could reflect a changed objective rather than a different solution to one optimization problem.
The task is to maximize \(\ell(\theta)\) over the admissible parameter space. If the software minimizes a negative log likelihood instead, compare its values with the sign reversed: a smaller negative log likelihood corresponds to a larger log likelihood. Constraints must also be respected. For instance, a covariance matrix must be positive definite, so starting values should be generated in a parameterization that produces valid covariance matrices rather than by assigning arbitrary entries.
Why the Comparisons Are Informative
At an interior differentiable local maximum, a necessary condition is that the score, the vector of first derivatives of the log likelihood, is zero. This gives a reason to inspect gradients as well as objective values. The condition is necessary, not sufficient: a point with a small gradient need not be a maximum, and numerical and boundary complications can affect the calculation. Detailed derivative verification is a separate diagnostic. Here, the gradient is used as a screening measure of whether an optimizer has plausibly reached a stationary point.
The basic justification follows from a first-order Taylor expansion. For a small change \(h\) in the parameter vector,
If an interior point has a nonzero gradient, choosing a sufficiently small step in the gradient direction gives a first-order increase: for \(h=c\nabla\ell(\theta)\) with small \(c>0\), the leading change is \(c\lVert\nabla\ell(\theta)\rVert^2>0\). Thus the point cannot be an interior local maximum. Conversely, a small gradient only says that no large first-order improvement is indicated; it does not establish that the point is a maximum or that the best solution has been found.
Raw gradient components can be hard to compare when parameters use different units or scales. A derivative with respect to a variance parameter may have a different numerical magnitude from one with respect to a regression coefficient simply because their scales differ. A useful screening summary is the norm of the gradient in scaled coordinates. If \(u_j=(\theta_j-\theta_{j,\mathrm{ref}})/s_j\), where \(s_j\) is a meaningful positive scale, then \(\partial\ell/\partial u_j=s_j\,\partial\ell/\partial\theta_j\). Report the scaling convention when using such a norm; it is not an invariant pass/fail threshold.
Record the model, data, constraints, likelihood approximation, and optimizer settings so every run targets the same objective.
Vary plausible values across important parameter blocks, including association and covariance parameters, while respecting the model's admissible parameter space.
For each start, save the terminal estimates, attained log likelihood, convergence status, and an appropriately scaled gradient summary.
Check both objective values and parameter vectors. Investigate runs with lower likelihood, materially different estimates, large gradients, or warnings rather than selecting solely by a software status label.
Worked Comparisons
Worked Example: Two Candidate Solutions With Different Likelihoods
Suppose three hypothetical starts are fitted to the same joint model with identical numerical settings. The reported log likelihoods are \(-412.80\), \(-413.25\), and \(-412.80\). The first and third runs give nearly identical estimates, including an association estimate of \(0.71\); the second gives an association estimate of \(-0.18\). Their scaled gradient norms are \(0.002\), \(0.006\), and \(0.002\), respectively. Assume the software reports successful termination for all three.
The largest log likelihood is \(-412.80\), attained by starts one and three. The log-likelihood difference between this value and the second run is
At the respective fitted parameter vectors, the likelihood ratio is \(\exp(\Delta\ell)=\exp(0.45)\approx1.568\). Thus the likelihood at the better candidate is about 1.57 times the likelihood at the second candidate, for this fixed dataset and model. This ratio is a comparison of likelihood values, not a probability that one parameter vector is true.
The near agreement of starts one and three in likelihood, estimates, and gradient norm supports reproducibility of that candidate. The second run has a lower likelihood and a materially different association estimate, so it should not be treated as an equally supported fit merely because it terminated successfully. The runs do not prove that the first solution is globally optimal; they show that, among these candidates, it is the strongest and is reproduced by two starts.
Worked Example: A Close Likelihood Is Not Enough
Consider two hypothetical terminal fits with log likelihoods \(-286.40\) and \(-286.70\). Their gradient components with respect to three original parameters are, respectively, \((0.004,-0.002,0.030)\) and \((0.001,0.020,-0.400)\). To summarize gradients, suppose the chosen parameter scales are \(s=(1,0.5,0.02)\), so each scaled component is \(s_j\) times the corresponding raw derivative.
For the first fit, the scaled gradient is \((0.004,-0.001,0.0006)\). Its Euclidean norm is
For the second fit, the scaled gradient is \((0.001,0.010,-0.008)\), giving
The first fit has the higher log likelihood by \(0.30\), and its scaled gradient norm is smaller. These facts make it the more convincing candidate of the two. The second fit's objective is fairly close, but its larger scaled gradient suggests that it may not yet be near a stationary point. The conclusion is not that a particular norm is universally acceptable: interpretation depends on the scaling, numerical precision, and fitting procedure. The scaling and the component values should therefore accompany the summary.
Worked Example: Repeated Fits Reveal a Competing Basin
In a hypothetical eight-start analysis, five runs end at \(\ell=-527.10\), with association estimates near \(0.64\), covariance estimates near \(0.31\), and small scaled gradient norms. Two runs end at \(\ell=-528.02\), with association estimates near \(-0.35\), covariance estimates near \(0.27\), and similarly small scaled gradient norms. One run terminates with a warning, \(\ell=-527.11\), and a scaled gradient norm of \(0.20\).
The warned run should not be counted as a reliable competing optimum: its gradient is relatively large, and the warning indicates that the numerical termination needs investigation. The other seven runs form two reproducible candidate groups. The difference between their log likelihoods is
The likelihood at the higher-valued candidate is about 2.51 times that at the lower-valued one. The second group nevertheless has small gradients, so the lower value cannot be dismissed simply as an unfinished run. These results indicate competing, plausibly stationary solutions for this optimization. The substantially different association estimates warrant particular attention: the likelihood surface may have separate favorable regions, or the parameter may be weakly identified. Multiple starts alone cannot distinguish those explanations or establish that either candidate is the unique global maximum.
Designing Starts and Diagnosing Trouble
A useful set of initial values should explore different plausible regions rather than repeat almost identical values. Varying one parameter block at a time can help identify which changes lead to different fits, while some starts should vary several blocks together because their effects may interact. Include starts near reasonable estimates as well as deliberately dispersed but admissible values. For covariance parameters, use a parameterization that preserves validity, as discussed in covariance parameterizations and numerical stability.
For reproducibility, document how starts were generated, the optimizer and its settings, the numerical integration settings, and the outcome of every run. If random perturbations are used, record the random seed. A run that fails, hits an invalid parameter region, or stops with a warning is not evidence for an optimum. Reparameterizing or adjusting an initial value may be appropriate, but it should not silently change the model or objective.
Several patterns are diagnostically useful:
- Same estimates and objective from many starts: evidence that the solution is robust to the starts tried, but not proof of global optimality.
- Similar objectives but different estimates: possible weak identification, a flat region, or genuinely competing solutions. Compare the parameters that matter scientifically and do not report a single solution without noting the sensitivity.
- Different objectives and different estimates: the higher objective is preferable if both runs are valid fits to the same objective, but the lower fit may still reveal a competing stationary region.
- Large gradients or warnings: a convergence label should not override evidence that the reported point may not be adequately optimized.
- Boundary estimates: interpret cautiously. At a boundary, the interior zero-gradient condition need not apply in the same way, and a parameter may be effectively constrained by the edge of its admissible space.
A multi-start comparison is not a substitute for other checks. Similar output across starts can occur because the starts all lie in the same basin; numerical integration error can also affect apparent differences. Conversely, tiny differences in the last reported digits need not indicate distinct solutions. The next diagnostic steps depend on the observed pattern and may include examining the likelihood surface or checking derivative calculations. The central task here is to identify when a single successful run is not enough.
How This Check Differs From Alternatives
A single-start fit is computationally cheaper, but provides no direct evidence about sensitivity to the initial value. Multiple starts address that limitation by exploring more than one initial point. A profile likelihood investigates how the objective changes as a selected parameter is fixed or varied while other parameters are optimized; it can clarify a weakly supported parameter direction, but it is not the same as checking whether several starts reach the same solution. Likewise, information-matrix diagnostics assess local curvature and uncertainty near a fit, while multi-start analysis asks whether the optimizer reaches different candidate regions. These checks answer related but distinct questions.
The practical conclusion should be proportionate to the evidence. Say that the best likelihood found was reproduced by several dispersed starts, or that materially different candidate solutions were observed. Avoid saying that multi-start fitting has proven a global maximum, or that a particular estimate is uniquely determined when alternative fits remain plausible.
Check Your Understanding
Answer each question using the distinctions between objective comparison, stationarity, and evidence from repeated starts.
- Two runs use the same data and model, with log likelihoods \(-301.2\) and \(-301.8\). Which has the larger likelihood, and by what likelihood ratio?
- Why is it invalid to rank fits by their log likelihoods if they used different numerical integration settings?
- Several starts give nearly identical estimates and log likelihoods, but all starts were close together. What conclusion is justified, and what is not?
- A fit has a small gradient norm but a lower log likelihood than another fit with a small gradient norm. What should be concluded?
Solutions. (1) The fit at \(-301.2\) has the larger likelihood. The log-likelihood difference is \((-301.2)-(-301.8)=0.6\), so the ratio is \(\exp(0.6)\approx1.822\). (2) Different integration settings can define different numerical approximations to the objective, so the difference cannot be attributed solely to the starting values or the attained solution. (3) The results show reproducibility within the region explored by those starts. They do not show that other regions, including one containing a better solution, have been searched. (4) Both fits may be near stationary, but the higher-log-likelihood fit is preferable among them if both are valid and use the same objective. The lower fit may still be a competing local solution; a small gradient alone does not make it the best candidate.