Why Estimation Limits Need a Budget
An estimation run can stop because its convergence conditions are satisfied, because it encounters a numerical or model failure, or because it reaches a user-specified computational limit. These outcomes are not interchangeable. A run that reaches its iteration or evaluation cap has been interrupted by a safeguard; the cap does not establish that the estimates are sufficiently stable or that convergence has occurred.
Limits are useful because population-model evaluations can be costly, especially when many subjects, observations, or parameters are involved. But a cap chosen without reference to the optimizer's work can stop a healthy run too early. Conversely, a very large cap does not fix a poorly scaled model, an uninformative data set, or an optimizer that is repeatedly unable to make progress. The practical task is to estimate the expected computational work, allow a reasonable margin, and then interpret the termination record alongside the convergence evidence.
The distinction matters because one iteration may require several objective evaluations. A gradient may be approximated by finite differences, a line search may try several candidate parameter vectors, and the algorithm may evaluate the objective again to check a proposed step. In other implementations, some of this work is reused or counted differently. Analytic gradients can reduce the number of separate objective calls needed for derivative information, but their computation still consumes time. Thus, an iteration cap alone is not a reliable estimate of run time.
The earlier material on gradient-based convergence criteria explains how convergence is assessed from quantities such as parameter changes, objective changes, or gradients. Here the concern is different: how much work a run is permitted to do before it must stop. Keep the two settings separate in configuration and in reporting.
Relating Iterations, Evaluations, and Cost
For a rough planning calculation, let \(I\) be the anticipated number of iterations and let \(q\) be the average number of objective-function evaluations per iteration. If each evaluation takes approximately \(t\) seconds, a first-order wall-time estimate is
This approximation assumes that the per-evaluation cost and the number of evaluations per iteration are reasonably stable. A more realistic budget allows for an initial evaluation, final checks, and iteration-to-iteration variation:
Here, \(q_k\) is the number of objective evaluations associated with iteration \(k\), and \(t_k\) is their representative cost. This expression follows by adding the time spent on each evaluation, then including work outside the counted iterations. It also shows why a single average can understate cost when some regions of parameter space take longer to evaluate than others. The approximation is a planning tool, not a guarantee of run time.
Parameter dimension gives a useful starting point for predicting evaluation counts when gradients are numerical. With \(p\) estimated parameters, a forward-difference gradient commonly requires about \(p+1\) objective evaluations at a point: one at the base parameter vector and one for each perturbed parameter. If the base value is already available and reused, the additional calls may be closer to \(p\). A line search can add evaluations for trial steps. These counts are approximate because implementations may reuse results, use different difference schemes, or count requests differently. Check the software's definition of its evaluation counter rather than treating a nominal count as universal.
A useful practical workflow is to run a pilot with the same data, model, estimation method, and computing environment intended for the final analysis. Record elapsed time, iterations, evaluations, and whether the objective-evaluation cost is steady. Use this evidence to set limits that permit the expected work plus a margin. The margin should reflect uncertainty in convergence behavior and run-time variability; no single multiplier is appropriate for every model.
Confirm whether an iteration means an accepted update or an attempt, and whether evaluations include gradient perturbations, line-search trials, or final checks.
Use a pilot run and record elapsed time and evaluation count. Include the actual data volume and model structure, rather than extrapolating from a much smaller model without checking.
Forecast evaluation demand from the pilot and expected optimizer behavior. Set iteration and evaluation limits high enough for that work, while retaining a finite safeguard against unexpectedly long runs.
After fitting, compare the actual counts and trace with the convergence conditions. A cap reached before those conditions are satisfied is a capped run, not a converged result.
Worked Budgets for Population Models
Worked Example: Finite-Difference Work for a Small PopPK Model
Hypothetical setup. A small population pharmacokinetic model estimates \(p=12\) parameters using a deterministic optimizer. A pilot suggests about 40 iterations may be needed. Assume the software computes a forward-difference gradient using 13 objective evaluations per iteration, including the base point, and that a line search averages three additional trial evaluations per iteration. Suppose each evaluation takes 0.80 seconds. Assume no evaluations are reused and that one initial evaluation is needed before the counted iterations.
The expected evaluations per iteration are \(13+3=16\). Including the initial evaluation, the total is
At 0.80 seconds per evaluation, the estimated computation time is
A planning cap near 641 evaluations would leave essentially no allowance for variability. A more defensible cap would be selected from the pilot's observed variation and the available compute budget, then checked against the software's actual counting rules. The calculation estimates work; it does not show that 40 iterations are enough for convergence.
Worked Example: Analytic-Gradient Budget for a Moderate Model
Hypothetical setup. Consider a population model with 200 subjects and 800 observations, estimated with an analytic gradient. In a pilot, an objective-and-gradient request takes 1.4 seconds, and each of two line-search trials takes 1.0 second. The pilot suggests 50 iterations. Assume one combined objective-and-gradient request and two trial evaluations per iteration, plus one initial objective evaluation costing 1.0 second. Assume this timing remains representative across the run.
The evaluation count is
The time estimate distinguishes the combined request from the trial evaluations:
This budget is based on measured cost rather than assuming that one iteration equals one evaluation. If later iterations take longer, or the search tries more candidate steps, the actual run can exceed the estimate. The evaluation cap should therefore be checked against the expected work and the pilot's variability, not copied mechanically from the iteration count.
Worked Example: Scaling a Budget for a Larger Model
Hypothetical setup. A larger population model has 600 subjects and 2,400 observations. A short pilot completes 10 iterations in 6 minutes, with 46 objective evaluations. Assume the pilot is representative enough for a provisional estimate, and the planned run is expected to need 70 iterations. The pilot's average evaluations per iteration are \(46/10=4.6\), and its average time per evaluation is \(360/46=7.826\) seconds.
Using those averages, the expected evaluation count for 70 iterations is
The corresponding time estimate is
This is an extrapolation, not a guarantee. A 10-iteration pilot may not capture later line-search behavior, changing evaluation cost, or whether the optimizer needs more iterations than expected. A prudent budget would include a margin informed by additional pilot runs or observed variability. If the estimate is close to a hard computing limit, the appropriate response is to improve the forecast or redesign the computational plan, not to claim that the model will finish in 42 minutes.
Choosing and Interpreting the Limits
Set the iteration cap and the evaluation cap for different purposes. The iteration cap protects against an optimizer taking many update steps; the evaluation cap protects against expensive internal work per step. For example, a model with finite-difference gradients or repeated line-search trials may use many evaluations even when its iteration count is modest. An evaluation cap can therefore bind first. Conversely, an algorithm may make many inexpensive updates and reach its iteration cap without using the full evaluation allowance.
Use the pilot to identify which cap is likely to bind. If the run is expected to require \(I\) iterations and about \(q\) evaluations per iteration, a provisional evaluation forecast is \(I q\), with extra allowance for initialization, final checks, and variability. Set both caps to accommodate that forecast when feasible. If an evaluation cap is so low that it is likely to interrupt the anticipated run, it is not a meaningful convergence safeguard: it is a planned early stop and should be treated as such.
The expected convergence behavior must be judged from evidence, not from a universal iteration count. Similar fits may converge in different numbers of updates because of starting values, parameterization, data, and the local shape of the objective. A pilot's count is informative only to the extent that the final run uses comparable settings and conditions. Repeated runs that need substantially different work should widen the budget estimate and prompt investigation of the cause.
Some estimation procedures use stochastic updates or schedules rather than the same iteration structure as a deterministic optimizer. Their iteration count may refer to a prescribed sequence of updates, while objective evaluations may be occasional or defined differently. In that case, interpret caps according to the algorithm's own counting rules and keep its scheduled computational work distinct from a deterministic optimizer's accepted steps. Do not translate one method's iteration count directly into another method's without evidence.
Diagnostics, Alternatives, and Limitations
When a run stops, inspect its termination reason and counts together. Reaching an iteration or evaluation cap before convergence conditions are met means the result is incomplete with respect to those conditions. If the objective is still improving and the gradient or parameter changes remain outside the specified tolerances, a larger budget may be warranted, provided the trace suggests steady progress. If many evaluations occur with little useful progress, simply raising the cap may only spend more time on the same behavior. Review the objective trace, numerical stability, scaling, and the optimizer's behavior before deciding.
A practical alternative to raising a cap indiscriminately is to improve the computational plan: benchmark the actual model, use a better-informed starting point when justified, or use derivative calculations and algorithm settings suited to the model. These are not substitutes for checking convergence. Likewise, a smaller cap can be appropriate for exploratory runs used to detect gross problems, but such runs should not be presented as final estimates unless they satisfy the required convergence checks.
There are important limits to budget extrapolation. Evaluation time may vary by parameter vector, and large models may not scale proportionally with subject or observation counts. Some computations reuse intermediate results; others incur overhead not reflected in an evaluation counter. Parallel hardware, software versions, and resource contention also affect elapsed time. A count is often more reproducible than wall time, but only if the software's count definition is recorded. Report the iteration cap, evaluation cap, actual counts, termination reason, and the pilot or timing assumptions used to plan the run.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A run reaches its iteration cap while its convergence conditions remain unmet. What does this establish about the run?
- A numerical-gradient calculation uses 8 parameters. Under a forward-difference scheme with one base evaluation, approximately how many evaluations are needed for one gradient calculation, before any line-search trials?
- A pilot averages 5 evaluations per iteration. Estimate the evaluation count for 30 iterations, ignoring initial and final evaluations.
- A hypothetical fit needs 180 evaluations at 2.5 seconds each. What is the estimated evaluation time in minutes?
- Why can an evaluation cap be reached before an iteration cap even when the iteration count is modest?
Solutions. (1) It establishes only that the run stopped at its safeguard; it does not establish convergence. The unmet conditions mean the run should be identified as capped. (2) The approximate count is \(8+1=9\) evaluations: one at the base point and one perturbation for each parameter. (3) \(30(5)=150\) evaluations. This is a forecast based on the pilot average, not a guaranteed count. (4) \(180(2.5\ \mathrm{s})=450\ \mathrm{s}\), and \(450/60=7.50\) minutes. (5) An iteration can require several evaluations, for example to calculate a numerical gradient and assess trial steps, so total evaluations can accumulate faster than iterations.