Why Replicate-Level Failures Matter
A trial simulation evaluates a specified design and analysis procedure across many generated trial data sets. That evaluation is only interpretable if the analysis can be tracked for every replicate, including replicates for which fitting or downstream calculations do not produce a usable result. Otherwise, a reported rejection rate, bias, or coverage can describe only the subset that happened to finish successfully.
A numerical warning does not always mean an analysis is unusable, and a software message saying “converged” does not guarantee that the estimate is reliable. The useful question is not simply whether the software stopped, but whether the result satisfies prespecified numerical and model-based checks that make it suitable for the intended calculation. Those checks should be defined before reviewing simulation results.
These categories overlap but are not interchangeable. A boundary estimate can be a valid result, such as an estimated variance of zero when zero is allowed. A singular fit may be numerically converged yet signal that the fitted covariance structure is not supported by that data set. Conversely, a nonconverged fit can sometimes return a plausible-looking estimate that must not be treated as valid merely because the estimate is finite.
Define Statuses Before Running the Simulation
For each simulated replicate, preserve enough information to determine what happened from data generation through the final decision. At minimum, record the replicate identifier, the simulated scenario and parameters, whether data generation completed, the analysis status, key estimates and standard errors when available, the optimizer status, and the final endpoint required by the simulation. Save warnings and diagnostic quantities rather than reducing all outcomes to a single “success” flag.
A practical status scheme can distinguish: valid fit; fit completed but a diagnostic requires review; nonconvergence; singular fit; boundary estimate; and downstream analysis failure. More than one flag may apply to the same replicate. For example, a fit may converge, have a variance estimate at its lower boundary, and then fail when a standard error cannot be calculated. Status definitions should state which conditions are automatic failures and which trigger review. Avoid changing these definitions after seeing which choice produces more favorable operating characteristics.
State which outputs must exist and which diagnostics must pass for a replicate to count as analyzable. Identify boundary results that are allowable under the model.
Store the optimizer code, objective value, iteration information, gradient or score diagnostics, Hessian information where appropriate, warnings, and relevant parameter estimates.
Tabulate each status by scenario and inspect whether failures cluster by generated sample size, event count, covariate pattern, or other simulated features.
Calculate operating characteristics using the defined treatment of failures, and report both the failure rate and any conditional summaries among usable fits.
What Numerical Diagnostics Can and Cannot Establish
An optimizer generally seeks a parameter value that minimizes an objective function or maximizes a likelihood. A stopping message may indicate that a change in the objective, parameter values, or gradient met the software’s tolerances. These are numerical criteria, not proofs that the statistical model is identifiable, appropriate, or well estimated.
For a likelihood maximization, a small gradient near an interior solution supports the claim that the algorithm is near a stationary point. It does not by itself establish that the point is a maximum rather than a saddle point, that the likelihood is well behaved, or that the estimate is precise. The curvature, often summarized by the Hessian, should be examined using suitable scale-aware checks. A poorly conditioned or non-positive-definite information matrix can indicate weak identification, a flat direction, or proximity to a boundary. The meaning of a Hessian warning depends on the parameterization and model; it should be investigated, not mechanically translated into either “valid” or “failed.”
Useful checks include refitting from more than one reasonable starting value, comparing objective values and estimates, inspecting gradients and curvature, and verifying that all required derived quantities are finite and defined. Where parameters have constrained ranges, confirm that the reported estimate respects those constraints and determine whether a boundary value is expected or concerning. In nonlinear mixed-effects models, inspect estimated variances and correlations as well as the optimizer status; a covariance matrix can be singular even when the fixed-effect estimates appear stable.
Worked Example: Classifying Fit Outcomes
In a hypothetical simulation of 200 trial replicates, the analysis is a mixed-effects model with a random intercept variance constrained to be nonnegative. The prespecified review rule requires a finite objective and estimates, an acceptable optimizer status, and no unresolved numerical warning. An estimated variance of zero is flagged for review, not automatically classified as a failure.
Suppose 184 replicates meet the numerical checks. Of these, 12 have a random-intercept variance estimate of zero; in 10, the remaining estimates and required outputs are stable across alternative starts, while in 2 they are not. Another 10 replicates do not meet the optimizer stopping criteria, and 6 return a non-finite standard error.
The 10 zero-variance fits with stable checks are boundary estimates, but they are not automatically analysis failures under this rule. The 2 zero-variance fits with unresolved instability require the prespecified review outcome; if the rule says unresolved instability makes the required result unusable, count them as failures. The 10 nonconverged fits are convergence failures. The 6 non-finite standard errors are downstream analysis failures, even if their optimization status was otherwise acceptable.
Thus, 184 replicates pass the initial numerical checks, but 12 have a flagged boundary or instability requiring explicit disposition. If the two unstable boundary fits are unusable, the final usable count is 182 and the analysis-failure count is 18: 10 nonconverged, 6 with invalid standard errors, and 2 unresolved fits. The 10 stable boundary fits remain in the analysis set, with their boundary status reported separately. This classification follows the stated rule rather than treating every boundary estimate as either automatically valid or automatically failed.
Boundary Estimates and Singular Fits
Boundary estimates require interpretation in light of the parameter space and scientific model. A variance parameter cannot be negative, so an estimate of zero lies on the boundary. Zero may represent a scientifically meaningful limiting case, or it may arise because a particular replicate contains too little information to estimate that variance. In either case, reporting the boundary estimate and checking its consequences is more informative than replacing it with a small positive number without justification.
A singular fit is a statement about the fitted random-effects covariance structure, not simply a synonym for failed optimization. For example, a random-effects covariance matrix may have an eigenvalue effectively equal to zero, indicating that the estimated structure has fewer supported dimensions than specified. Correlations near plus or minus one can accompany this pattern. Such a fit can produce unstable standard errors or make the model’s intended decomposition unsupported by the data. The analyst should check whether estimates and conclusions are sensitive to the singular direction and whether the model’s required inferential output remains defined.
Diagnostics are model-dependent. A variance at zero may be a legitimate estimate under a model that allows zero, while a covariance estimate outside its permitted range is invalid. A generic threshold for “near zero” or “near singular” is not universally appropriate: numerical scale, parameterization, sample size, and the intended inferential task matter. Specify tolerances using the model and software, and report them.
Why Excluding Failed Replicates Can Distort Results
Let \(S=1\) indicate that a replicate yields a usable analysis under the prespecified rule, and \(R=1\) indicate that the rule’s trial decision is a rejection. If a failed analysis is operationally treated as no rejection, then the rejection probability of that complete procedure is
because \(R=1\) can occur only when \(S=1\). The conditional rejection rate among successful fits, \(P(R=1\mid S=1)\), is not the same quantity as the unconditional rejection probability of this procedure unless all replicates succeed or the rates happen to coincide. If failure instead leads to a fallback analysis or an inconclusive decision, that operational rule defines a different outcome and must be applied to every replicate.
The more general concern is selection: the probability of success may depend on the generated data. For example, sparse events, extreme covariate patterns, or estimates near a boundary may make fitting harder and may also be associated with the trial endpoint. The successful replicates can then differ systematically from all simulated trials. A complete-case summary describes performance conditional on successful analysis, not necessarily performance in the target population of simulated trials.
Worked Example: Complete-Case Rejection Rate Versus Procedure Performance
Consider 1,000 hypothetical replicates under a null scenario. The prespecified operational rule declares success only when the primary model produces a usable result; a failed primary analysis leads to no rejection. The model succeeds in 900 replicates, of which 180 reject the null.
The rejection rate among successful fits is
But the operational rejection rate across all simulated trials is
These answer different questions. The 20.0% rate is conditional on a usable fit. The 18.0% rate is the probability that the specified procedure rejects when failures result in no rejection. If this is a type I error assessment, the unconditional rate is the relevant one for that operational rule; the 10.0% analysis-failure rate must also be reported. Discarding the 100 failures and reporting only 20.0% would conceal how often the procedure could not produce its primary analysis.
Whether “no rejection” is the right treatment depends on the decision process being simulated. It is not a universal convention. If the trial would repeat a fit, use a prespecified fallback, or declare the result inconclusive, represent that action explicitly. Do not count a failed fit as a successful non-rejection for one summary and silently omit it for another. Performance summaries should state their denominator and the rule used for failed replicates.
Worked Example: Detecting Selective Failure
Suppose 500 hypothetical alternative-scenario replicates are divided by a generated baseline-risk feature into two groups. In the lower-risk group, 20 of 250 fits fail and 69 of the 230 successful fits reject. In the higher-risk group, 80 of 250 fits fail and 102 of the 170 successful fits reject.
The failure rates are \(20/250=8.00\%\) and \(80/250=32.0\%\), respectively. Among successful fits, rejection rates are
The pooled complete-case rate is \(171/400=42.75\%\). It weights the two groups according to their numbers of successful fits, not their proportions in the simulated target population. If failure leads to no rejection, the overall operational rejection rate is instead \(171/500=34.2\%\). The gap between the group-specific failure rates is a warning that exclusion is selective. The complete-case rate alone cannot establish how the primary analysis would have behaved in failed replicates; it only describes those that succeeded.
Comparison, Limitations, and a Practical Reporting Standard
There are several defensible summaries, but they answer different questions. An unconditional operational rate applies the trial’s prespecified response to every replicate, including failure and fallback rules. A conditional rate among usable fits describes the analysis when it succeeds and should be labelled as conditional. A sensitivity analysis can compare plausible failure dispositions, such as no rejection versus a specified fallback, but each disposition is a distinct procedure. Simply deleting failed replicates is not a neutral solution: it changes the population being summarized and can introduce selection.
| Approach | What it summarizes | Important qualification |
|---|---|---|
| Apply the operational rule to all replicates | Performance of the specified procedure | The rule for failure must be explicit and realistic. |
| Summarize usable fits only | Performance conditional on analysis success | Report the denominator and failure rate; success may be selective. |
| Use a prespecified fallback | Performance of a primary-plus-fallback procedure | Apply the fallback to every eligible failure and document its conditions. |
| Exclude failures without a rule | Only the retained subset | Generally does not estimate the full procedure’s operating characteristics. |
Simulation cannot reveal the unobserved primary-analysis result that a failed replicate would have produced if the fitting algorithm had succeeded. Nor can any single diagnostic threshold guarantee that an analysis is scientifically adequate. Report the status definitions, software and tolerances relevant to convergence, counts by status, scenario-specific failure rates, and the denominator for every performance measure. Where failures are frequent or concentrated in important scenarios, improve the model or numerical strategy, define a justified fallback, or acknowledge that the proposed procedure is not reliably evaluable under those conditions.
Check Your Understanding
Answer each question using the distinction between a fit diagnostic and the operating rule for the trial procedure.
- In 400 replicates, 360 fits are usable and 90 of those reject. If failure is defined to produce no rejection, calculate the conditional and unconditional rejection rates.
- A variance estimate is zero, but the optimizer converged and all required outputs are stable. Must this replicate automatically be counted as an analysis failure? Explain.
- What does a complete-case rejection rate estimate when fit success depends on a simulated data feature?
- Name two checks that can help investigate a converged fit with a suspicious Hessian or unstable estimates.
Solutions. (1) The conditional rate is \(90/360=25.0\%\). The unconditional rate under the stated failure rule is \(90/400=22.5\%\); the failure rate is \(40/400=10.0\%\). (2) No. A zero variance is a boundary estimate, not automatically a failure. Apply the prespecified rule and check whether the required outputs and intended inference remain usable. (3) It estimates rejection conditional on success. If success is selective, that subset may not represent all simulated trials, so the conditional rate need not equal the procedure’s unconditional operating characteristic. (4) Examples include refitting from alternative reasonable starting values, inspecting gradients and curvature, and checking estimate stability and required derived quantities. The chosen checks should match the model and be specified before interpreting the result.