Diagnosing Joint Model Fit And Convergence

Why Joint-Model Diagnostics Need More Than One Question

A joint efficacy-toxicity model can produce a dose recommendation even when its computation, information content, or fit deserves concern. Those are distinct issues. A sampler may fail to explore a well-defined posterior; a sampler may accurately explore a posterior that is broad because the data barely distinguish parameter values; or the model may fit poorly despite stable computation. Diagnostics are useful only when they identify which of these problems is plausible.

The model considered here assigns each participant an efficacy outcome and a toxicity outcome at an assigned dose. Both outcomes may be binary, ordinal, or otherwise modelled jointly. The particular likelihood can vary, but the diagnostics below require that the analyst state the fitted model and assumptions clearly: participants' outcome pairs are independent conditional on assigned dose and included covariates; the joint likelihood represents the recorded outcomes; and the dose-response and dependence structure have been specified. For Bayesian fitting, the prior must also be stated. If dose assignment adapts to prior outcomes, the assignment history should be recorded; when inference conditions on assigned doses and the allocation probabilities contain no unknown outcome-model parameters, the outcome likelihood can be evaluated conditional on those doses. Adaptive allocation can nevertheless leave some dose regions with little information.

In practice, diagnose in layers. First check computation, then ask whether the data provide information about the parameters, and then inspect whether fitted predictions reproduce important features of the observed joint outcomes. The earlier material on posterior predictive checks introduced comparison of observed data with replicated data; here the emphasis is on localizing patterns and choosing a response to them.

Checking Computation Before Interpreting Posterior Summaries

For simulation-based posterior inference, inspect trace plots by chain and parameter, and examine numerical summaries such as split-\(\hat R\) and effective sample size. A trace plot should show chains moving through the same posterior region rather than remaining in separate bands, drifting persistently, or making only rare jumps between regions. A satisfactory-looking trace is not a proof of convergence, so use multiple diagnostics together.

For \(m\) chains with \(n\) retained draws each, a basic variance comparison underlying \(\hat R\) uses the average within-chain variance \(W\), the between-chain variance \(B\), and the estimated marginal variance \(\hat V\):

If chains explore the same stationary distribution, their within-chain and between-chain variation should become similar, bringing \(\hat R\) close to 1. Modern implementations commonly use split chains and rank-based refinements, so software-reported values need not equal this basic calculation exactly. A value notably above 1 is a warning to investigate, not a direct measure of how wrong an estimate is. Effective sample size (ESS) estimates how many independent draws would provide comparable Monte Carlo information; strong autocorrelation can make ESS much smaller than the number of stored draws.

This example also shows why one number should not dictate the decision. Check the trace, \(\hat R\), bulk and tail ESS, sampler warnings, and whether relevant functions of parameters are adequately sampled. If a dose recommendation depends on a derived probability or the joint distribution of two parameters, assess computation for that quantity as well as for individual coefficients. A stable mean for one parameter does not guarantee adequate exploration of a tail or joint region.

Identifiability and Sparse Dose Information

Structural identifiability is a property of the model's mapping from parameters to observable-data distributions. If distinct parameter values always imply distinct distributions, the parameter is structurally identifiable under that model. Practical identification is different: a model can be structurally identifiable yet have data so sparse or unbalanced that the likelihood is nearly flat along some parameter combinations. This often occurs in dose finding when few participants are observed at some doses, when outcomes are rare, or when several flexible dose-response and dependence parameters are estimated together.

A prior can regularize a weakly informed parameter and make posterior computation stable. It cannot make the data themselves distinguish parameter values. Therefore, compare prior and likelihood information, inspect likelihood profiles or posterior contours where appropriate, and check whether estimates change substantially under reasonable prior alternatives. This is a diagnostic of how much the data contribute, not a reason to choose a prior merely because it produces a narrow interval.

For less extreme cases, look for strong posterior correlations, broad or irregular likelihood profiles, and estimates that move substantially when a small number of observations or plausible prior choices change. These are warnings of weak practical identification, not automatic proof that the model is invalid. They should prompt a substantive review of which parameters the available dose allocation and outcome counts can support.

Residual Patterns in Joint Outcomes

A joint model can reproduce each marginal response rate while failing to reproduce the pairing of efficacy and toxicity outcomes, or it can fit the joint table at one dose while missing a dose trend. Inspect observed and model-expected counts for the joint outcome categories across dose groups. For an observed count \(O\) and its fitted expected count \(E\), the Pearson residual is \(r=(O-E)/\sqrt{E}\). Its sign indicates whether the observed count exceeds or falls below expectation; its magnitude gives a scale for discrepancy relative to the model's count variability. For sparse expected counts, treat these residuals descriptively rather than relying on a large-sample reference distribution.

Plot residuals or observed-versus-predicted summaries by dose and outcome category rather than examining only a single overall discrepancy. A pattern that repeats across neighbouring doses is more informative than one isolated cell, especially when expected counts are small. If posterior predictive summaries show a systematic pattern, first verify coding, outcome definitions, dose labels, and the calculation of expected values. Then consider whether the model structure or the available information can plausibly account for the pattern. Do not use convergence remedies to address a lack of fit: they answer different questions.

Posterior Dependence and Remedies Matched to the Problem

Joint models often have correlated posterior parameters. For example, a steeper efficacy dose slope and a shifted efficacy intercept can produce similar predictions over the observed dose range. Dependence parameters can also be associated with marginal dose-response parameters. Correlation between parameter draws is not the same thing as association between the efficacy and toxicity outcomes: the former describes uncertainty about model parameters, while the latter is a feature represented by the joint outcome distribution.

A practical comparison of remedies is useful. For chain separation or poor mixing, investigate initialization and sampler diagnostics, then address parameter scaling, parameterization, or model geometry before extending runs. For weak identification, assess dose and outcome information, simplify unsupported parameters, and report prior sensitivity rather than presenting prior-driven precision as data-driven evidence. For patterned residuals, verify the data and inspect the model's ability to represent the relevant outcome pattern; changing sampler settings will not repair systematic prediction discrepancies. These remedies are not interchangeable.

A common mistake is to say, “The chains converged, so the joint model fits and its parameters are identified.” This reasoning is wrong: chains can accurately sample a broad, weakly identified posterior, and stable computation does not guarantee predictions compatible with observed data. The correction is to report the three assessments separately: computational adequacy, information about parameters, and predictive fit. Likewise, a narrow posterior interval should not be described as proof of identification unless the likelihood itself meaningfully distinguishes the parameter values.