Why Look at Residual Patterns?
A hierarchical model can produce a reasonable overall fit while representing one or more baskets poorly. A single unusually high residual may point to an outlying basket; residuals that rise with a covariate may reveal a systematic trend; and a wider spread of residuals than the model predicts may indicate that the assumed distribution of basket effects is inadequate. These patterns matter because borrowing information relies on the model’s description of how baskets relate to one another.
A residual is a difference between an observed outcome and a model-based expectation. In a basket analysis, it is useful to examine these differences by basket, rather than relying only on an overall summary. The purpose is diagnostic: a residual pattern can suggest where the model’s assumptions are strained, but it does not by itself identify the correct replacement model or establish that a clinical mechanism caused the pattern.
A Basket-Level Residual Model
For a concrete setting, suppose basket \(j\) has \(n_j\) patients and \(Y_j\) responses. A basic binary-outcome model is
Here \(\mu\) is the population-level log-odds, \(u_j\) is the basket-specific deviation, and \(\tau\) describes between-basket variation on the log-odds scale. This model assumes conditional binomial sampling within baskets, normally distributed exchangeable basket effects, and no additional systematic structure omitted from the linear predictor. If covariates or a treatment-control contrast are part of the intended model, the fitted probability must incorporate them; the residual should compare the observation with that model’s prediction on the outcome scale.
A residual calculated using a single fitted probability is useful for a first look, but it does not account fully for uncertainty about that probability. Let \(m_j=E(p_j\mid y)\) and \(v_j=\operatorname{Var}(p_j\mid y)\). For a replicated count \(Y_j^{\mathrm{rep}}\) drawn from the posterior predictive distribution,
The variance includes both binomial sampling variation and posterior uncertainty about \(p_j\). To see why, apply the law of total variance: \(\operatorname{Var}(Y_j^{\mathrm{rep}}\mid y)=E\{\operatorname{Var}(Y_j^{\mathrm{rep}}\mid p_j,y)\mid y\}+\operatorname{Var}\{E(Y_j^{\mathrm{rep}}\mid p_j,y)\mid y\}\). The first term is \(n_jE[p_j(1-p_j)\mid y]\); the second is \(n_j^2v_j\). Substituting \(E[p_j(1-p_j)\mid y]=m_j(1-m_j)-v_j\) gives the displayed result. A standardized predictive residual can therefore be formed as \((Y_j-n_jm_j)/\sqrt{\operatorname{Var}(Y_j^{\mathrm{rep}}\mid y)}\).
Worked Example: A High Response Count in One Basket
Suppose a hypothetical basket has \(n_j=20\) patients and \(Y_j=12\) responses. The fitted model gives posterior mean response probability \(m_j=0.30\) and posterior variance \(v_j=0.0025\). The posterior mean expected count is \(20(0.30)=6\), so the raw residual is \(12-6=6\) responses.
Ignoring uncertainty in \(p_j\), the conditional binomial standard deviation at \(p_j=0.30\) is \(\sqrt{20(0.30)(0.70)}=\sqrt{4.2}=2.049\). The corresponding conditional Pearson residual is \(6/2.049=2.928\).
Including posterior uncertainty, the predictive variance is \(20(0.30)(0.70)+(20^2-20)(0.0025)=4.2+380(0.0025)=5.15\). Thus the standardized predictive residual is \(6/\sqrt{5.15}=2.644\). It remains large, but is smaller because uncertainty in the fitted probability is included.
For an additional check, suppose 80 of 4,000 hypothetical posterior predictive counts for this basket are at least 12. The estimated upper-tail proportion is \(80/4000=0.0200\), with Monte Carlo standard error approximately \(\sqrt{0.0200(0.9800)/4000}=0.00221\). This is evidence that a count this high is relatively unusual under the fitted model, not a formal frequentist p-value or the probability that the model is false. The basket merits investigation alongside the model’s other diagnostics.
Reading Patterns, Not Just Large Residuals
A useful display places basket-level residuals against quantities that might explain systematic structure: a clinically meaningful subgroup feature, a standardized covariate, fitted response probability, sample size, or basket order when order has a substantive interpretation. For discrete outcomes, plot points can take only a limited set of values, especially in small baskets. That discreteness is expected; the question is whether the pattern is more pronounced than the fitted model predicts.
- One extreme basket: a single large residual can indicate an unusual basket, data issue, or influential observation. Check the basket definition, outcome coding, follow-up, and relevant patient or treatment features before proposing a model change.
- Residuals trending with a covariate: a gradual pattern suggests that the model may be missing a covariate effect, nonlinear relationship, or interaction. A trend should be assessed against uncertainty and predictive replications, not inferred from a line through a few noisy points.
- Too much spread: residuals that are more dispersed than replicated residuals can suggest that the model understates between-basket variation, has an unsuitable effect distribution, or omits a source of variation.
- Clusters or tails: separated groups or repeated extreme values may be inconsistent with a single smooth distribution of basket effects. Check whether the baskets are clinically coherent and whether a distinct mechanism or outcome process is plausible.
- Dependence on basket size: raw residuals naturally tend to be larger for larger baskets. Compare standardized or predictive residuals, and check whether residual spread still changes with \(n_j\).
These are clues, not diagnoses. For example, an apparent cluster can arise by chance when many baskets are inspected, and an outlier may reflect ordinary binomial variability. A residual pattern becomes more compelling when it is also unusual in posterior predictive replications and has a plausible clinical or design explanation.
Worked Example: Residuals Trending Across a Covariate
Consider four hypothetical baskets with \(n_j=20\) each. A model predicts the same response probability, \(p_j=0.40\), for all four, while their standardized covariate values are \(-1.5,-0.5,0.5,1.5\). The observed response counts are \(4,6,10,12\).
The expected count in each basket is \(20(0.40)=8\), and the conditional standard deviation is \(\sqrt{20(0.40)(0.60)}=\sqrt{4.8}=2.191\). The raw residuals are \(-4,-2,2,4\); the Pearson residuals are approximately \(-1.826,-0.9129,0.9129,1.826\), respectively.
The residuals increase in the same order as the covariate. For a simple numerical description, the cross-product of centered covariate values with residuals is \((-1.5)(-4)+(-0.5)(-2)+(0.5)(2)+(1.5)(4)=14\). Dividing by the sum of squared centered covariate values, \(1.5^2+0.5^2+0.5^2+1.5^2=5\), gives a residual-on-covariate slope of \(14/5=2.8\) responses per covariate unit. This is a descriptive pattern, not a test or a reliable estimate of a treatment effect from four baskets.
The pattern suggests checking whether the covariate belongs in the model or whether a nonlinear effect is plausible. It does not prove an omitted effect: four baskets provide little information, and such an ordering can occur by chance. The next diagnostic question is whether replicated datasets from the fitted model show a trend at least this pronounced.
Posterior Predictive Patterns and Alternative Diagnostics
Posterior predictive checks compare a feature of the observed data with the same feature computed for data replicated from the fitted model. For residual diagnostics, the feature might be the largest absolute standardized residual, a residual-versus-covariate slope, or the overall spread of residuals. For each posterior draw, simulate a replicated dataset under the model, calculate the chosen feature for both observed and replicated data using a consistent definition, and compare the results. A posterior predictive tail proportion describes how often the replicated feature is at least as extreme as the observed feature under this procedure.
This comparison helps calibrate an apparent pattern against the model’s own predictive variability. It is not a guarantee that the model is adequate: posterior predictive checks can have limited ability to reveal deficiencies, particularly when the same data informed the posterior. The earlier material on Posterior Predictive Checks Across Subgroups develops the general comparison; here the emphasis is on choosing residual-based features that reveal structure across baskets.
Leave-One-Subgroup-Out Predictive Diagnostics provide a complementary view. They assess prediction for a basket held out of the fit, whereas an in-sample residual is calculated after that basket has contributed to estimating the hierarchy. This distinction matters because shrinkage can pull an unusual basket’s fitted effect toward the overall distribution, making its in-sample residual appear less extreme. A surprising in-sample pattern should not be dismissed merely because a held-in fit adapts to it; conversely, a single weak leave-one-out prediction does not alone establish a systematic defect.
| Diagnostic | What it highlights | Main caution |
|---|---|---|
| Basket-level residual display | Direction, trend, clustering, or extreme baskets | Raw residual size depends on basket size and outcome variability |
| Posterior predictive residual comparison | Whether a chosen pattern is unusual under the fitted model | In-sample checks can be conservative and are not formal p-values |
| Leave-one-subgroup-out prediction | How well the model predicts a basket excluded from fitting | Prediction can be uncertain for a small or genuinely distinct basket |
| Single overall fit summary | Broad agreement for one selected feature | Can hide offsetting or subgroup-specific misfit |
Worked Example: Excessive Residual Spread
Suppose six hypothetical baskets each have \(n_j=20\), and the fitted model’s representative response probability is \(0.40\) for each. The observed response counts are \(2,3,4,16,17,18\). The expected count per basket is 8, and the conditional variance is \(20(0.40)(0.60)=4.8\).
The raw residuals are \(-6,-5,-4,4,5,6\). A simple measure of their standardized spread is \(T=\sum_j (Y_j-n_jp_j)^2/\{n_jp_j(1-p_j)\}\). Here, \(T=(36+25+16+16+25+36)/4.8=154/4.8=32.08\). The six residuals are balanced around zero, but their squared size is large relative to the assumed binomial variation.
Now suppose 30 of 4,000 posterior predictive replications have a value of this same discrepancy at least 32.08, giving \(30/4000=0.00750\) and Monte Carlo standard error \(\sqrt{0.00750(0.9925)/4000}=0.00136\). Under this hypothetical fitted model, the observed spread is unusually large. That directs attention to the model’s representation of basket-to-basket variation, possible omitted structure, and the clinical comparability of the baskets. It does not establish which explanation is correct. In an actual check, calculate the discrepancy consistently for observed and replicated data, including the parameter draw and predictive uncertainty specified by the model.
Assumptions, Failure Modes, and Limits
Residual conclusions depend on both the outcome model and the diagnostic being used. A binomial Pearson residual assumes that the conditional variance is \(n_jp_j(1-p_j)\); extra patient-level dependence or overdispersion can make it misleading. For continuous outcomes, residuals are formed on the response scale relative to the model mean and assessed using the model’s predictive variance. For survival outcomes, censoring and the time-to-event structure must be respected; a raw observed-minus-expected event count is not automatically an appropriate residual. In every case, use predictions for the actual observed design and outcome process.
For binary data, the residual distribution is discrete and often asymmetric near probability boundaries. Normal-reference cutoffs for Pearson residuals should therefore not be treated as exact. Posterior predictive comparisons are generally more informative when basket sizes differ, outcomes are sparse, or probabilities are near zero or one. Also inspect whether the model actually converged and whether posterior draws adequately represent uncertainty: unstable computation can create apparent patterns or obscure real ones.
A common mistake is to call every residual above 2 an outlier, or to treat a predictive tail proportion below 0.05 as proof of model failure. The correction is to inspect the pattern, account for uncertainty and discreteness, compare a pre-specified or clearly motivated discrepancy with predictive replications, and consider how many baskets and summaries were examined. A small tail proportion is a reason to investigate, not a binary verdict. Residual diagnostics also have limited power with few baskets, and they cannot resolve whether an unusual basket reflects measurement error, a distinct mechanism, or an unmodeled but genuine treatment-effect difference.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can a Pearson residual based on a posterior mean probability understate uncertainty in a basket’s replicated count?
- A basket has \(n_j=25\), \(Y_j=15\), and fitted probability \(p_j=0.40\). Calculate its raw residual and conditional Pearson residual.
- Residuals are increasingly positive as a clinically relevant covariate increases. Name two possible explanations and one important caution.
- What is the main difference between an in-sample residual check and leave-one-subgroup-out prediction?
Solutions. (1) It treats the fitted probability as fixed and omits posterior uncertainty in that probability; predictive variance includes both parameter uncertainty and binomial sampling variation. (2) The expected count is \(25(0.40)=10\), so the raw residual is \(15-10=5\). The conditional standard deviation is \(\sqrt{25(0.40)(0.60)}=\sqrt{6}=2.449\), giving a Pearson residual \(5/2.449=2.041\). (3) The model may omit a covariate effect or nonlinear structure, or the pattern may arise by chance; check it against predictive replications and account for the small number of baskets and multiple patterns inspected. (4) The in-sample residual uses a basket that contributed to fitting, while leave-one-subgroup-out prediction assesses prediction after excluding that basket from the fit.