Binned Residual Trends and Uncertainty

Why Bin Residuals?

A residual-versus-prediction or residual-versus-time plot can contain many points, making a modest systematic pattern difficult to see. Binning groups nearby values of the horizontal variable and summarizes the residuals in each group. The resulting sequence of summaries can make a broad trend easier to inspect, while uncertainty intervals indicate how precisely each bin mean is estimated.

A binned plot is a summary, not a replacement for the observations. Its appearance depends on the residual definition, the variable used to form bins, the boundaries, and how uncertainty is calculated. A changing sequence of bin means can suggest a pattern worth investigating; it does not by itself establish a cause or prove that a model is misspecified.

Definition: Let \(r_i\) be the chosen residual for observation \(i\), and let \(x_i\) be the value used for binning, such as a prediction or time. For bin \(b\), let \(I_b\) be the set of observations whose \(x_i\) values fall in that bin, and let \(m_b\) be its size. The binned residual mean is \(\bar{r}_b = \sum_{i \in I_b} r_i / m_b\). It describes the average residual among observations assigned to that bin.

The horizontal location of a bin summary is often shown at the mean or midpoint of the \(x_i\) values in that bin. State which convention is used, especially when bins are unequal in width or observations cluster near one edge. For time, bins might represent prespecified intervals such as 0–2 and 2–4 hours. For predictions, they might be fixed-width intervals or intervals chosen to contain roughly equal numbers of observations.

Uncertainty Around a Bin Mean

An interval around \(\bar{r}_b\) describes uncertainty in estimating the mean residual for that bin. It does not show the range that contains most individual residuals. That spread is reflected by the residual values themselves and their sample standard deviation.

For a simple pointwise confidence interval, suppose the \(m_b\) residuals in a bin are independent observations from a distribution with a common mean and variance, and that the residuals are approximately normally distributed. With a small bin, the normality assumption matters more. Let \(s_b\) be the sample standard deviation of the residuals in the bin. The standard error is \(s_b/\sqrt{m_b}\), and a two-sided \(100(1-\alpha)\%\) t interval is:

\bar{r}_b \ \pm\ t_{1-\alpha/2,\,m_b-1}\frac{s_b}{\sqrt{m_b}}

The t multiplier accounts for estimating the residual standard deviation from the bin data. The interval is pointwise: it applies to the mean in one bin under the stated assumptions. A set of pointwise 95% intervals is not a simultaneous 95% band for all bin means or for an entire trend.

Why the Standard Error Has This Form

If the residuals in a bin are independent with common variance \(\sigma^2\), the variance of their mean is the variance of their sum divided by \(m_b^2\). Independence makes the variance of the sum equal to \(m_b\sigma^2\), so the variance of the mean is \(\sigma^2/m_b\). Replacing the unknown \(\sigma\) with the sample standard deviation \(s_b\) gives the estimated standard error.

\operatorname{Var}(\bar{r}_b) = \operatorname{Var}\left(\frac{1}{m_b}\sum_{i \in I_b}r_i\right) = \frac{m_b\sigma^2}{m_b^2} = \frac{\sigma^2}{m_b}, \qquad SE(\bar{r}_b)=\frac{s_b}{\sqrt{m_b}}

The independence assumption deserves particular attention in pharmacometric data. Several observations from one subject are usually not equivalent to observations from different subjects: residuals from the same subject may be correlated. Treating every row as independent can make the standard error too small. When within-subject dependence matters, use a method that respects subjects, such as calculating one summary per subject and analyzing those summaries, or resampling whole subjects rather than individual rows. The target of the summary must also be clear: an observation-weighted mean and an equally weighted mean across subjects need not be the same.

Worked Example: A Pointwise Interval for One Prediction Bin

Suppose a hypothetical prediction bin contains four independent, dimensionless residuals: \(-0.8\), \(-0.4\), \(0.2\), and \(0.6\). Assume the observations are independent and the residual distribution is sufficiently close to normal for a t interval to be reasonable. Calculate the bin mean and its 95% interval.

The mean is:

\bar{r}_b = \frac{-0.8-0.4+0.2+0.6}{4} = \frac{-0.4}{4}=-0.100

The squared deviations from \(-0.100\) are \(0.49\), \(0.09\), \(0.09\), and \(0.49\). Their sum is \(1.16\), so the sample variance and standard deviation are:

s_b^2=\frac{1.16}{4-1}=0.3867, \qquad s_b=\sqrt{0.3867}=0.6218

Thus \(SE=0.6218/\sqrt{4}=0.3109\). With \(df=3\), the 95% t multiplier is approximately \(3.182\), giving:

-0.100 \pm 3.182(0.3109) = -0.100 \pm 0.989 = [-1.089,\ 0.889]

Under the assumptions, this is a 95% confidence interval for the mean residual in this bin. It includes zero, but that does not prove that the mean is exactly zero. It is also wide: four observations provide limited precision.

Bin Choice Changes the Picture

There is no universally correct number of bins. Narrow bins retain local detail but may have noisy means. Wide bins are more stable but can average over a real change, or conceal a local feature. Fixed-width bins preserve the scale of the horizontal variable; quantile bins can make bin counts more even, but their widths may vary substantially. Either strategy can be misleading if chosen after repeatedly inspecting the residual pattern and selecting the version that looks most favorable.

Boundaries also matter. A value on a boundary must be assigned consistently, for example by defining bins as left-closed and right-open except for the final bin. Record the binning rule and show the number of observations per bin. If the purpose is to compare diagnostics across runs, using a consistent rule can make plots more comparable; if the prediction ranges differ greatly, the same rule may leave some runs with poorly populated bins.

Worked Example: Two Reasonable Binnings, Different Apparent Detail

Consider eight hypothetical observations ordered by prediction. Their prediction values are \(1,2,3,4,5,6,7,8\), and their residuals are \(-1,-1,-1,-1,1,1,1,1\). Compare four fixed-width bins, each containing two observations, with three wider bins.

With four bins, the residual means are:

\frac{-1-1}{2}=-1,\quad \frac{-1-1}{2}=-1,\quad \frac{1+1}{2}=1,\quad \frac{1+1}{2}=1

The sequence is \(-1,-1,1,1\), making the transition between the fourth and fifth observations clear. Now use bins containing observations 1–3, 4–6, and 7–8. The means are:

\frac{-1-1-1}{3}=-1,\quad \frac{-1+1+1}{3}=\frac{1}{3},\quad \frac{1+1}{2}=1

The wider-bin sequence, \(-1, 0.333, 1\), suggests a gradual rise rather than locating the abrupt change. Both summaries are arithmetically correct; they answer questions at different resolutions. The original residual plot is needed to see the observations and judge whether the selected binning obscures an important feature.

Repeated Observations and Subject-Level Uncertainty

If a bin contains repeated records from the same subjects, the ordinary t interval can understate uncertainty because it counts correlated records as independent information. One practical alternative, when the target is an average across subjects, is to calculate each subject’s mean residual within the bin and then summarize those subject means. This treats the subject, rather than each record, as the independent unit. It is appropriate only when subjects are independent and the subject-level summaries correspond to the intended estimand.

A subject-level bootstrap is another option: sample subjects with replacement, keeping all records from each selected subject together, and recalculate the binned summary for each resample. The resulting distribution can be used to form an interval, subject to adequate numbers of independent subjects and a suitable bootstrap procedure. Resampling individual records instead would break the within-subject structure.

Worked Example: An Interval Based on Independent Subjects

Suppose five hypothetical subjects each contribute several observations to the same time bin. Their within-bin mean residuals are \(-0.4\), \(0.0\), \(0.2\), \(0.4\), and \(0.8\). Assume the subjects are independent and the subject means are approximately normal. Here the target is the equally weighted mean across subjects.

The mean of the subject summaries is \(1.0/5=0.200\). The deviations from \(0.200\) are \(-0.6,-0.2,0,0.2,0.6\); their squared sum is \(0.80\). Therefore:

s_{\text{subject}}=\sqrt{\frac{0.80}{5-1}}=0.4472, \qquad SE=\frac{0.4472}{\sqrt{5}}=0.2000

Using \(df=4\) and \(t_{0.975,4}=2.776\), the 95% interval is:

0.200 \pm 2.776(0.200) =0.200\pm0.555 =[-0.355,\ 0.755]

This interval concerns the equally weighted mean of the five subject-level summaries, not necessarily the mean across all observation records. If subjects contribute different numbers of observations, those two targets can differ. The uncertainty calculation must match the estimand as well as the dependence structure.

Reading the Plot and Choosing an Approach

A useful diagnostic display shows the individual residuals as well as the bin means and intervals. Points reveal spread, outliers, clustering, and gaps that a mean can hide. The bin summaries provide a compact view of average direction. If bin boundaries are uneven, show their ranges or the number of records in each bin so readers can interpret the horizontal positions.

Binning is not the only way to summarize a residual trend. A scatterplot without bin summaries preserves every observation but can be visually dense. A smooth curve can show gradual changes without imposing hard boundaries, but its appearance depends on the smoothing method and its settings. Binned means are transparent and easy to reproduce, but can change when boundaries change and can miss within-bin structure. These displays complement one another; none removes the need to inspect the data and state how the summary was made.

A binned plot is most useful when the binning rule reflects the diagnostic question, the intervals match the sampling structure, and the individual observations remain available for context. Avoid interpreting overlapping pointwise intervals as a formal test of whether bins differ: overlap is not a general test for a trend, and a series of intervals does not account for simultaneous comparisons. Likewise, an interval that includes zero is not evidence that all residual behavior in the bin is satisfactory.

Common Mistakes and Practical Checks

Confusing interval width with residual spread. A confidence interval describes precision of the bin mean, not the range of individual residuals. Show the residual points or a spread summary as well.
Using the row count as the independent sample size. With repeated observations, rows from one subject may be correlated. Use a dependence-aware uncertainty calculation.
Changing bins until a preferred pattern appears. The trend can be sensitive to boundaries. State the rule, inspect reasonable alternatives, and avoid selecting bins solely for a desired visual result.
Reading a pointwise interval as a whole-plot guarantee. Separate 95% intervals do not provide 95% simultaneous coverage for all bins or the full trend.
Hiding the data behind summaries. A bin mean can conceal opposing residuals or local structure. Retain the underlying points and report bin counts.

Key takeaway: Binned residual means can clarify broad trends, but their uncertainty depends on the independent sampling unit and their appearance depends on bin definitions. Report the binning rule, show the data where practical, and interpret the intervals only for the quantity and assumptions they represent.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A bin contains \(m_b=9\) independent residuals with sample standard deviation \(s_b=0.6\). What is the estimated standard error of the bin mean?
Why can treating 40 records from four subjects as 40 independent observations give an overly narrow interval?
A plot uses quantile bins. What is one advantage and one limitation of this choice compared with fixed-width bins?
What does a pointwise 95% interval around one bin mean describe, and what does it not describe?
Two binnings of the same residual data show different trends. What should be checked before deciding that one plot reveals the model behavior?

Solutions. (1) \(SE=0.6/\sqrt{9}=0.200\) residual units. (2) Records from the same subject may be correlated, so they do not supply as much independent information as 40 unrelated observations; a row-level calculation can underestimate uncertainty. (3) Quantile bins can make counts more even, but their widths vary and may blur the horizontal scale. Fixed-width bins preserve equal ranges but can have uneven counts. (4) Under its assumptions, it gives a confidence interval for the mean residual in that bin; it is not an interval for individual residuals or a simultaneous band for the full trend. (5) Check the bin boundaries, counts, horizontal range, and individual residual plot; the different summaries may reflect different resolutions rather than a calculation error.