Why Residual Size Is Not the Whole Story
A subject can have a large residual without materially affecting the population estimates. Conversely, a subject can have fairly modest residuals yet strongly affect an estimated parameter because that subject contributes information in a part of the design that few other subjects cover. These are different diagnostic questions: does the subject fit poorly, and does the fitted model depend heavily on that subject?
In an NLME analysis, observations from one subject are a cluster, and that subject can affect both the population parameters and the estimated between-subject variability. Thus, a useful review combines residual and individual-fit displays with subject-level influence checks. The results are prompts for investigation, not automatic rules for deleting data.
These descriptions are not interchangeable. A large residual indicates discrepancy between an observation and a prediction; leverage concerns how much information or sensitivity the subject contributes; influence concerns the actual change in a fit when the subject is removed. A subject may have any combination of these properties.
Residual Flags and Subject-Level Displays
The earlier material on raw, weighted, and conditional residuals and on standardizing residuals establishes why residual size must be judged on an appropriate scale. In a residual display, a residual far from the main cloud is a reason to inspect the observation and its context. When examining subjects, however, do not reduce a whole profile to its largest residual without looking at time, predictions, sampling density, and the subject’s individual-fit display.
A practical screening convention is to investigate standardized residuals with absolute values around 2 or larger, and especially those around 3 or larger. These are heuristics, not universal cutoffs or formal outlier tests. The reference distribution may be approximate, residuals within one subject are dependent, and many residuals are examined at once. A single large value can occur by chance, particularly in a large data set.
Worked Example: A Large Residual Within an Otherwise Adequate Profile
Suppose two hypothetical subjects have concentrations measured at three times. The individual predictions are shown with the observations. For this illustration, assume an additive residual standard deviation of 0.50 concentration units at every time, so the standardized residual is the raw residual divided by 0.50.
| Subject | Time | Observation | Individual prediction | Raw residual | Standardized residual |
|---|---|---|---|---|---|
| A | 1 | 4.20 | 4.00 | 0.20 | 0.40 |
| A | 2 | 2.90 | 3.00 | -0.10 | -0.20 |
| A | 3 | 3.35 | 2.00 | 1.35 | 2.70 |
| B | 1 | 3.80 | 4.00 | -0.20 | -0.40 |
| B | 2 | 3.10 | 3.00 | 0.10 | 0.20 |
| B | 3 | 2.15 | 2.00 | 0.15 | 0.30 |
For subject A at time 3, the residual is \(3.35-2.00=1.35\) concentration units, and the standardized residual is \(1.35/0.50=2.70\). This is a point worth checking. Subject B has no comparably large residual in these three observations. But the table alone does not establish that subject A is an influential subject: it says nothing about how the population fit changes if A is omitted. Nor does the large residual establish an error; the observation, timing, assay record, and model adequacy all need review.
Leverage in a Local Approximation
In a nonlinear model, fitted values depend nonlinearly on parameters. A common way to understand leverage is to approximate the model locally by a linear model around the fitted parameter vector. Let \(\boldsymbol{\theta}\) denote the population parameters and let \(\boldsymbol{\mu}(\boldsymbol{\theta})\) be the vector of model predictions. Near the fitted estimate \(\hat{\boldsymbol{\theta}}\), a first-order Taylor expansion gives
Here \(J\) is the sensitivity matrix: its entries describe how predictions change locally as parameters change. If \(W\) is the working weight matrix (the inverse of the working residual covariance in a weighted least-squares approximation), the local projection matrix for the whitened predictions is
This expression assumes the local information matrix \(J^\mathsf{T}WJ\) is invertible. It is the weighted analogue of the familiar linear-regression projection matrix. Its diagonal elements in a single-observation setting describe how much each observation’s fitted value is locally tied to that observation. For a subject with several observations, the corresponding block \(H_{ii}\) can be summarized in various ways, such as by its trace. That is a local, model-dependent leverage measure, not a universal NLME subject-leverage statistic.
Subject Deletion and Influence
The most direct diagnostic is a leave-one-subject-out comparison. Fit the same model to all subjects, then refit it after removing subject \(i\), keeping the model specification, data handling, and estimation procedure otherwise consistent. Denote the full-data estimate by \(\hat{\boldsymbol{\theta}}\) and the estimate without subject \(i\) by \(\hat{\boldsymbol{\theta}}_{(-i)}\). The parameter change is
For parameters with different units, compare changes relative to their uncertainty or to a meaningful scale, rather than comparing raw differences. One possible scaled summary for \(q\) parameters with estimated covariance matrix \(\widehat{\mathrm{Cov}}(\hat{\boldsymbol{\theta}})\) is
This gives a dimensionless measure of the deletion change relative to parameter uncertainty. It is a diagnostic summary, not a test with a universal cutoff. In NLME models, deletion can affect fixed effects, variance components, and predictions differently; variance estimates near a boundary and convergence problems can make such summaries especially difficult to interpret.
Worked Example: Leverage and Influence in a Linearized Illustration
Consider three hypothetical subject-level summaries \(y\) related to a covariate \(x\) by a local straight-line approximation with equal weights: \((x,y)=(0,2),(1,3),(2,5)\). This small example illustrates the distinction between residual and leverage. It is not a substitute for fitting and checking a nonlinear mixed-effects model.
The least-squares slope is \( \hat{\beta}_1=\sum (x_i-\bar{x})(y_i-\bar{y})/\sum (x_i-\bar{x})^2 \). Here \(\bar{x}=1\), \(\bar{y}=10/3\), the numerator is \((-1)(2)+(0)(3)+(1)(5)=3\), and the denominator is \((-1)^2+0^2+1^2=2\). Therefore \(\hat{\beta}_1=3/2=1.5\), and \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=10/3-1.5=11/6\), or approximately 1.833.
The fitted values at \(x=0,1,2\) are \(11/6,10/3,29/6\), approximately 1.833, 3.333, and 4.833. The residuals are approximately 0.167, -0.333, and 0.167. Thus, neither endpoint has a large residual in this small example.
For a line with an intercept, leverage is \(h_i=1/n+(x_i-\bar{x})^2/\sum (x_j-\bar{x})^2\). With \(n=3\), the endpoint leverages are \(1/3+1/2=5/6\) each, while the middle leverage is \(1/3\). The endpoints have higher leverage because their covariate values are farther from the design center.
Deleting the subject at \(x=2\) leaves \((0,2)\) and \((1,3)\), giving slope \(1\) and intercept \(2\). The slope changes from \(1.5\) to \(1\), a change of \(-0.5\). Deleting the subject at \(x=0\) leaves \((1,3)\) and \((2,5)\), giving slope \(2\) and intercept \(1\), a slope change of \(+0.5\). This illustrates why high leverage can matter even when the residual is small. The tiny sample makes these changes unstable; it is the distinction, not a numerical cutoff, that carries over to NLME diagnostics.
Worked Example: Interpreting a Leave-One-Subject-Out NLME Refit
Suppose a hypothetical NLME model estimates typical clearance as \(5.20\) L/h with a standard error of \(0.20\) L/h using all subjects. After omitting one subject and refitting the same model, typical clearance is \(4.78\) L/h. The full-data fit estimates the between-subject standard deviation on the log-clearance scale as \(0.30\), with standard error \(0.05\); the deletion fit gives \(0.24\).
For typical clearance, the deletion change is \(5.20-4.78=0.42\) L/h. Relative to the full-data standard error, this is \(0.42/0.20=2.10\). The relative change from the full-data estimate is \(0.42/5.20=0.0808\), or about 8.08%. For the log-clearance standard deviation, the change is \(0.30-0.24=0.06\), which is \(0.06/0.05=1.20\) full-data standard errors.
These hypothetical results suggest that the subject materially affects the typical clearance estimate and also affects the estimated variability. They do not show that the subject is erroneous or should be excluded. The next steps are to examine that subject’s observations and individual predictions, verify records and model implementation, and check that both fits converged to comparable solutions. If the deletion fit did not converge reliably, the apparent parameter change could reflect estimation failure rather than subject influence.
Practical Review and Common Pitfalls
A disciplined review separates flagging from explanation. First locate the unusual observation or subject in residual and individual-fit displays. Then check whether the pattern is supported by the data record and whether it persists across appropriate displays. Finally, if the question concerns impact on population estimates, use a subject-deletion refit rather than inferring influence from residual size alone.
Identify unusual standardized residuals or an individual profile that departs from its predictions. Record which observations and times are involved.
Review the data record, sampling pattern, and individual-fit display. Consider whether the model is systematically inadequate, rather than assuming the subject is at fault.
Refit without the subject when feasible, compare parameters and predictions on meaningful scales, and verify convergence and estimation quality.
Describe the diagnostic and the deletion effect. Do not remove a subject solely because a residual or influence measure crosses a heuristic threshold.
A common mistake is to call the subject with the largest residual “the most influential.” That reasoning confuses discrepancy with impact. A correction is to inspect residuals for unusual fit, examine design and sensitivity for potential leverage, and compare full-data and deletion fits for actual influence. A second mistake is to interpret a large deletion change as proof of a data error. The change may be scientifically meaningful, reflect a sparse or unusual design region, reveal model sensitivity, or result from unstable estimation.
The main alternatives answer different questions. Residual displays are efficient for locating lack of fit but do not measure parameter impact. Local leverage summaries can identify subjects with unusual design or sensitivity, but rely on a local approximation and depend on how the NLME model and covariance are represented. Leave-one-subject-out refitting is more direct about impact, but can be computationally demanding and may become unstable when few subjects remain or parameters are weakly identified. No single diagnostic replaces the others, and deletion findings should be interpreted alongside the scientific model and data quality.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A subject has one standardized residual of 3.1, but omitting the subject barely changes the population estimates. Which diagnostic property is clear, and which is not?
- Why can a subject with small residuals still have high leverage?
- A full-data clearance estimate is 6.0 L/h and the estimate after omitting one subject is 5.7 L/h. Calculate the deletion change and its percentage relative to the full-data estimate.
- Why should a leave-one-subject-out fit that failed to converge not be interpreted as evidence of strong influence?
Solutions. (1) The large residual flags an unusual observation; the small parameter change indicates little measured influence on those estimates. (2) Leverage reflects design location and model sensitivity, not residual size; a subject in a sparsely represented region can have high leverage while fitting closely. (3) The change is \(6.0-5.7=0.3\) L/h, or \(0.3/6.0=0.05=5\%\) of the full-data estimate. (4) A failed fit does not provide a reliable estimate to compare: the difference may arise from estimation failure rather than a genuine effect of removing the subject.