Why Put Discrete Markers on a Common Residual Scale?
A residual for a binary or count marker cannot be interpreted in the same way as an unstandardized difference between an observed value and a fitted mean. Binary outcomes have only two possible values, while counts are discrete and often strongly skewed. Their raw differences therefore have restricted, unequal distributions that can make familiar residual plots difficult to interpret. Randomized quantile residuals address this by mapping each observed value to a point on a probability scale and then to a standard normal scale.
In a joint model, this calculation applies to the fitted longitudinal distribution for a marker at a particular subject and visit. It is a diagnostic of that distribution, not by itself a test of the event-time component or of the association between the marker and event process. The calculation can use the fitted marker distribution conditional on the subject’s random effects, or a marginal distribution that integrates over those effects; the choice must match the diagnostic question.
For a discrete outcome, the observed value corresponds to an interval of probability, not a single cumulative probability. The random draw selects a point within that interval. For a binary marker coded 0 and 1, for example, the interval is \((0,F_{ij}(0)]\) when the observed value is 0, and \((F_{ij}(0),1]\) when it is 1. The residual is the standard normal quantile of the selected point. In computation, endpoint conventions have probability zero under continuous randomization; implementations should nevertheless guard against numerical values of exactly 0 or 1, where the normal quantile is infinite.
Why the Transformation Works
The key result is that if the fitted conditional distribution is the true distribution, and the randomization is performed as defined, then \(U_{ij}\) is uniform on \((0,1)\). Consequently, \(r_{ij}\) has a standard normal distribution. This is a distributional benchmark, not a claim that fitted residuals in a finite sample will look exactly normal.
To see the result, suppose the possible outcomes are ordered and have probabilities \(p_k\). The cumulative probability intervals associated with these outcomes partition \((0,1)\), and the interval for outcome \(k\) has length \(p_k\). Conditional on observing that outcome, the randomized value is uniform within its interval. For any point inside that interval, its density contribution from outcome \(k\) is \(p_k(1/p_k)=1\); adding contributions across the intervals leaves density 1 throughout \((0,1)\). Thus \(U_{ij}\) is uniform. Applying the inverse standard normal cumulative distribution function gives \(r_{ij}\sim N(0,1)\).
This argument assumes that the conditional probabilities used to form the intervals are correct. For a conditional diagnostic, it also assumes that the chosen conditioning set is appropriate. In practice, random effects are estimated, model parameters are estimated, and measurements from one subject may remain dependent after conditioning. Therefore standard normality is an ideal reference, not an exact finite-sample guarantee. If the longitudinal model assumes conditional independence given random effects, that assumption matters when interpreting within-subject patterns in the residuals.
Worked Calculations for Discrete Markers
Worked Example: A Binary Marker
Suppose a hypothetical fitted model gives \(P(Y_{ij}=1\mid\text{conditioning set})=0.20\) at one visit, so \(P(Y_{ij}=0\mid\text{conditioning set})=0.80\). The observed outcome is 1, and the generated randomization value is \(V_{ij}=0.50\).
For the observed value 1, \(F_{ij}(1^-)=P(Y_{ij}=0)=0.80\) and \(F_{ij}(1)=1\). The randomized probability is
Therefore \(r_{ij}=\Phi^{-1}(0.90)\approx1.282\). This is a positive residual: the realized randomized point falls above the median of the fitted distribution. It does not say that the observed 1 is impossible or that the model is necessarily wrong; under the fitted model, a 1 has probability 0.20.
Worked Example: An Ordinal Marker
For a hypothetical ordinal marker with categories 0, 1, and 2, suppose the fitted category probabilities are 0.20, 0.50, and 0.30. The observed value is 1, and the randomization draw is \(V_{ij}=0.40\).
The lower and upper cumulative probabilities for category 1 are \(F_{ij}(1^-)=0.20\) and \(F_{ij}(1)=0.20+0.50=0.70\). Thus
The negative residual means that this randomized point is below the standard normal median. The calculation uses the probability assigned to the observed category; it does not treat category labels as equally spaced measurements. If the category probabilities were different at another visit, that visit would use its own fitted cumulative probabilities.
Worked Example: A Poisson Count Marker
Suppose a hypothetical count marker has a fitted conditional Poisson distribution with mean \(\lambda=1.5\), and the observed count is 2. For a Poisson variable, \(P(Y=k)=e^{-\lambda}\lambda^k/k!\). The cumulative probability just below 2 and the cumulative probability at 2 are
Using the randomization draw \(V_{ij}=0.60\), first find the interval width, \(0.8088468-0.5578254=0.2510214\). Then
The residual is moderately positive on the standard normal scale. The calculation depends on the fitted conditional mean, which could vary by subject and visit through covariates or random effects; it is not generally appropriate to use one common Poisson mean for every observation.
Reading Normal Quantile and Time Plots
A normal quantile plot compares ordered randomized quantile residuals with the corresponding theoretical quantiles of a standard normal distribution. If the marker distribution is reasonably calibrated, the points should lie roughly along a straight line. Systematic curvature, tail departures, or substantial skewness can indicate a mismatch in the fitted outcome distribution. For example, a count model that understates the frequency of large counts may produce an excess of large positive residuals.
A time plot places residuals against visit time, often with separate symbols or panels for subjects. Look for a changing center, changing spread, or a systematic pattern over time. A curve or trend can suggest that the mean trajectory or time structure is inadequate; a changing spread can signal variance or distributional misfit. Runs of residuals with similar signs within subjects may indicate remaining temporal dependence or an inadequate conditional structure. A plot of residuals against fitted probabilities or means can also reveal that misfit is concentrated in a particular range of predictions.
Because the residuals are randomized, rerunning the randomization can change individual points and sometimes the apparent shape of a plot, especially with small samples or very discrete outcomes. Use a reproducible random seed for reporting, and consider whether conclusions persist across a few independent randomizations. Formal envelopes or simulation-based reference plots can help account for sample size and parameter estimation, but they should reproduce the fitting and conditioning choices used for the actual diagnostic.
Examine plots at more than one level. A pooled normal quantile plot can look satisfactory while a subgroup or a particular time region is misfit. Conversely, a handful of unusual points need not imply broad distributional failure. In a joint model, observed marker records may also be affected by event-dependent follow-up or missingness. A plot of residuals among observed visits describes those observed records under the chosen conditioning scheme; it does not automatically diagnose the distribution of unobserved marker values after dropout.
Alternatives, Diagnostics, and Limits
Pearson residuals divide the observed-minus-fitted-mean difference by a model-based standard deviation. Deviance residuals summarize each observation’s contribution to a likelihood-based discrepancy. Both can be useful, but for discrete outcomes their possible values and distributions need not be standard normal. Randomized quantile residuals instead use the entire fitted conditional distribution, placing binary, ordinal, and count outcomes on a common reference scale. No one residual plot replaces checking whether the chosen outcome family, link, predictors, and random-effects structure are scientifically plausible.
| Diagnostic | What it provides | Main caution |
|---|---|---|
| Randomized quantile residual | A standard normal reference scale under a correct fitted discrete distribution | Values depend on the randomization and on the fitted CDF |
| Pearson residual | A standardized difference from the fitted mean | May be discrete, skewed, or non-normal |
| Deviance residual | A signed likelihood-based measure of discrepancy | Its reference distribution is not automatically standard normal observation by observation |
Other failure modes include using the wrong fitted probabilities for an observation, neglecting overdispersion or excess zeros in a count distribution, and treating an ordinal response as a count without justification. With conditional residuals, estimated random effects are substituted for unknown subject effects; this can make residuals look more concentrated than the ideal reference and can leave dependence from estimation. A marginal residual instead uses a CDF integrated over the random-effects distribution, and answers a different question. The conditioning choice should be stated and kept consistent when comparing plots.
Randomized quantile residuals are diagnostic summaries, not a standalone hypothesis test and not proof that the full joint model is correct. They may flag distributional misfit but generally do not identify its cause. Interpretation is strongest when normal quantile plots, time plots, and checks against fitted values are read together with the model assumptions. They also do not resolve whether an unusual observation is influential; that is a separate diagnostic question.
State whether the marker CDF is conditional on estimated subject effects or marginal over their distribution, and identify the covariates and visit time used.
For each observed marker, compute the fitted probabilities below and at the observed value.
Draw \(V_{ij}\) uniformly on \((0,1)\), interpolate within the probability interval, and apply \(\Phi^{-1}\).
Use normal quantile, time, and fitted-value plots; check whether conclusions are robust to randomization and consider dependence and observation patterns.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- A binary marker has fitted probability 0.30 of being 1. The observed value is 0 and \(V_{ij}=0.50\). Find \(U_{ij}\) and \(r_{ij}\).
- An ordinal outcome has probabilities 0.10, 0.60, and 0.30 for categories 0, 1, and 2. For an observed category 2 and \(V_{ij}=0.25\), find the probability interval and \(U_{ij}\).
- Why is the unrandomized value \(F_{ij}(Y_{ij})\) not generally uniform for a discrete marker?
- A normal quantile plot looks approximately straight, but a time plot shows increasingly positive residuals at later visits. Give one interpretation and one caution.
Solutions. (1) For an observed 0, the interval is from 0 to 0.70. Thus \(U_{ij}=0+0.50(0.70)=0.35\), and \(r_{ij}=\Phi^{-1}(0.35)\approx-0.3853\). (2) For category 2, the interval is \((0.70,1]\), since the cumulative probability through category 1 is 0.70. Therefore \(U_{ij}=0.70+0.25(1-0.70)=0.775\). (3) The CDF jumps at possible outcomes, so its values have point masses rather than a continuous uniform distribution; randomization within each outcome’s CDF interval fills the jumps. (4) Increasingly positive residuals may indicate that the fitted marker distribution underpredicts outcomes later in follow-up, for example because of a time-related mean or distributional mismatch. The pattern should be checked across subjects and against the randomization and fitted-value plots; estimated effects and the observed follow-up pattern also affect interpretation.