PD Diagnostics With Sparse and Irregular Sampling

Why Sampling Structure Changes a Diagnostic

A PD model may describe a response that rises, reaches a maximum, and declines, while the data contain only a few observations at widely separated times. Another study may collect measurements at irregular times, with some participants contributing many observations and others only one or two. In either case, a residual plot or predictive check does not contain the same amount of information everywhere along the trajectory. A pattern based on many densely sampled observations can look more convincing than a pattern supported by only a few observations, even when the latter occurs in a scientifically important phase.

The goal is not to treat every sparse-data diagnostic as unreliable. It is to state what the diagnostic summarizes, which observations contribute to it, and how much independent information supports the interpretation. Residual-versus-time plots and predictive checks remain useful, but their display and summary should reflect the actual observation schedule.

Here, “conditional” indicates that the prediction is calculated at the conditioning level chosen for the diagnostic, such as using the subject’s estimated random effects for an individual fit. A population-level prediction instead averages over the relevant between-subject variation. Those two predictions answer different questions and should not be mixed within one residual plot without explanation. For non-Gaussian responses, use residuals appropriate to the observation distribution, such as simulation-based residuals, rather than assuming that a raw difference has constant variance or a standard normal distribution.

A time phase is a prespecified interval or scientifically meaningful part of the response trajectory, such as early response, peak, or late response. The boundaries should be chosen using the design or scientific question, not selected after inspecting residuals to make a pattern appear stronger. With sparse data, a phase may contain very few observations; that limitation should be visible in the summary.

What Uneven Information Does to Summaries

A pooled residual mean is an observation-weighted average. If phase \(k\) contains \(n_k\) observations and has residual mean \(\bar e_k\), the overall mean is

Thus, phases with more recorded observations contribute more to the pooled value. If observations are clustered within subjects, a subject with many measurements can also dominate. This may be appropriate when the target is the average residual among the observations actually collected. It is not automatically appropriate when the question concerns average performance across subjects or across the response trajectory.

A subject-balanced summary first calculates a summary within each subject and then averages those subject summaries, giving each subject equal weight. A phase-balanced summary can likewise give each prespecified phase equal weight. These answer different questions from the observation-weighted mean. No weighting scheme is universally correct: the diagnostic should match the target of interpretation, and the number of subjects and observations supporting each component should be reported.

Reading Residual Plots When Times Are Irregular

A residual-versus-time plot should show the actual observation times, rather than placing observations into equally spaced positions that imply a regular schedule. When several observations come from one subject, distinguish subjects visually or summarize within subject as well as across observations. Otherwise, a dense cluster from one person can look like broad evidence for a population-level time pattern.

Useful displays for sparse or irregular designs include plotting all residuals against actual time, showing the number of observations or subjects in each time interval, and displaying phase-specific summaries with their underlying counts. Standardized residuals can help when residual variability changes with time or predicted response, but standardization does not create information where few observations exist. A standardized residual retains its sign because its divisor, \(\sigma_{ij}\), is positive.

Time bins can make a display easier to read, but the binning decision matters. Wide bins may combine different response phases and hide a localized mismatch; narrow bins may contain too few observations to summarize meaningfully. Where observations fall at distinct planned times, displaying those times directly may be more informative than imposing bins. If bins are used, state their boundaries and show how many observations and subjects contribute to each.

Predictive Checks Must Respect the Observation Schedule

A predictive check compares observed responses with responses generated under the fitted model. To interpret it when sampling is sparse or irregular, generate replicated data at the actual observation times and for the same subjects or design structure as the observed data. A check on a dense, regular grid may be useful for visualizing a model-implied trajectory, but it does not show how well the model predicts the sparse observations that were actually collected.

For continuous PD, visual predictive checks can compare observed summaries with corresponding summaries from simulated data. With uneven sampling, compute those summaries within meaningful time regions and display the number of observed data points contributing to each region. If the scientific target gives equal importance to subjects, summarize or resample at the subject level rather than allowing densely observed subjects to determine the entire check. Population-level and individual-conditional predictions should also be kept distinct: the former assesses population predictions, while the latter assesses predictions conditional on individual information.

Sparse checks are especially uncertain in regions with few independent subjects. One can make that limitation concrete for a simple percentile check. If a continuous predictive distribution is calibrated, an observation has probability \(0.10\) of falling below its 10th percentile. For \(m\) independent subjects evaluated once each at a specified phase, the number \(K\) below that percentile follows a binomial distribution with \(m\) trials and probability \(0.10\). This calculation is not appropriate if the observations are dependent in a way ignored by the model; repeated measurements from the same subject should not be counted as independent trials.

A Practical Diagnostic Sequence

Common Failure Modes and Alternatives

A frequent mistake is to treat each plotted point as an independent piece of evidence. When one subject supplies many measurements, the plotted point count can greatly exceed the number of independent subjects. The correction is to show subject identity and, where relevant, summarize at the subject level or use predictive simulations that preserve within-subject dependence.

Another mistake is to interpret a smooth curve through sparse residuals as a well-supported time trend. A smoother can interpolate between observed times, but it cannot establish what happened in a long unobserved interval. Show raw observations and support counts; interpret trends between sparse times cautiously. This is particularly important when competing response shapes would differ mainly between observation times.

A third mistake is to compare observations with predictive envelopes calculated under a different schedule or a different level of conditioning. The correction is to simulate the design actually observed and to label whether predictions are population-level or conditional on subject information. A further error is to read a negative residual as underprediction: with \(e_{ij}=y_{ij}-\mu_{ij}\), a negative value means the predicted mean is above the observed response.

Alternative summaries have complementary strengths. Raw residual plots preserve timing and individual patterns but can be visually crowded. Time-bin summaries are compact but depend on bin boundaries and may conceal variation within a bin. Subject-balanced summaries reduce dominance by dense schedules but may be noisy when subjects contribute few observations. Schedule-matched predictive checks incorporate the model and design, but they can still have low diagnostic power when few subjects are observed in a critical phase. Combining these views is generally more informative than choosing one as universally best.