Sparse Individual Data and Weakly Identified Forecasts

When Sparse Data Leave a Forecast Weakly Identified

An individual forecast can be computed even when that person's concentration data provide little information about the parameters driving the forecast. The calculation may still be mathematically valid, but its apparent specificity can be misleading: the population distribution may be doing most of the work. The central question is therefore not just how many samples were collected, but whether their timing and precision make the forecast-relevant parameters distinguishable from plausible alternatives.

The introductory material on updating individual parameter distributions establishes the role of the population distribution as a prior and the concentration observations as individual-level evidence. Here the focus is diagnosing how much that evidence changes the forecast. We use a simple one-compartment model to make the calculations transparent; in a nonlinear mixed-effects model, the same concern applies to the individual random effects or parameters that determine the predicted concentration.

Definition: An individual forecast is weakly identified by the individual's data when substantially different plausible individual parameter values remain compatible with those data, so that the posterior parameter distribution and resulting forecast depend appreciably on the population prior, modeling assumptions, or both. A forecast is prior-dominated when the observed data contribute little information relative to that prior. These are diagnostic descriptions, not binary tests with universal cutoffs.

A Working Model and Its Assumptions

Suppose a hypothetical patient receives a known intravenous bolus dose. Let \(C_0\) be the known concentration immediately after dosing, and let \(k=CL/V\) be the individual elimination rate in inverse hours. Under a one-compartment model with elimination, the concentration at time \(t\) is \(C(t)=C_0e^{-kt}\). Assume log-scale measurement error is additive, independent, and normally distributed with known standard deviation \(\sigma\). Thus the observed log concentration \(Y(t)\) follows the model below.

Y(t)=\log C_0-kt+\epsilon,\qquad \epsilon\sim N(0,\sigma^2).

For a single observation at time \(t>0\), define \(z=(\log C_0-Y(t))/t\). Then \(z=k-\epsilon/t\), so \(z\mid k\sim N(k,\sigma^2/t^2)\). Use a normal population prior approximation \(k\sim N(m_0,v_0)\). This conjugate calculation is a benchmark, not a claim that elimination rates can be negative: in an application, the normal approximation is reasonable only when its relevant probability mass is on positive, plausible rates. Otherwise a positive parameterization and an appropriate nonlinear posterior calculation are needed.

With this model, a sample is informative about \(k\) when it measures the concentration decline with adequate precision. If \(C_0\) is uncertain, dose history is incomplete, or more than one parameter is unknown, those uncertainties must also be modeled; the simplified calculation below does not resolve them.

Why Sampling Time Matters

In the log-concentration model, the sensitivity of the expected observation to \(k\) is \(\partial E[Y(t)\mid k]/\partial k=-t\). A later sample therefore produces a larger expected change in log concentration for a given change in \(k\). With fixed log-error variance, its likelihood is more informative about \(k\). Combining the normal prior and the one-observation likelihood gives the posterior mean and variance:

v_{\mathrm{post}}=\left(\frac{1}{v_0}+\frac{t^2}{\sigma^2}\right)^{-1}, \qquad m_{\mathrm{post}}=v_{\mathrm{post}}\left(\frac{m_0}{v_0}+\frac{t^2z}{\sigma^2}\right).

The likelihood precision is \(t^2/\sigma^2\); the prior precision is \(1/v_0\). The posterior mean is their precision-weighted average of \(m_0\) and \(z\). The observation receives little weight when its precision is small relative to the prior precision. In that case the posterior mean will be close to \(m_0\) if the observation is not sufficiently far from \(m_0\) to offset its small weight; low precision alone does not guarantee a small shift.

The algebra also explains the sampling-time result. Taking the reciprocal of the posterior variance adds prior precision and data precision:

\frac{1}{v_{\mathrm{post}}} =\frac{1}{v_0}+\frac{t^2}{\sigma^2}.

This is a justification within the stated model, not a universal recommendation to sample as late as possible. In practice, concentrations may become difficult to quantify, other processes may matter, and several parameters may trade off against one another. Timing must be judged in relation to the forecast and the actual model.

Worked Example: The Same Elimination Signal at Two Sampling Times

Consider hypothetical data with \(C_0=10\) mg/L, prior \(k\sim N(0.12,0.04^2)\) per hour, and log-error standard deviation \(\sigma=0.10\). First suppose a sample at \(t=1\) hour is \(9.05\) mg/L. The transformed observation is \(z=-\log(9.05/10)/1=0.09982\) per hour. The likelihood variance for \(z\) is \(0.10^2/1^2=0.0100\).

The prior precision is \(1/0.0016=625\), and the observation precision is \(1/0.0100=100\). Therefore

m_{\mathrm{post}}=\frac{625(0.12)+100(0.09982)}{725}=0.1172\ \mathrm{h}^{-1}, \qquad v_{\mathrm{post}}=\frac{1}{725}=0.001379\ \mathrm{h}^{-2}.

Now suppose instead that a sample at \(t=8\) hours has the concentration \(10e^{-0.8}=4.493\) mg/L, giving \(z=0.1000\) per hour. Its transformed observation variance is \(0.10^2/8^2=0.00015625\), and its precision is \(6400\). The posterior is

m_{\mathrm{post}}=\frac{625(0.12)+6400(0.1000)}{7025}=0.1018\ \mathrm{h}^{-1}, \qquad v_{\mathrm{post}}=\frac{1}{7025}=0.0001423\ \mathrm{h}^{-2}.

The later sample has pulled the posterior more strongly toward the observed rate and reduced its variance. To compare forecasts at \(T=12\) hours, suppose the future log-concentration measurement has the same independent error standard deviation, \(0.10\). The predictive distribution is normal on the log scale, with mean \(\log(10)-12m_{\mathrm{post}}\) and variance \(12^2v_{\mathrm{post}}+0.10^2\). The corresponding concentration forecast is lognormal.

For the 1-hour sample, using the unrounded posterior mean and \(\log(10)\), the log-scale mean is approximately \(0.8960\), and the predictive variance is \(144(0.001379)+0.0100=0.2086\). The median forecast is \(e^{0.8960}=2.45\) mg/L; the central 95% prediction interval is approximately \(e^{0.8960\pm1.96\sqrt{0.2086}}=(1.00,6.00)\) mg/L. For the 8-hour sample, using the unrounded posterior mean and \(\log(10)\), the log-scale mean is approximately \(1.0812\), and the predictive variance is \(144(0.0001423)+0.0100=0.03050\). Its median is \(2.95\) mg/L and its central 95% prediction interval is approximately \((2.09,4.15)\) mg/L. In this simplified model, the later sample provides stronger information about \(k\), so uncertainty in the 12-hour forecast is smaller.

Prior Sensitivity as a Diagnostic

A useful diagnostic is to repeat the individual forecast under a small set of scientifically plausible population priors. The purpose is not to choose whichever prior gives a preferred answer. It is to find out whether the individual data meaningfully constrain the forecast despite reasonable changes in prior location or spread. If posterior forecasts shift markedly, the observed data have not overwhelmed those prior differences.

Worked Example: Changing the Population Prior

Suppose a hypothetical sample is collected at \(t=2\) hours with \(C_0=10\) mg/L and \(C(2)=10e^{-0.30}=7.408\) mg/L. Thus \(z=0.1500\) per hour. Let \(\sigma=0.20\), so the variance of \(z\) is \(0.20^2/2^2=0.0100\), with precision 100. Compare two plausible normal approximations: prior A, \(N(0.08,0.04^2)\), and prior B, \(N(0.20,0.08^2)\), with rates in inverse hours.

Prior A has precision 625. The posterior mean and variance are \([625(0.08)+100(0.15)]/725=0.08966\) per hour and \(1/725=0.001379\) \(\mathrm{h}^{-2}\). Prior B has precision \(1/0.0064=156.25\). Its posterior mean and variance are \([156.25(0.20)+100(0.15)]/256.25=0.1805\) per hour and \(1/256.25=0.003902\) \(\mathrm{h}^{-2}\). The same observation leaves quite different posterior rates under the two priors.

For a forecast at \(T=10\) hours, retain \(C_0=10\) mg/L and assume future log-error standard deviation \(0.20\). Under prior A, the log predictive mean is \(2.3026-10(0.08966)=1.4060\), and the variance is \(100(0.001379)+0.0400=0.1779\). The median is \(4.08\) mg/L and the central 95% prediction interval is approximately \((1.78,9.33)\) mg/L. Under prior B, using the unrounded posterior mean and \(\log(10)\), the log mean is approximately \(0.4977\), and the variance is \(100(0.003902)+0.0400=0.4302\). The median is \(1.64\) mg/L and the interval is approximately \((0.455,5.95)\) mg/L. The large difference between these forecasts is evidence that this one sample has not identified the forecast-relevant rate independently of the prior. It is not evidence that either prior is correct.

Reading Forecast Uncertainty and Recognizing Failure

A wide prediction interval may signal weak individual information, but width alone does not identify its cause. It can also reflect irreducible measurement or process variability, uncertainty in other parameters, or a long forecast horizon. The decomposition in the simple model makes this explicit: the predictive log variance is \(T^2v_{\mathrm{post}}+\sigma^2\). The first term is uncertainty in the individual rate carried into the forecast; the second is future measurement error. A narrow interval is not proof of strong identification if, for example, the model has omitted an important source of uncertainty.

In an NLME analysis with several individual parameters, inspect the posterior distributions of parameters that materially affect the forecast, their dependence, and the posterior predictive distribution itself. Sparse data may constrain one combination of parameters while leaving another weakly determined. For example, a single concentration can be compatible with different combinations of volume and clearance. A one-parameter calculation with known \(C_0\) cannot reveal that confounding.

Useful warning signs include:

Posterior parameter summaries barely change from the population prior, or change mainly because of prior assumptions.
Reasonable alternative priors produce meaningfully different forecasts despite the same individual observations.
Forecast intervals are broad relative to the clinical decision range, or widen substantially when a weakly informed parameter is included.
Observed concentrations are poorly predicted, residual patterns remain, or a sample is near the assay's quantification limit, undermining the assumed likelihood.
Different parameter combinations fit the sparse observations similarly but imply divergent future concentrations.

A Gaussian benchmark can also fail when parameter constraints matter, the posterior is skewed or multimodal, the likelihood is strongly nonlinear, or measurement errors are not adequately represented by the assumed model. In those situations, report the uncertainty using a suitable posterior calculation rather than treating the conjugate approximation as definitive.

How This Differs from Simpler Forecasts

Approach	What it uses	Limitation with sparse data
Population-only forecast	The population distribution, without individual updating	Does not reflect what the individual's observations say.
Plug-in individual forecast	A single fitted individual parameter value	Can conceal parameter uncertainty and yield overly narrow prediction intervals.
Posterior predictive forecast	The individual posterior distribution and future variability	Properly propagates uncertainty, but can remain prior-sensitive when individual data are weak.

The posterior predictive approach is not a remedy for missing information: it represents the uncertainty that remains. Compared with a plug-in forecast, it makes parameter uncertainty visible in the interval. Compared with a population-only forecast, it incorporates individual observations, while prior sensitivity indicates how much those observations actually changed the result. Neither a posterior mean nor a prediction interval should be interpreted as a guarantee about a particular future observation.

Key takeaway: Diagnose weak identification by considering the observation model, sampling time, posterior change from the population prior, sensitivity to plausible priors, and the forecast's uncertainty. A forecast can be computed even when these checks show that individual data contribute little.

Check Your Understanding

Use the stated model and assumptions. Solutions follow each question.

With fixed log-error standard deviation, why does a later observation carry more information about \(k\) in the one-compartment log-concentration model?
If the likelihood is much less precise than the prior, what can and cannot be concluded about the posterior mean?
In the predictive log variance \(T^2v_{\mathrm{post}}+\sigma^2\), identify the two sources of uncertainty.
Why might one concentration fail to identify clearance and volume separately?

Solutions. (1) The log-concentration sensitivity to \(k\) is \(-t\), so the likelihood precision for \(k\) from one sample is \(t^2/\sigma^2\), which increases with \(t\) under the stated assumptions. (2) The observation receives little weight in the precision-weighted mean. The posterior mean is close to the prior mean if the observation is not sufficiently far from it to compensate for that small weight; low precision alone does not determine the shift. (3) \(T^2v_{\mathrm{post}}\) is uncertainty in the individual rate propagated to the forecast time, and \(\sigma^2\) is future log-measurement error. (4) A concentration can depend on multiple parameters through a shared predicted value, allowing different parameter combinations to fit the same observation. Additional information or assumptions would be required to distinguish those combinations.