Two Different Questions Hidden in “Uncertainty”
A dose-response model can leave us uncertain in two fundamentally different ways. Patients given the same dose may have different outcomes: this is patient-to-patient variability. Separately, the data may not determine the model’s population dose-response curve precisely: this is epistemic uncertainty, or uncertainty due to limited knowledge. The first describes differences among patients; the second describes how well the model quantities are known.
The distinction matters in development decisions. Patient variability informs questions such as how broadly benefit or risk may be distributed at a dose. Epistemic uncertainty informs how confidently a population-level prediction can support a decision. More precise estimation can reduce epistemic uncertainty, but it does not make patients more alike.
Describe the Mean and the Patient Distribution Separately
Let \(d\) denote dose, and let \(Y_i(d)\) denote the outcome for patient \(i\) at that dose. A useful conceptual decomposition is
Here, \(\mu(d)\) is the population mean response at dose \(d\), and \(\eta_i(d)\) is patient \(i\)'s deviation from that mean. Across patients, the deviations have mean zero and a patient-level standard deviation \(\sigma_{\text{patient}}(d)\). In practice, patient-level spread may include both biological differences and outcome noise, depending on how the model defines the outcome and its error terms.
The mean curve is not known exactly when it is estimated from data. For a particular dose, write \(\hat{\mu}(d)\) for its estimate and \(SE_{\hat{\mu}}(d)\) for the standard error representing estimation uncertainty. A confidence interval for \(\mu(d)\) describes uncertainty in the population mean estimate under a repeated-sampling procedure. It is not a range expected to contain most individual patients.
By contrast, an individual prediction interval describes a range for a future patient's outcome. It must account for patient-level spread, as well as uncertainty in the estimated mean when that uncertainty is included. Therefore, a population mean can be estimated precisely while individual outcomes remain widely dispersed.
Worked Example: Classifying Two Sources of Uncertainty
Suppose a hypothetical dose-response analysis estimates the mean change in a clinical score at 20 mg. The fitted mean is 18 score points, but patients receiving that dose do not all have the same change. Classify each statement.
“The mean response may be between 14 and 22 points.” If this is a confidence interval calculated for the population mean, it represents epistemic uncertainty about \(\mu(20)\). It does not say that most individual patient responses fall in this range.
“Individual responses at 20 mg are spread around the mean, with a standard deviation of about 6 points.” This describes patient-to-patient variability, conditional on the dose and model. It is not a measure of how precisely the mean has been estimated.
“A future patient's outcome is uncertain.” That prediction can reflect both: the patient may differ from the population mean, and the estimated mean itself may be uncertain. Its interval must be defined according to which sources are included.
Worked Dose-Response Example: Mean Confidence Interval Versus Individual Prediction
Consider a hypothetical study using a simple mean dose-response model over a limited dose range:
Dose \(d\) is measured in mg and response in score points. At 20 mg, the estimated population mean is \(2+0.8(20)=18\) points. Suppose the standard error of that estimated mean is 2 points, and the estimated patient-level standard deviation at this dose is 6 points. For this example, assume the mean estimate is approximately normal, patient deviations are approximately normal, and the patient deviation is independent of estimation error. These are simplifying assumptions for the calculations, not facts guaranteed by the model.
Confidence interval for the population mean. Using the normal approximation, a 95% confidence interval is the estimate plus or minus 1.96 standard errors:
Thus, the interval is about 14.1 to 21.9 points. Under the repeated-sampling assumptions of the procedure, 95% of intervals constructed this way would cover the fixed population mean. It is not correct to say that there is a 95% frequentist probability that this particular interval contains the mean, or that 95% of patients respond within it.
Approximate prediction interval for one future patient. Under the stated independence and normality assumptions, uncertainty in the estimated mean and patient-level variability contribute separate variances. The approximate predictive standard deviation is
The corresponding 95% prediction interval is
This much wider interval is for an individual future outcome under the assumptions above. It is not a confidence interval for the mean. If the target were the distribution of patients conditional on treating the estimated mean as fixed, the calculation would use patient variability alone; including mean-estimation uncertainty produces the predictive interval shown here.
Why the Prediction Variance Adds
The calculation above follows from a short variance decomposition. Represent estimation error in the mean by \(\delta(d)\), and the future patient's deviation by \(\eta(d)\). In an uncertainty representation for prediction, write the future outcome as
Assume both deviations have mean zero and are independent, with variances \(SE_{\hat{\mu}}^2(d)\) and \(\sigma_{\text{patient}}^2(d)\). The variance of a sum equals the sum of the variances plus twice the covariance. Independence makes the covariance zero, giving
This is a prediction calculation, not a claim that a fixed frequentist parameter literally changes from sample to sample. In repeated sampling, the estimated mean varies around the fixed mean; in a Bayesian representation, uncertainty about the mean can instead be represented by a distribution. The approximation also depends on the assumptions stated above. For example, dependence between the mean estimate and a patient-level quantity would require a covariance term rather than simple addition.
Worked Example: Estimating a Responder Rate at a Dose
In a hypothetical group of 60 patients treated at a selected dose, 42 meet a prespecified response criterion. The observed responder proportion is
The 70% is the observed fraction of patients who responded in this sample. It estimates the underlying responder rate, but is not itself a statement that the rate is known without uncertainty. To illustrate uncertainty in that rate, use the Wilson 95% confidence interval for a binomial proportion. With \(z=1.96\), its denominator, center, and half-width are
The interval is \(C\mathbin{\pm}H\), or approximately \([0.575,\ 0.801]\). Under the binomial sampling assumptions and the interval procedure, this describes uncertainty in the underlying responder proportion, not the range of responses across patients. Patient outcomes are represented by the responder and nonresponder categories; the interval quantifies how imprecisely the population rate is estimated from this sample.
This interpretation assumes the 60 outcomes can reasonably be treated as independent observations with a common response probability at the dose. Dependence, inconsistent response definitions, or meaningful differences in treatment conditions could make that simple binomial summary inadequate.
Displays, Diagnostics, and Decision Use
A clear display labels which quantity varies and which quantity is uncertain. For a dose-response analysis, a ribbon around the estimated mean curve can show a confidence interval for the mean curve. A separate prediction ribbon, distribution plot, or interval for individual outcomes can show patient-level spread together with whatever parameter uncertainty it includes. A legend should name the target and the interval type; “95% interval” alone is not enough.
| Display or summary | What it describes | Interpretation |
|---|---|---|
| Confidence interval around the mean curve | Epistemic uncertainty in the estimated population mean response | Repeated-sampling coverage for the mean under the stated procedure |
| Patient outcome distribution at a dose | Patient-to-patient variability conditional on the model | How outcomes may differ across patients at that dose |
| Prediction interval for a future patient | Individual variability and, if included, mean-estimation uncertainty | A range for a future outcome under the predictive assumptions |
| Confidence interval for a responder proportion | Epistemic uncertainty in the underlying population proportion | Sampling uncertainty about the rate, not an individual outcome range |
Check whether the interval width and the plotted quantity make sense. A very narrow confidence band for the mean can coexist with broad patient-level spread. If a purported individual prediction interval is no wider than the confidence interval for the mean despite substantial patient variability, check whether patient-level variation was omitted. If increasing the study size is said to reduce biological differences among patients, the two sources have been confused: larger samples can improve estimation precision, but do not by themselves shrink the underlying patient distribution.
The distinction also supports different decisions. A treatment may have a well-estimated mean response but still show substantial variation, which matters when judging how consistently patients may benefit. Conversely, a model may imply limited patient variation but have a poorly estimated mean, so the population-level prediction remains uncertain. Neither summary alone answers every benefit-risk question.
Common Pitfall and Limits
Wrong reasoning: “The 95% confidence interval for the mean is 14.1–21.9 points, so about 95% of patients will respond between 14.1 and 21.9 points.” The interval was calculated using the standard error of the estimated mean; it describes uncertainty in that mean, not patient-to-patient spread.
Correction: State the target before interpreting an interval. Use a confidence interval for uncertainty in a population quantity, and a prediction interval or outcome distribution for individual responses. If a future-patient prediction includes both sources, say so and state the assumptions used to combine them.
These summaries are only as credible as their model and data. Normal approximations can be poor for bounded, skewed, or highly discrete outcomes. Estimated variability may be uncertain too, and the simple variance addition above does not cover every dependence structure or model limitation. A display can communicate the uncertainty represented by a model; it cannot ensure that omitted sources of uncertainty are absent.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A dose-response mean has a narrow confidence interval, but patient responses at that dose are widely spread. Which source is small, and which is large?
- In the worked dose example, what would the approximate 95% interval for individual outcomes be if estimation uncertainty were ignored and the mean were treated as fixed?
- Does the 95% confidence interval for the mean response describe the range containing 95% of future patients? Explain.
- What does the observed proportion \(42/60\) describe, and what does its confidence interval describe?
Solutions. (1) Epistemic uncertainty in the mean estimate is small; patient-to-patient variability is large. (2) Using the patient standard deviation alone gives \(18\mathbin{\pm}1.96(6)=18\mathbin{\pm}11.76\), or approximately 6.24 to 29.76 score points, under a normal patient-outcome assumption. This conditional interval omits estimation uncertainty. (3) No. It describes uncertainty in the population mean under a repeated-sampling procedure; individual outcomes require a patient distribution or prediction interval. (4) The value \(42/60=0.700\) is the observed sample responder fraction. Its Wilson interval, approximately 0.575 to 0.801, describes sampling uncertainty about the underlying responder proportion, under the binomial assumptions.