Visualizing Predictive Uncertainty With Density and CDF Plots

Why Show Predictive Uncertainty?

A decision-relevant prediction is more informative when it shows more than a central estimate. For example, the predicted treatment benefit might be above a decision threshold in most plausible model realizations, or it might be close to the threshold with substantial probability on either side. A plot of the predictive distribution makes the range, shape and concentration of possible outcomes visible; a cumulative distribution function (CDF) plot makes probabilities below or above specified values especially easy to read.

These plots display uncertainty in an outcome under a stated model and set of assumptions. They do not, by themselves, establish that the model is adequate or that a predicted outcome will occur. The interpretation depends on what was predicted, which uncertainties were included, and the population or scenario to which the prediction applies. The earlier material on prediction intervals and on confidence and credibility intervals in decisions distinguishes predictive outcomes from uncertainty about model parameters; a density or CDF plot should preserve that distinction.

Definition: Let \(Y\) be a decision-relevant predictive outcome, with predictive density \(f_Y(y)\) when \(Y\) is continuous. Its cumulative distribution function is \(F_Y(y)=P(Y\leq y)\). A density describes probability concentration per unit of outcome, while a CDF gives the probability accumulated up to a value.

A density plot places outcome values on the horizontal axis and density on the vertical axis. Density has reciprocal units: if \(Y\) is measured in milligrams per litre, \(f_Y(y)\) is measured per milligram per litre. The height at a point is not the probability of that exact value. For a continuous outcome, the probability of one exact value is zero; probabilities are represented by areas over intervals.

A CDF plot also has outcome values on the horizontal axis, but its vertical axis ranges from zero to one and reports cumulative probability. It is nondecreasing, approaches zero at the lower extreme of the outcome range and approaches one at the upper extreme. For a continuous outcome, the probability between two values is the difference between the CDF at those values.

From Density to Probability

The relationship between the two displays follows directly from the definition of a density. For any \(a<b\), the probability that the outcome falls between \(a\) and \(b\) is the area under the density curve over that interval. The CDF accumulates that same area from the lower end of the outcome range.

F_Y(y)=\int_{-\infty}^{y} f_Y(t)\,dt,\qquad P(a<Y\leq b)=F_Y(b)-F_Y(a)=\int_a^b f_Y(t)\,dt

For a continuous outcome, including or excluding either endpoint does not change the probability. The tail probability above a threshold \(c\) is \(1-F_Y(c)\). Thus, a density plot answers “where is probability concentrated?” and a CDF plot answers “how much probability has accumulated by this value?” Neither display changes the underlying prediction; each emphasizes a different reading task.

Worked Example: Reading a Predictive Density

Hypothetical model prediction. Suppose the predicted treatment benefit \(Y\), measured in points on a clinical outcome scale, is \(Y\sim N(24,6.25)\). This normal predictive distribution has mean 24 points and standard deviation \(\sqrt{6.25}=2.5\) points. The normal model is assumed to be appropriate for this illustrative prediction.

The density at the mean is

f_Y(24)=\frac{1}{2.5\sqrt{2\pi}}\exp\left(-\frac{(24-24)^2}{2(2.5)^2}\right) =\frac{1}{2.5\sqrt{2\pi}}\approx 0.1596\ \text{per point}.

This is the curve’s height at 24 points, not a 15.96% probability of exactly 24 points. To find the probability that benefit is between 20 and 28 points, standardize the endpoints. They are 1.6 standard deviations below and above the mean, respectively:

P(20\leq Y\leq 28) =\Phi(1.6)-\Phi(-1.6) =0.9452-0.0548 =0.8904.

The density plot shows a symmetric concentration around 24 points; the area from 20 to 28 points is about 89.04%. On the CDF plot, the same result is the vertical difference between \(F_Y(28)=0.9452\) and \(F_Y(20)=0.0548\). The probability is conditional on the stated predictive model and assumptions, not a guarantee about a future observed benefit.

Reading a CDF at a Decision Threshold

A threshold \(c\) divides the outcome scale into two regions. If larger values are desirable, the predictive probability of exceeding the threshold is \(1-F_Y(c)\); if smaller values are desirable, the probability of being at or below it is \(F_Y(c)\). Whether the relevant side is “above” or “below” must come from the decision definition, not from the shape of the plot. The earlier material on threshold plots and probabilistic sensitivity analysis provides the decision context; the CDF gives a direct visual readout for a chosen predictive outcome and scenario.

For an empirical CDF based on \(M\) predictive draws \(y^{(1)},\ldots,y^{(M)}\), the estimate at \(c\) is the fraction of draws no greater than \(c\). This produces a step function. The complementary fraction estimates the probability above \(c\). The draws need to represent the predictive distribution being communicated; in particular, the simulated values should include the uncertainties intended for that prediction and retain relevant dependence among uncertain inputs.

Worked Example: Estimating a Threshold Probability From Predictive Draws

Hypothetical prediction. Ten predictive simulations give treatment benefits, in points, of 18, 22, 24, 27, 29, 31, 33, 35, 38 and 42. Assume these draws represent the predictive distribution of interest. A decision rule treats benefit greater than 30 points as crossing the threshold.

Five of the ten draws are at or below 30 points, so the empirical CDF at the threshold is

\widehat{F}_Y(30)=\frac{\#\{y^{(m)}\leq 30\}}{M} =\frac{5}{10}=0.50.

The draws above 30 are 31, 33, 35, 38 and 42 points. Therefore, the estimated probability of exceeding the threshold is

\widehat{P}(Y>30)=1-\widehat{F}_Y(30)=1-0.50=0.50.

On the empirical CDF, the step at 30 points reaches 0.50; on a plot of the complementary probability, the value above the threshold is also 0.50. This is a simulation-based estimate from only ten draws, so it is coarse and subject to Monte Carlo variability. A larger, adequately sampled set of predictive draws would ordinarily be needed for a stable estimate. The result describes the model’s predictive distribution, not the probability that the decision rule itself is correct.

Building and Checking a Density Histogram

When the predictive distribution is represented by simulations rather than a closed-form density, a density histogram can provide a useful approximation. For equal-width bins of width \(w\), the height for a bin is its count divided by the total number of draws and by \(w\). This scaling makes the total area of the bars equal to one. The approximate probability in a bin is its bar height multiplied by its width, or equivalently its count divided by the total number of draws.

The empirical CDF can be plotted from the same simulations without binning. At each outcome value, it is the proportion of simulated outcomes at or below that value. The histogram and empirical CDF therefore summarize the same simulated distribution, but binning affects the histogram’s appearance while it does not affect the empirical CDF’s steps.

Worked Example: Connecting a Density Histogram to a CDF

Hypothetical simulation output. A model produces 1,000 predictive draws of a positive exposure outcome, measured in exposure units. The counts in six equal-width bins are shown below. Assume the bins partition the displayed range and that no draws fall outside it.

Outcome interval	Draw count	Density height per unit	Cumulative proportion at upper edge
0 to 10	100	0.010	0.10
10 to 20	250	0.025	0.35
20 to 30	300	0.030	0.65
30 to 40	200	0.020	0.85
40 to 50	100	0.010	0.95
50 to 60	50	0.005	1.00

For the 20-to-30 bin, the density height is \(300/(1{,}000\times 10)=0.030\) per exposure unit. Its area is \(0.030\times10=0.30\), matching \(300/1{,}000\) of the draws. The CDF at 30 units is the cumulative fraction through the first three bins: \((100+250+300)/1{,}000=0.65\). Thus, the estimated probability above 30 units is \(1-0.65=0.35\). The table’s increasing cumulative proportions would form a nondecreasing CDF; the histogram’s bar areas, rather than bar heights alone, represent bin probabilities.

Diagnostics, Alternatives, and Limitations

A useful display is not simply attractive; it must accurately represent the predictive distribution and make its decision-relevant features legible. Check the following before interpreting a plot:

Check the axes and units. State the outcome and its units, identify whether the vertical axis is density or cumulative probability, and mark any threshold on the outcome axis.
Check normalization and coverage. A density should have total area one (up to numerical approximation). A CDF should not decrease and should approach one after the full simulated range. If plotted values omit tails, make that truncation apparent.
Check binning or smoothing. Histogram bin width can hide a narrow peak or create apparent irregularity. A smoothed density can obscure modes or suggest detail unsupported by the number of draws. Compare reasonable choices rather than selecting one because it gives a preferred appearance.
Check the tails and shape. A symmetric summary may conceal skewness or multiple modes. Include enough horizontal range to show important tail probability, and inspect whether a display makes materially different regions look deceptively similar.
Check the simulation and conditioning. Monte Carlo noise can make a tail probability unstable, especially with few draws. Confirm what population, treatment, assumptions and uncertainty sources the draws condition on. A plot cannot repair unrepresented uncertainty or dependence.

The density and CDF complement other summaries rather than replacing them. A point estimate is compact but hides distributional spread and shape. An interval communicates a selected range but may conceal asymmetry or multiple modes; the density adds shape, while the CDF can show exactly how probability accumulates around a threshold. A histogram is transparent for simulated data but sensitive to bins. A smooth density curve is visually continuous but depends on a smoothing choice. An empirical CDF avoids bin-width choices and supports direct threshold reads, though its steps may be visually coarse for small samples.

Neither plot identifies why uncertainty exists or which assumption drives it. They also do not establish calibration, model validity, causal interpretation, or the consequences of a decision. For discrete outcomes, the CDF has jumps at possible values and probability can occur at an exact threshold; therefore, distinguish \(P(Y\leq c)\) from \(P(Y<c)\) when the decision rule requires it. For continuous outcomes the distinction has probability zero, but it can matter for rounded or count outcomes.

Common Pitfall

Common mistake: “The density curve is tallest at 24 points, so there is a 16% probability that the benefit equals 24 points.” This confuses density height with probability. In the normal example, the height is about 0.1596 per point, while the probability of the exact value 24 is zero for a continuous outcome. Probability is an area over an interval, such as the approximately 89.04% area from 20 to 28 points.

A related mistake is reading a CDF value as the probability of being above the plotted value. By definition, \(F_Y(c)\) is the probability at or below \(c\); the probability above is \(1-F_Y(c)\). Confirm the direction of the outcome and decision rule before describing a threshold crossing.

Key takeaway: Use a density plot to show where predictive probability is concentrated and a CDF plot to show how much probability lies at or below each outcome. Read interval probabilities as density areas or CDF differences, and read upper-tail probabilities as one minus the CDF at the threshold.

Check Your Understanding

Answer each question using the stated definitions and show the probability calculation where requested.

A continuous outcome has \(F_Y(12)=0.30\). What is the probability that it exceeds 12?
A density is 0.04 per unit at an outcome value. Does this mean the probability of that exact value is 0.04? Explain.
In 200 predictive draws, 46 outcomes are greater than a decision threshold \(c\). Estimate the probability of exceeding \(c\).
A density histogram has a bar of height 0.02 per unit and width 5 units. What probability does that bar represent approximately?
For a discrete outcome, why might \(P(Y<c)\) differ from \(P(Y\leq c)\)?

Solutions. (1) \(P(Y>12)=1-F_Y(12)=1-0.30=0.70\). (2) No. Density is probability per unit; for a continuous outcome the probability of one exact value is zero, and interval probabilities are areas. (3) \(46/200=0.23\), so the estimated exceedance probability is 0.23. (4) The approximate probability is area \(0.02\times5=0.10\). (5) A discrete outcome can have positive probability at \(c\), so \(P(Y\leq c)=P(Y<c)+P(Y=c)\).