Uncertainty In Absolute-Risk Predictions

Why a Predicted Risk Is Uncertain

An absolute-risk prediction is calculated from estimated model quantities, not known population values. Even when the covariate profile and prediction horizon are specified exactly, the fitted regression coefficients and baseline hazard are estimated from finite data. If the goal is an average risk for a target population, that population may also be represented by a sample whose covariate composition varies from sample to sample.

These sources are conceptually distinct, but they can affect the same prediction at once. A coefficient changes how covariates modify risk; the baseline hazard sets the underlying risk level; and the target sample determines which covariate profiles are averaged. The resulting predicted risk is generally nonlinear in these quantities, so a small change in one input need not produce a proportional change in risk.

Definition: For a specified cause \(k\), horizon \(t\), and covariate profile \(x\), an absolute-risk prediction is an estimate \(\hat F_k(t\mid x)\) of the probability of experiencing cause \(k\) by time \(t\), before any competing event. Prediction uncertainty is variation in that estimate across repeated samples or plausible model inputs. It is not the same as the random variation in one individual's eventual event outcome.

The distinction between an individual's outcome and the probability assigned to that individual matters. A person either experiences the event of interest by \(t\), experiences a competing event first, or remains event-free through \(t\). A predicted cumulative incidence is a probability for the profile, not a guarantee about that person's outcome.

Three Sources of Uncertainty

For a fitted Fine–Gray model with one covariate profile \(x\), one common representation is

\hat F_k(t\mid x) = 1-\exp\left\{-\hat\Lambda_{0k}^{*}(t)\exp(x^\mathsf{T}\hat\beta_k)\right\},

where \(\hat\beta_k\) is the estimated regression-coefficient vector and \(\hat\Lambda_{0k}^{*}(t)\) is the estimated baseline cumulative subdistribution hazard. The formula makes the sources visible: uncertainty in \(\hat\beta_k\), uncertainty in \(\hat\Lambda_{0k}^{*}(t)\), and their joint effect on the predicted risk. Estimates of the coefficients and baseline are obtained from the same fitted model, so their estimation errors need not be independent.

Regression-coefficient uncertainty. The estimated coefficients describe covariate associations on the model's hazard scale. Sampling variability in them changes the relative risk contributions of covariate profiles. This effect depends on the profile: a coefficient may have little impact when its covariate is near the reference value and a larger impact at more distant values.
Baseline-hazard uncertainty. The estimated baseline determines the risk scale at the prediction horizon. Its precision depends on the information available over follow-up and on the model and estimation choices. Even with coefficients held fixed, a different plausible baseline estimate can change the predicted cumulative incidence.
Target-population sampling uncertainty. A standardized risk averages profile-specific risks over a target population. If that population is represented by a sample, its observed mix of covariates can differ from the mix in the underlying population. The average prediction can therefore vary even if the fitted model is held fixed.

These are estimation uncertainties under a chosen model and target definition. They do not automatically account for a wrong functional form, omitted predictors, incorrect event or censoring information, or a target population that differs systematically from the population represented by the data. Such issues are possible sources of prediction error, but are not captured merely by measuring sampling variation in fitted parameters.

Worked Example: Coefficient Uncertainty Changes Profile Risk

Suppose a hypothetical Fine–Gray model has one binary covariate \(x\), an estimated coefficient \(\hat\beta=\log(2)\), and estimated baseline cumulative subdistribution hazard \(\hat\Lambda_0^*(t)=0.10\) at the chosen horizon. Calculate the predicted risk for \(x=1\). Then examine the effect of perturbing the coefficient while holding the baseline fixed.

For \(x=1\), \(\exp(x\hat\beta)=\exp(\log 2)=2\). Thus

\hat F(t\mid 1) =1-\exp\{-0.10(2)\} =1-\exp(-0.20) \approx 0.1813.

Now hold the baseline estimate at 0.10 and change the coefficient so that the relative-risk multiplier is 1.8 or 2.2. The corresponding risks are

1-\exp\{-0.10(1.8)\} =1-\exp(-0.18) \approx 0.1647, \qquad 1-\exp\{-0.10(2.2)\} =1-\exp(-0.22) \approx 0.1975.

These are sensitivity calculations, not a confidence interval: the chosen perturbations have not been calibrated to a stated sampling probability. They demonstrate that coefficient uncertainty changes the profile prediction through the nonlinear risk formula. The calculation also assumes the fitted Fine–Gray model is an appropriate description for this prediction; it does not test that assumption.

Baseline Estimates and the Nonlinear Risk Scale

A baseline-hazard estimate may change the predicted probability even when the relative effect of covariates is fixed. For a cause-specific hazard model with constant cause-specific hazards \(h_1\) and \(h_2\) over a one-unit interval, the cumulative incidence for cause 1 is

F_1(1) = \frac{h_1}{h_1+h_2} \left\{1-\exp[-(h_1+h_2)]\right\}.

This expression assumes the two hazards are constant over the interval, that the causes are mutually exclusive first events, and that the model's hazards apply to the profile being predicted. It shows why the cause 1 risk is not just \(1-\exp(-h_1)\): the competing cause affects the probability of remaining at risk for cause 1.

Worked Example: Perturbing the Baseline Cause-Specific Hazard

Consider hypothetical constant cause-specific hazards over one time unit. At the reference profile, suppose \(h_1=0.20\) and \(h_2=0.10\), giving

F_1(1) = \frac{0.20}{0.20+0.10} \{1-\exp[-(0.20+0.10)]\} = \frac{0.20}{0.30}(1-\exp[-0.30]) \approx 0.1728.

Suppose the cause 1 hazard is the baseline hazard multiplied by a fixed relative effect of 2. A 10% increase or decrease in that baseline changes \(h_1\) to 0.22 or 0.18, while \(h_2\) remains 0.10. Under the same constant-hazard assumptions, the increased-hazard prediction is

\frac{0.22}{0.22+0.10} \{1-\exp[-(0.22+0.10)]\} = \frac{0.22}{0.32}(1-\exp[-0.32]) \approx 0.1883.

For the decreased hazard, the calculation is

\frac{0.18}{0.18+0.10} \{1-\exp[-(0.18+0.10)]\} = \frac{0.18}{0.28}(1-\exp[-0.28]) \approx 0.1570.

The risk changes from about 0.1728 to 0.1883 or 0.1570 per one-time-unit horizon. These values are not obtained by multiplying the original risk by 1.10 or 0.90; the cumulative-incidence formula is nonlinear and incorporates the competing hazard. As in the prior example, the perturbations illustrate sensitivity and are not probability-calibrated uncertainty intervals.

Uncertainty From the Target Population

For a target population with covariate distribution \(G\), the standardized risk is the average of profile-specific risks:

R_k(t;G) = \int F_k(t\mid x)\,dG(x).

In practice, a sample of \(n_T\) target individuals often represents \(G\), yielding \(\hat R_k(t)=n_T^{-1}\sum_{i=1}^{n_T}\hat F_k(t\mid X_i)\). Holding the fitted model fixed does not make this average constant across target samples. If the target is a fixed, fully enumerated finite group and the covariates are known, there is no target-sampling component for that group; model-estimation uncertainty remains. If the target sample is intended to represent a larger population, its sampling variation is relevant.

Worked Example: Target-Sample Composition Changes the Average

A hypothetical fitted model assigns risk 0.10 to each low-risk profile and 0.30 to each high-risk profile at a specified horizon. In a target sample of 10 people, four have the high-risk profile and six the low-risk profile. With the model held fixed, the standardized estimate is

\hat R = \frac{4(0.30)+6(0.10)}{10} = \frac{1.20+0.60}{10} = 0.18.

If a different sample of 10 from the same target population contains six high-risk and four low-risk profiles, its average is

\hat R_{\mathrm{alt}} = \frac{6(0.30)+4(0.10)}{10} = \frac{1.80+0.40}{10} = 0.22.

For a simple illustration of the sampling scale, suppose target profiles are independently sampled and the population proportion with the high-risk profile is \(p=0.40\). Each person's model-assigned risk is \(0.10+0.20I\), where \(I\) is 1 for a high-risk profile and 0 otherwise. With the model fixed, the standard error of the sample average due only to sampling profile composition is

SE(\hat R) = 0.20\sqrt{\frac{p(1-p)}{n_T}} = 0.20\sqrt{\frac{(0.40)(0.60)}{10}} \approx 0.0310.

This standard error concerns variation in the target-sample composition under the stated independent-sampling assumption. It excludes uncertainty in the regression coefficients and baseline, and does not include outcome randomness for any one person.

Putting the Sources Together

Let \(D\) denote the model-fitting data and \(T\) the target sample. When these samples are independent, a useful conceptual variance decomposition follows from the law of total variance:

\operatorname{Var}_{D,T}(\hat R) = E_T\!\left\{\operatorname{Var}_D(\hat R\mid T)\right\} + \operatorname{Var}_T\!\left\{E_D(\hat R\mid T)\right\}.

To justify the identity, write \(\hat R-E(\hat R)=[\hat R-E(\hat R\mid T)]+[E(\hat R\mid T)-E(\hat R)]\), square, and take expectations. The cross-product has expectation zero because \(E\{\hat R-E(\hat R\mid T)\mid T\}=0\). The first term is average variation from model fitting for a given target sample; it includes the joint uncertainty in regression coefficients and baseline estimates. The second is variation in the model-averaged prediction across target samples. Independence makes this separation especially interpretable; if the same observations inform both model fitting and target composition, the sources are not cleanly separated in this way.

This decomposition is a way to organize uncertainty, not a recipe for adding independently calculated standard errors. In particular, coefficient and baseline estimates may be correlated, and ignoring their covariance can misstate the uncertainty in a nonlinear predicted risk. A fixed-profile prediction has no target-sample averaging component, while a standardized prediction can have both model-fitting and target-sampling components.

Diagnostics, Alternatives, and Limitations

Before interpreting an uncertainty assessment, check what it actually varies. If it varies coefficients but freezes the baseline, it omits baseline-estimation uncertainty. If it uses a fixed covariate profile, it says nothing about variability from sampling a target population. If its target is a sampled group, clarify whether inference concerns that particular group or a larger population represented by it. Also examine whether the prediction horizon is supported by adequate follow-up and whether the model's covariate specification and event coding are defensible. A narrow uncertainty summary cannot repair weak data support or a systematically unsuitable model.

Different approaches answer different questions. A profile-specific prediction holds \(x\) fixed and focuses on uncertainty in the fitted model. Standardization averages predictions across a target covariate distribution and may add target-sampling uncertainty. Sensitivity analysis changes selected inputs to show how predictions respond, but unless perturbations are derived from a sampling distribution it does not provide a confidence interval. Formal interval procedures quantify sampling uncertainty under specified assumptions; which uncertainty they include depends on what is re-estimated and what is held fixed. These methods do not, by themselves, measure uncertainty from unmodeled structure or guarantee transportability to a different population.

A frequent mistake is to report the estimated risk as if all uncertainty were captured by the coefficient standard errors. This reasoning overlooks the estimated baseline and, for standardized predictions, the target population. The correction is to name the estimand and identify which data and quantities vary under the uncertainty assessment. A predicted risk of 0.18 is an estimate at the specified horizon and target; it is not a complete statement of precision or model adequacy.

Key takeaway: Absolute-risk uncertainty can arise from regression coefficients, baseline hazards, and target-population sampling. These sources can interact, and the uncertainty reported is meaningful only when its target and included sources are clear.

Check Your Understanding

Use the assumptions in each question and show the distinction between model and target-population uncertainty.

A risk prediction is made for one fully specified covariate profile. Which of the three sources discussed here is absent by construction?
A standardized risk averages profile-specific risks in a sampled target group. Name two sources of variation that can affect the average.
For a Fine–Gray prediction, suppose the estimated baseline cumulative subdistribution hazard is held fixed while the coefficient changes. What source is being varied, and why is the resulting range not automatically a confidence interval?
In the target-sampling example, what is the average risk if 5 of 10 people have predicted risk 0.30 and the other 5 have predicted risk 0.10?
Why can coefficient and baseline uncertainty not generally be combined by treating the two fitted quantities as independent?

Solutions. (1) Target-population sampling uncertainty is absent because there is no target-sample average; coefficient and baseline estimation uncertainty can remain. (2) Model-fitting uncertainty, including coefficients and baseline, and sampling variation in the target covariate composition can both matter. (3) Coefficient uncertainty is being varied. The range is not automatically a confidence interval because the perturbations have not necessarily been calibrated to a sampling distribution or coverage level. (4) The average is \(\{5(0.30)+5(0.10)\}/10=(1.50+0.50)/10=0.20\). (5) They are estimated within the same fitted model and can have correlated sampling errors; ignoring that dependence can misstate the variation of the nonlinear risk prediction.