Uncertainty Intervals for Predicted Binary Probabilities

Why a Predicted Probability Needs an Interval

A fitted binary-response model can report a probability such as 0.62 for an individual with specified covariates. That number is an estimate, not a certainty: the model coefficients were estimated from a finite sample. A confidence interval describes the resulting sampling uncertainty about the conditional probability at those covariate values. It does not say that the individual outcome itself is uncertain by exactly the interval’s width.

The interval is usually constructed on the model’s index scale, where coefficient uncertainty is approximately normal, and then transformed back to the probability scale. This is especially useful because it respects the probability range and naturally allows uncertainty to be asymmetric around the estimated probability.

The Model and the Target

Let \(Y_i\) be a binary outcome and \(x_i\) a vector of observed covariates, including an intercept if the model has one. In a logit or probit model, the conditional mean of \(Y_i\), which is also the conditional probability that \(Y_i=1\), is linked to a linear index:

For logit, \(G(t)=\Lambda(t)=1/(1+\exp(-t))\); for probit, \(G(t)=\Phi(t)\), the standard normal cumulative distribution function. Both links are increasing and map any real index to a probability strictly between zero and one.

Suppose the fitted coefficient vector is \(\hat{\beta}\), with estimated covariance matrix \(\widehat V\). The estimated index at \(x\) is \(\hat{\eta}(x)=x^\top\hat{\beta}\), and the estimated probability is \(\hat p(x)=G(\hat{\eta}(x))\). For the usual large-sample interval, assume the binary-response model is correctly specified, observations provide independent information, the coefficients are identified, and the estimator is in a regular large-sample setting with nonsingular information. The covariance estimate must be appropriate for the sampling design and dependence structure. The covariate vector \(x\) is treated as fixed for this pointwise calculation.

Constructing an Interval by Transforming the Index

Because the index is linear in the estimated coefficients, its estimated variance follows directly from the coefficient covariance matrix. A large-sample confidence interval for the index is then transformed using the model’s increasing link function.

The calculation has four steps: evaluate the index at the covariate values of interest, find its standard error, form the index-scale interval, and apply \(G\) to both endpoints. For a 95% interval, \(z_{0.975}\) is approximately 1.96. For logit, apply \(\Lambda\); for probit, apply \(\Phi\). Do not transform the standard error itself or apply the link to the coefficient limits separately.

Why the Transformation Works

The justification uses the large-sample normal approximation for the coefficient estimator and the fact that the link is increasing. At fixed \(x\), the index is a linear combination of the coefficients, so its estimated variance is \(x^\top\widehat Vx\). Under the regularity conditions above, the standardized index error is approximately standard normal:

Consequently, the index interval from \(\hat{\eta}(x)-z_{1-\alpha/2}\widehat{SE}\) to \(\hat{\eta}(x)+z_{1-\alpha/2}\widehat{SE}\) covers \(\eta(x)\) with approximate probability \(1-\alpha\) over repeated samples. Since \(G\) is increasing, the event that the index lies between these endpoints is exactly the event that \(G(\eta(x))\) lies between the transformed endpoints. Thus the transformed interval inherits the index interval’s approximate coverage for \(p(x)\). The approximation comes from large-sample inference; transforming does not make it an exact finite-sample interval.

Why the Interval Can Be Asymmetric

A symmetric interval for the index generally becomes asymmetric when mapped to the probability scale because the link is curved. This asymmetry is not a defect: probability changes for a given change in the index depend on where the index lies. The transformation also ensures that the endpoints remain in the probability range.

Conditional-Probability Uncertainty Is Not Outcome Uncertainty

For a fixed \(x\), the model says \(Y\mid x\) is Bernoulli with event probability \(p(x)\). Even if \(p(x)\) were known exactly, a future outcome would still vary: it would equal 1 with probability \(p(x)\) and 0 with probability \(1-p(x)\). That Bernoulli variation is distinct from sampling uncertainty in estimating \(p(x)\).

A confidence interval such as \([0.46,0.76]\) describes uncertainty about the conditional probability parameter at the specified profile. It does not mean that a future person’s outcome has a 95% chance of lying between 0.46 and 0.76; outcomes are zero or one. Nor does it imply that 95% of future outcomes will fall within the interval. Predicting an individual outcome requires a different target, while the interval here concerns the model’s conditional mean probability.

The target also matters across covariate profiles. A pointwise interval at one \(x\) is not automatically an interval for an average probability over a population, and it does not provide simultaneous coverage for probabilities at every possible \(x\). Those questions require defining the corresponding aggregate or joint target and its uncertainty.

Common Mistakes and Practical Checks

• Reporting a coefficient interval as a probability interval. Coefficients and indices are on the link scale. First form the interval for \(x^\top\beta\), then transform its endpoints with the inverse link.
• Using the wrong standard error. The standard error for the index is \(\sqrt{x^\top\widehat Vx}\), not the standard error of one coefficient unless the target depends only on that coefficient.
• Calling it an individual prediction interval. The interval quantifies uncertainty in \(P(Y=1\mid x)\), not the randomness of a future zero-or-one outcome.
• Ignoring the target profile. State the covariate values to which the probability refers. Changing \(x\) changes both the index and, generally, its standard error.
• Overstating coverage. This is an approximate pointwise interval under the model and inference assumptions, not a guarantee of exact coverage or simultaneous coverage over many profiles.

Before reporting an interval, check that the coefficient covariance estimate is appropriate, the index variance is nonnegative, the correct link has been used, and the result is described in context as a confidence interval for a conditional event probability. In a fitted binary model, the final probability bounds should lie between zero and one.