Logit-Scale Intervals for Probabilities

Why Use the Logit Scale?

A probability must lie between zero and one, but a standard Wald interval formed by adding and subtracting a margin on the probability scale can extend beyond those limits. It also imposes symmetry around the estimate, even when uncertainty near a boundary is naturally asymmetric. The logit transformation provides another scale on which to form an approximately symmetric interval, while its inverse maps the endpoints back into the unit interval.

The method is an asymptotic confidence-interval construction. It relies on an estimator that is approximately normal and sufficiently precise estimation of its variance. The transformation does not eliminate small-sample difficulties, and it requires care when the observed estimate is exactly zero or one.

Definition: For a probability $p$ with $0<p<1$, its logit is $$ \operatorname{logit}(p)=\log\!\left(\frac{p}{1-p}\right). $$ The inverse-logit, or expit, function is $$ \operatorname{expit}(u)=\frac{e^u}{1+e^u}, \qquad -\infty<u<\infty. $$ These functions are inverses: $\operatorname{expit}(\operatorname{logit}(p))=p$. The logit maps probabilities to the real line, and the expit maps real numbers to probabilities strictly between zero and one.

Deriving the Logit-Scale Standard Error

Suppose the parameter of interest is a fixed probability $p\in(0,1)$, and an estimator $\hat p$ satisfies $\sqrt n(\hat p-p)\xrightarrow{d}N(0,V(p))$, where $0<V(p)<\infty$. Suppose also that $\hat p\xrightarrow{p}p$. The Delta Method applies to the smooth function $g(p)=\operatorname{logit}(p)$. Its derivative is $g'(p)=1/[p(1-p)]$, so the asymptotic variance on the logit scale is multiplied by the square of this derivative.

Theorem: Under the conditions just stated, $$ \sqrt n\{\operatorname{logit}(\hat p)-\operatorname{logit}(p)\} \xrightarrow{d} N\!\left(0,\frac{V(p)}{p^2(1-p)^2}\right). $$ If $\hat V$ is a consistent estimator of $V(p)$, then a plug-in estimate of the standard error of $\operatorname{logit}(\hat p)$ is $$ \widehat{\operatorname{se}}_{\operatorname{logit}} = \frac{\sqrt{\hat V}}{\sqrt n\,\hat p(1-\hat p)}, $$ provided $0<\hat p<1$.

Here is the Taylor-expansion justification. Because $p$ is interior and $\hat p\xrightarrow{p}p$, the probability that $\hat p$ is interior tends to one. On that event, the mean-value theorem gives a value $p_n^*$ between $p$ and $\hat p$ such that $\operatorname{logit}(\hat p)-\operatorname{logit}(p)=g'(p_n^*)(\hat p-p)$. Consistency implies $p_n^*\xrightarrow{p}p$, and continuity of $g'$ at $p$ implies $g'(p_n^*)\xrightarrow{p}g'(p)$. Slutsky's theorem then yields the displayed normal limit.

\sqrt n\{\operatorname{logit}(\hat p)-\operatorname{logit}(p)\} = g'(p_n^*)\sqrt n(\hat p-p) \xrightarrow{d} N\!\left(0,\frac{V(p)}{p^2(1-p)^2}\right).

For a two-sided confidence level $1-\alpha$, let $z_{1-\alpha/2}$ denote the $1-\alpha/2$ quantile of the standard normal distribution. The interval is formed on the logit scale around $\operatorname{logit}(\hat p)$, then both endpoints are transformed back:

\left[ \operatorname{expit}\!\left\{\operatorname{logit}(\hat p) -z_{1-\alpha/2}\widehat{\operatorname{se}}_{\operatorname{logit}}\right\}, \; \operatorname{expit}\!\left\{\operatorname{logit}(\hat p) +z_{1-\alpha/2}\widehat{\operatorname{se}}_{\operatorname{logit}}\right\} \right].

For a Bernoulli sample $X_1,\ldots,X_n$ with success probability $p$, take $\hat p=\bar X$. The Bernoulli variance is $p(1-p)$, and the sample proportion is a consistent estimator of $p$. Substituting $\hat V=\hat p(1-\hat p)$ gives $\widehat{\operatorname{se}}_{\operatorname{logit}}=1/\sqrt{n\hat p(1-\hat p)}$.

Result: For a Bernoulli sample with $0<\hat p<1$, the usual plug-in logit interval is $$ \left[ \operatorname{expit}\!\left\{\operatorname{logit}(\hat p) -\frac{z_{1-\alpha/2}}{\sqrt{n\hat p(1-\hat p)}}\right\}, \; \operatorname{expit}\!\left\{\operatorname{logit}(\hat p) +\frac{z_{1-\alpha/2}}{\sqrt{n\hat p(1-\hat p)}}\right\} \right]. $$

To verify the variance substitution, write $K=\sum_{i=1}^n X_i$, so $\hat p=K/n$. Independence gives $\operatorname{Var}(\hat p)=p(1-p)/n$, and the Bernoulli central limit theorem gives $\sqrt n(\hat p-p)\xrightarrow{d}N(0,p(1-p))$. Also, the law of large numbers gives $\hat p\xrightarrow{p}p$. By continuity, $\hat p(1-\hat p)\xrightarrow{p}p(1-p)$. Therefore $\hat V=\hat p(1-\hat p)$ is consistent, and substitution in the theorem's standard-error formula gives $\sqrt{\hat p(1-\hat p)}/[\sqrt n\,\hat p(1-\hat p)] =1/\sqrt{n\hat p(1-\hat p)}$.

Worked Constructions

Worked Example: A Binomial Proportion

In $n=100$ independent trials, suppose there are $x=12$ successes. Construct the 95% logit interval for the success probability. Use $z_{0.975}=1.96$.

The observed proportion is $\hat p=12/100=0.12$, which is strictly between zero and one. Its logit is $\log(0.12/0.88)=\log(0.13636\ldots)\approx-1.9924$.

The estimated standard error on the logit scale is \[ \frac{1}{\sqrt{100(0.12)(0.88)}}= \frac{1}{\sqrt{10.56}}\approx0.3078. \] The margin is $1.96(0.3078)\approx0.6033$. Thus the logit-scale endpoints are $-1.9924-0.6033=-2.5957$ and $-1.9924+0.6033=-1.3891$.

Back-transforming gives $\operatorname{expit}(-2.5957)\approx0.0694$ and $\operatorname{expit}(-1.3891)\approx0.1995$. The approximate 95% interval is therefore $[0.069,0.200]$. It is asymmetric around $0.12$, but both endpoints lie strictly between zero and one.

Worked Example: A Probability From a Logistic Model

A fitted logistic model estimates a linear predictor $\hat\beta=-0.40$ for a particular covariate pattern, with estimated standard error $0.25$. The probability for that pattern is $\hat p=\operatorname{expit}(\hat\beta)$. Construct the approximate 95% interval using the normal approximation for $\hat\beta$.

Here the logit of the probability is the linear predictor itself, so its estimated standard error is already $0.25$. The 95% interval on this scale is $-0.40\pm1.96(0.25)=-0.40\pm0.49$, or $[-0.89,0.09]$.

Transform each endpoint separately: $\operatorname{expit}(-0.89)=e^{-0.89}/(1+e^{-0.89})\approx0.291$, and $\operatorname{expit}(0.09)=e^{0.09}/(1+e^{0.09})\approx0.522$. The probability interval is approximately $[0.291,0.522]$. This calculation uses the model's estimated uncertainty for the linear predictor; it does not replace that uncertainty estimate with the Bernoulli sample-proportion formula.

Worked Example: A Small Estimated Probability

In $n=50$ Bernoulli trials, exactly one success is observed. Compare the 95% logit interval with the ordinary Wald interval, using $z_{0.975}=1.96$.

The estimate is $\hat p=1/50=0.02$. The logit estimate is $\log(0.02/0.98)\approx-3.8918$, and the logit standard error is \[ \frac{1}{\sqrt{50(0.02)(0.98)}}=\frac{1}{\sqrt{0.98}}\approx1.0102. \] The margin is $1.96(1.0102)\approx1.9799$, giving logit endpoints approximately $-5.8717$ and $-1.9119$. Their inverse logits are approximately $0.0028$ and $0.1288$. The logit interval is thus $[0.0028,0.1288]$.

For comparison, the ordinary Wald standard error is $\sqrt{0.02(0.98)/50}=\sqrt{0.000392}\approx0.0198$. Its endpoints are $0.02\pm1.96(0.0198)$, or approximately $[-0.0188,0.0588]$. The negative lower endpoint cannot be a probability. The logit interval respects the parameter range, although the wide interval also signals considerable uncertainty from the small sample.

Reading the Interval and Handling Boundaries

The logit construction is symmetric around the estimate on the transformed scale, not on the probability scale. Its endpoints can be understood through odds: if the logit-scale margin is $c$, then exponentiating the endpoint logits multiplies the estimated odds $\hat p/(1-\hat p)$ by $e^{-c}$ or $e^c$. Mapping those odds back to probabilities generally produces unequal distances from $\hat p$.

For a fixed interior $p$, a consistent estimator is close to $p$ with probability tending to one. Thus, in the Bernoulli setting, the probability that $\hat p$ equals zero or one tends to zero as $n$ increases. But when the observed estimate is exactly zero or one, its logit is undefined and the plug-in standard error formula cannot be used. One must not substitute zero or one into the formula and treat the result as an ordinary logit interval. A different interval procedure or an explicitly specified adjustment is needed for such data.

Common pitfall: A probability interval cannot be made into a logit interval by transforming only its center or by applying the inverse logit to the original-scale standard error. The uncertainty must first be estimated on the logit scale—using the derivative $1/[p(1-p)]$, or a model-based standard error already on that scale—and then both logit endpoints must be inverse-transformed. The plug-in formula also requires $0<\hat p<1$.

The logit-scale method is useful when a probability parameter is interior and an approximately normal estimator, together with a defensible variance estimate, is available. It produces endpoints in the allowable range and accommodates asymmetry on the probability scale. It is still an asymptotic method: respecting the range does not guarantee accurate coverage in a small sample, particularly when data are sparse or estimates are near a boundary.

Check Your Understanding

Use the definitions and construction above to answer the following.

What are the logit and inverse-logit of $p=0.5$?
If $\sqrt n(\hat p-p)\xrightarrow{d}N(0,V(p))$, what is the asymptotic variance of the logit-transformed estimator?
For a Bernoulli sample with $n=200$ and $x=40$, write the plug-in logit standard error before calculating any interval endpoints.
Why does the Bernoulli logit interval formula fail when $x=0$ or $x=n$?
In a logistic model, if a coefficient estimate is the estimated logit for a covariate pattern, what scale should its standard error describe before the endpoints are inverse-transformed?

Logit-Scale Intervals for Probabilities

What You'll Learn