The Probit Link and Normal Quantiles

Why the Probit Link Uses Normal Quantiles

A Bernoulli response has a mean that is also a probability, so its possible values lie between zero and one. A linear predictor, by contrast, can take any real value. A link function connects these two scales. The probit link makes that connection using quantiles of the standard Normal distribution: it assigns a Normal quantile to each possible Bernoulli mean.

This choice has a convenient inverse. Once a model supplies a linear predictor, the standard Normal cumulative distribution function turns it directly into a probability. The response is still Bernoulli; using the Normal distribution function to specify its mean does not make the observed response Normally distributed.

Definition: Let $\Phi(z)=P(Z\leq z)$ be the cumulative distribution function of $Z\sim N(0,1)$, and let $\Phi^{-1}(p)$ be its quantile function for $0<p<1$. The probit link for a Bernoulli mean $p$ is $$ g(p)=\Phi^{-1}(p). $$ Its inverse link is $g^{-1}(\eta)=\Phi(\eta)$, where $\eta$ is a real-valued linear predictor.

From a Bernoulli Mean to a Linear Predictor

In a Bernoulli model, $Y_i$ takes the value one for an event and zero otherwise. Conditional on the covariates, its mean is $p_i=P(Y_i=1\mid x_i)$. In a probit model, the mean and linear predictor are connected by

\eta_i=g(p_i)=\Phi^{-1}(p_i).

A common specification makes the predictor linear in the covariates: $\eta_i=x_i^\mathsf{T}\beta$, where $x_i$ is the covariate vector and $\beta$ is the vector of coefficients. Combining the equations gives $p_i=\Phi(x_i^\mathsf{T}\beta)$. This formula is the basic calculation for obtaining fitted probabilities from a probit model.

The link is well-defined for every probability strictly between zero and one. The quantile $\Phi^{-1}(p)$ is the value $z$ for which the area under the standard Normal curve to the left of $z$ is $p$. Thus the link-scale value for a mean of $0.80$, for example, is the 80th percentile of the standard Normal distribution.

Worked Example: Transforming a Bernoulli Mean

Suppose a hypothetical Bernoulli outcome has event probability $p=0.80$. Find its probit-scale value and check the result.

By definition, $\eta=\Phi^{-1}(p)$. Substituting $p=0.80$,

\eta=\Phi^{-1}(0.80)\approx 0.8416.

To check, return to the probability scale: $\Phi(0.8416)\approx 0.8000$. The probit-scale value is therefore about $0.8416$, with no physical unit: it is a standard Normal quantile. This calculation transforms a specified probability; it does not estimate the probability from data.

From a Linear Predictor to a Fitted Probability

A fitted linear predictor can be any real number. To turn it into a fitted event probability, apply the inverse link, not the link itself. If the predictor is $\hat{\eta}_i$, then

\hat{p}_i=\Phi(\hat{\eta}_i) =\int_{-\infty}^{\hat{\eta}_i}\frac{1}{\sqrt{2\pi}} \exp\!\left(-\frac{z^2}{2}\right)\,dz.

The integral is usually evaluated with a Normal distribution table or statistical software. A useful first check is that $\Phi(0)=0.5$; positive predictors give probabilities above $0.5$, and negative predictors give probabilities below $0.5$. By symmetry of the standard Normal distribution, $\Phi(-a)=1-\Phi(a)$.

Worked Example: Converting a Negative Predictor

For a hypothetical observation, suppose the fitted linear predictor is $\hat{\eta}=-0.5244$. Calculate the fitted event probability.

The inverse probit is the standard Normal CDF, so

\hat{p}=\Phi(-0.5244)\approx 0.3000.

Thus the fitted probability of the event is approximately $0.3000$, or $30.00\%$. The negative sign is consistent with a probability below $0.5$. As a check using symmetry, $\Phi(-0.5244)=1-\Phi(0.5244)\approx 1-0.7000=0.3000$.

Worked Example: Using Covariates to Obtain a Fitted Probability

Consider a hypothetical probit model with predictor $\hat{\eta}=-1.20+0.40x$, where $x$ is a recorded covariate. For an observation with $x=2$, calculate the fitted event probability.

First substitute the covariate value into the linear predictor:

\hat{\eta}=-1.20+0.40(2)=-1.20+0.80=-0.40.

Then apply the inverse link:

\hat{p}=\Phi(-0.40)\approx 0.3446.

The model therefore gives a fitted event probability of about $0.3446$, or $34.46\%$, for this observation. This is a model-based prediction on the response scale. The calculation alone does not establish that changing $x$ causes a change in the event probability.

Why the Transformation Works

The probit link is valid because the standard Normal CDF maps real numbers continuously and strictly increasingly onto probabilities between zero and one. In particular, its density is positive at every finite real number:

\Phi'(z)=\frac{1}{\sqrt{2\pi}}\exp\!\left(-\frac{z^2}{2}\right)>0 \quad\text{for every finite }z.

The CDF approaches zero as $z$ tends to negative infinity and approaches one as $z$ tends to positive infinity. Consequently, every probability $p$ strictly between zero and one corresponds to exactly one finite value $z$. That value is $\Phi^{-1}(p)$, and applying $\Phi$ to it returns $p$. This one-to-one mapping justifies both the link and its inverse.

Another way to understand the construction is to introduce a standard Normal variable $Z$. For any fixed $\eta$,

P(Z\leq\eta)=\Phi(\eta).

Thus setting $p=\Phi(\eta)$ expresses a Bernoulli probability as a Normal-CDF area. This is a mathematical representation of the mean; it does not require the observed binary outcome to have a Normal distribution or require a measured Normal latent response.

Assumptions and Practical Calculation Checks

The link itself is a transformation and does not specify every part of a statistical model. In the simple Bernoulli GLM setting considered here, the response for observation $i$ is binary, its conditional mean is $p_i$, and the model specifies $\Phi^{-1}(p_i)=x_i^\mathsf{T}\beta$. When using the ordinary independent-observation Bernoulli likelihood for fitting such a model, outcomes are assumed independent conditional on their covariates and model parameters. The probability calculation above is valid given a predictor, whether that predictor was supplied or estimated.

For accurate calculations, keep the scales distinct. A value such as $-0.40$ is a linear predictor; $\Phi(-0.40)\approx0.3446$ is a probability. A Normal table may report cumulative area to the left, area from zero to a positive value, or an upper-tail area. Confirm which convention it uses before reading off a value. With software, use the standard Normal CDF for a predictor and the standard Normal quantile function for a probability.

The probit link is one option for relating a Bernoulli mean to a linear predictor. The logit link is a familiar alternative. Both can yield probabilities from linear predictors, but their link-scale values are defined by different functions, so a numerical predictor on one scale should not be treated as the same numerical value on the other. Choosing between links is a modeling decision, not a reason to apply the wrong inverse function during prediction.

Common Mistakes and Limitations

Applying the link in the wrong direction. If the task is to find a probability from $\hat{\eta}$, use $\Phi(\hat{\eta})$, not $\Phi^{-1}(\hat{\eta})$. The quantile function takes a probability as input.
Calling a probit predictor a probability. A predictor of $-0.40$ is not a negative probability. It is a real number that must be transformed; here it corresponds to $\Phi(-0.40)\approx0.3446$.
Assuming the outcome is Normally distributed. A Bernoulli response remains zero or one. The Normal CDF defines the relationship between its conditional mean and the predictor.
Rounding too early. If a predictor or quantile is rounded before applying the CDF, the final probability can shift. Keep extra digits during intermediate calculations, then round the reported answer.
Expecting finite predictors to give exact endpoints. For every finite $\eta$, $0<\Phi(\eta)<1$. Exact probabilities of zero and one have no finite probit quantiles.

The link does not guarantee that the linear predictor describes the data well. Poor covariate specification, dependence not accounted for in the model, or inaccurate parameter estimates can all make fitted probabilities unsuitable. A correct transformation is necessary for probit predictions, but it cannot correct problems in the assumptions or the model that supplied the predictor.

Key takeaway: The probit link sends a Bernoulli mean $p$ to its standard Normal quantile, $\Phi^{-1}(p)$. To obtain a fitted probability from a linear predictor, use the inverse link $\Phi(\eta)$.

Check Your Understanding

Answer each question using the probit definition, then compare your work with the solutions.

What is the probit-scale value for a Bernoulli mean of $0.50$?
A hypothetical fitted predictor is $1.0364$. What is the approximate fitted event probability?
What probability corresponds approximately to a predictor of $-1.6449$?
Find the probit-scale value for a mean of $0.25$, using the standard Normal quantile.
A student says, “The predictor is $-0.40$, so the fitted probability is $-0.40$.” Identify the error and give the correct fitted probability.

Solutions. (1) $\Phi^{-1}(0.50)=0$, because half of the standard Normal area lies below zero. (2) $\hat{p}=\Phi(1.0364)\approx0.8500$, so the fitted event probability is about $85.00\%$. (3) $\Phi(-1.6449)\approx0.0500$, or $5.00\%$. (4) $\Phi^{-1}(0.25)\approx-0.6745$, since $\Phi(-0.6745)\approx0.2500$. (5) The predictor is on the link scale, not the probability scale. Apply the inverse link: $\Phi(-0.40)\approx0.3446$, so the fitted probability is about $34.46\%$.