The Probit Link and Latent-Normal Threshold Model

From a Latent Index to a Binary Probability

A binary outcome records whether an event occurred, not how close a case came to an underlying threshold. A latent-variable model represents that unobserved position with a continuous index, then records only whether the index crosses a cutoff. When the latent error is normally distributed, this threshold rule gives the probit model: a conditional probability expressed through the standard normal cumulative distribution function.

The normal distribution is an assumption about the unobserved part of the latent index conditional on the regressors. It is not a claim that the observed binary outcome is normally distributed. The outcome remains binary; the normal distribution supplies a particular mapping from the index to its probability.

Definition: For observation $i$, let $Y_i\in\{0,1\}$ be a binary outcome and let $\mathbf{x}_i$ be a vector of observed regressors, including a constant if the model has an intercept. The latent-index model is $$ Y_i^*=\mathbf{x}_i'\boldsymbol{\beta}+\varepsilon_i,\qquad Y_i=1\ \text{if}\ Y_i^*>0,\quad Y_i=0\ \text{otherwise}, $$ where the latent error satisfies $\varepsilon_i\mid\mathbf{x}_i\sim N(0,\sigma_\varepsilon^2)$, with $\sigma_\varepsilon>0$. The standard normal CDF is $\Phi(z)=P(Z\le z)$ for $Z\sim N(0,1)$. The resulting conditional probability is $P(Y_i=1\mid\mathbf{x}_i)=\Phi(\mathbf{x}_i'\boldsymbol{\beta}/\sigma_\varepsilon)$. The probit normalization sets $\sigma_\varepsilon=1$, so the model is written $P(Y_i=1\mid\mathbf{x}_i)=\Phi(\mathbf{x}_i'\boldsymbol{\beta})$.

Deriving the Probit Probability

Write the latent index before the error as $\eta_i=\mathbf{x}_i'\boldsymbol{\beta}$. Conditional on $\mathbf{x}_i$, the event $Y_i=1$ is the event that the latent index plus the error is positive. Its probability follows by rearranging the threshold inequality, then standardizing the normal error.

\begin{aligned} P(Y_i=1\mid\mathbf{x}_i) &=P(\eta_i+\varepsilon_i>0\mid\mathbf{x}_i)\\ &=P(\varepsilon_i>-\eta_i\mid\mathbf{x}_i)\\ &=P\left(\frac{\varepsilon_i}{\sigma_\varepsilon}> -\frac{\eta_i}{\sigma_\varepsilon}\,\middle|\,\mathbf{x}_i\right)\\ &=1-\Phi\left(-\frac{\eta_i}{\sigma_\varepsilon}\right)\\ &=\Phi\left(\frac{\eta_i}{\sigma_\varepsilon}\right). \end{aligned}

The fourth line uses that $\varepsilon_i/\sigma_\varepsilon$ is standard normal conditional on $\mathbf{x}_i$. The last line follows from the symmetry of the standard normal distribution: $1-\Phi(-z)=\Phi(z)$. For the other outcome, the conditional probability is $P(Y_i=0\mid\mathbf{x}_i)=1-\Phi(\eta_i/\sigma_\varepsilon)$. Thus the threshold model specifies both probabilities, and they sum to one.

The probit link is the inverse mapping: for a probability $p$ strictly between zero and one, $\Phi^{-1}(p)$ is the standard-normal quantile corresponding to $p$. In the normalized model, $\Phi^{-1}(P(Y_i=1\mid\mathbf{x}_i))=\mathbf{x}_i'\boldsymbol{\beta}$. The latent index is therefore on a standard-normal quantile scale, not on the probability scale.

Worked Example: Probability at a Given Latent Index

Suppose a hypothetical binary outcome follows the threshold model with a standard-normal latent error and $\eta_i=-0.8+0.5x_i$. For a case with $x_i=2$, calculate the conditional probability of $Y_i=1$. Treat the threshold, conditional normality, and unit error variance as assumptions of this illustrative model.

Step 1: Calculate the index. $$ \eta_i=-0.8+0.5(2)=0.2. $$

Step 2: Apply the probit probability formula. Since the error standard deviation is one, standardization leaves the index unchanged: $$ P(Y_i=1\mid x_i=2)=\Phi(0.2)\approx 0.5793. $$

Under the stated model, the predicted probability of the outcome for a case with $x_i=2$ is about 57.93%. This is a model-based conditional probability, not a claim that the latent index itself equals 57.93%, nor does this calculation establish that changing $x_i$ causes the outcome to change.

Why the Latent Error Variance Is Normalized

A binary observation reveals which side of the threshold the latent variable falls on, but it does not reveal the distance from that threshold. Consequently, the observed outcome cannot distinguish every change in the size of the latent index from a corresponding change in the size of the latent error. The probability formula makes this scale issue explicit: it depends on the ratio $\eta_i/\sigma_\varepsilon$, not on the index and error standard deviation separately.

Key result: In the threshold model, replacing $\boldsymbol{\beta}$ by $a\boldsymbol{\beta}$ and $\sigma_\varepsilon$ by $a\sigma_\varepsilon$, for any $a>0$, leaves every conditional probability unchanged: $$ \Phi\left(\frac{\mathbf{x}_i'(a\boldsymbol{\beta})}{a\sigma_\varepsilon}\right) =\Phi\left(\frac{\mathbf{x}_i'\boldsymbol{\beta}}{\sigma_\varepsilon}\right). $$ The observed binary probabilities therefore identify the coefficient-to-error-scale ratio, not the absolute latent scale. The conventional probit model fixes that scale by setting $\sigma_\varepsilon=1$.

The normalization is a choice of units for an unobserved variable, not an empirical finding that the latent error truly has variance one in some independently measured unit. Under the normalized model, the reported coefficients are expressed in units of the latent index measured relative to an error standard deviation of one. Rescaling the latent equation and its error together preserves the threshold classification and all model probabilities.

The threshold is conventionally set to zero. If the original rule is $Y_i=1$ when $\mathbf{x}_i'\boldsymbol{\gamma}+\varepsilon_i>c$, its probability is $\Phi((\mathbf{x}_i'\boldsymbol{\gamma}-c)/\sigma_\varepsilon)$. When the regressors include a constant, the threshold can be incorporated into the intercept. This does not remove the need to fix the latent scale: multiplying the latent components and threshold by the same positive number still leaves the observed classification unchanged.

Worked Example: Two Covariate Values Under a Normalized Model

Consider a hypothetical model for a binary outcome with $P(Y_i=1\mid x_i)=\Phi(-0.7+0.9x_i)$. Assume the outcome is generated by a zero-threshold rule and a conditionally standard-normal latent error, as required by this normalized probit specification. Compute the probabilities at $x_i=0$ and $x_i=1$.

At $x_i=0$, the latent index is $$ \eta_i=-0.7+0.9(0)=-0.7, $$ so $$ P(Y_i=1\mid x_i=0)=\Phi(-0.7)\approx 0.2420. $$

At $x_i=1$, the latent index is $$ \eta_i=-0.7+0.9(1)=0.2, $$ so $$ P(Y_i=1\mid x_i=1)=\Phi(0.2)\approx 0.5793. $$

The model assigns probabilities of about 24.20% and 57.93% at these two covariate values. These are conditional predictions under the stated model assumptions. Their difference is a comparison of two model probabilities; by itself, it does not establish a causal effect.

Worked Example: Equivalent Latent Scales

Suppose that for a hypothetical covariate vector, the unnormalized latent index is $1.2$ and the latent error standard deviation is $2$. Assume the zero-threshold model and conditional normality. Find the probability, then rescale the entire latent equation by a positive factor of three and verify that the probability is unchanged.

The standardized index is $1.2/2=0.6$, giving $$ P(Y_i=1\mid\mathbf{x}_i)=\Phi(0.6)\approx 0.7257. $$

Multiplying the index and error standard deviation by three gives an index of $3(1.2)=3.6$ and an error standard deviation of $3(2)=6$. The new standardized index is $3.6/6=0.6$, so $$ \Phi\left(\frac{3.6}{6}\right)=\Phi(0.6)\approx 0.7257. $$

Both latent representations imply a probability of about 72.57%. The normalized representation divides the latent index by its error standard deviation and sets that standard deviation to one. It does not change the model's probability predictions.

Assumptions and Interpretation

The probit probability formula follows from a small set of substantive and distributional assumptions. The outcome must be recorded according to the specified binary threshold rule; conditional on the regressors, the latent error must have mean zero and a normal distribution; and a scale must be fixed to define the coefficient units. In the conventional parameterization, the error is conditionally standard normal. If instead the error has conditional mean $m(\mathbf{x}_i)$, that mean must be included in the latent index before applying the threshold calculation. If its conditional variance varies with $\mathbf{x}_i$, the standardized index also varies with $\mathbf{x}_i$; the ordinary normalized probit formula no longer describes that model without further specification.

The standard normal density, often written $\phi(z)$, and the CDF $\Phi(z)$ are different functions. A threshold probability is a cumulative probability, so it uses $\Phi$, not $\phi$. The density describes relative concentration near a point; it is not itself the probability of crossing a threshold. This distinction matters when moving from the latent model to a numerical probability.

Probit and logit both map a linear index into a probability, but they use different cumulative distribution functions. Probit uses the standard normal CDF and its latent-normal threshold representation. That representation motivates the model and fixes its conventional coefficient scale; it does not make the latent variable directly observable.

Common Pitfall

A frequent error is to write $P(Y_i=1\mid\mathbf{x}_i)=\Phi(\mathbf{x}_i'\boldsymbol{\beta})$ while also claiming that the latent error has an arbitrary standard deviation $\sigma_\varepsilon$. Those statements do not generally match: with arbitrary error standard deviation, the probability is $\Phi(\mathbf{x}_i'\boldsymbol{\beta}/\sigma_\varepsilon)$. The shorter expression is correct only after the scale has been normalized or the coefficients have already been expressed relative to the error standard deviation.

A related misinterpretation is to treat a normalized coefficient as if it were measured in natural units of the observed binary outcome. The outcome takes only the values zero and one; the coefficient instead shifts a latent index on the normalized scale. Its probability meaning is mediated by $\Phi$. Later analysis can quantify changes in predicted probabilities, but the coefficient itself is not a probability change.

Key takeaway: A probit model arises when a binary outcome indicates whether a normally distributed latent index crosses a threshold. Standardizing the latent error yields the normal CDF probability formula, while fixing its variance to one resolves the arbitrary latent scale and defines the units of the coefficients.

Check Your Understanding

Use the threshold rule and the scale normalization to answer these questions.

Starting from $Y_i=1$ when $\eta_i+\varepsilon_i>0$, show why a conditionally normal error with standard deviation $\sigma_\varepsilon$ gives $P(Y_i=1\mid\mathbf{x}_i)=\Phi(\eta_i/\sigma_\varepsilon)$.
Under the normalized probit model, what probability corresponds to a latent index of $-0.4$?
Why can multiplying both the latent index and its error standard deviation by the same positive constant leave predicted probabilities unchanged?
Suppose someone uses $\Phi(\eta_i)$ but specifies a latent error standard deviation of $2$. What is the issue, and what probability formula matches that stated error scale?
Does a probit coefficient directly measure a change in the probability of the outcome? Explain what scale it uses instead.