Deriving Probit From A Latent Gaussian Threshold

From a Latent Threshold to a Binary Outcome

A binary outcome records which side of a boundary an observation falls on, not the distance from that boundary. One way to model such an outcome is to suppose that an unobserved, continuous response lies behind it, and that the recorded value is one precisely when this latent response exceeds a threshold. If the latent response has a Gaussian distribution whose mean depends linearly on covariates, the resulting conditional probability has the probit form.

The derivation also reveals an important limitation: binary observations do not determine the units in which the latent response is measured. Multiplying the latent response, its threshold, and its error by the same positive constant leaves every binary outcome unchanged. The conventional probit model resolves this scale ambiguity by fixing the latent error variance at one.

Definition: Let $Y\in\{0,1\}$ be a binary outcome and $x$ a covariate vector. A latent Gaussian threshold model assumes an unobserved variable $Y^*$ with conditional distribution $Y^*\mid x\sim N(x^\mathsf{T}\beta,\sigma^2)$, where $\sigma>0$, and records $Y=1$ if $Y^*>c$, and $Y=0$ otherwise. Here $c$ is a fixed threshold. The standard probit normalization uses $c=0$ and $\sigma^2=1$.

Write $\Phi(t)=P(U\leq t)$ for the standard normal cumulative distribution function, where $U\sim N(0,1)$. The symbol $x^\mathsf{T}\beta$ is the linear mean of the latent response. For the probability calculation, $x$ is treated as fixed, or equivalently the probability is conditional on $x$. Because a normal distribution is continuous, the probability of lying exactly at the threshold is zero, so it makes no difference here whether the threshold rule uses $>$ or $\geq$.

Deriving the Probit Probability

Under the definition above, $Y=1$ exactly when $Y^*>c$. Subtract the conditional mean and divide by the positive standard deviation $\sigma$. This expresses the event in standard normal units:

\begin{aligned} P(Y=1\mid x) &=P(Y^*>c\mid x)\\ &=P\left(\frac{Y^*-x^\mathsf{T}\beta}{\sigma}> \frac{c-x^\mathsf{T}\beta}{\sigma}\,\middle|\,x\right)\\ &=1-\Phi\left(\frac{c-x^\mathsf{T}\beta}{\sigma}\right)\\ &=\Phi\left(\frac{x^\mathsf{T}\beta-c}{\sigma}\right). \end{aligned}

The third line uses the standard normal distribution of $(Y^*-x^\mathsf{T}\beta)/\sigma$. The last line follows from symmetry of the standard normal distribution: $1-\Phi(-t)=\Phi(t)$. Thus the latent threshold model gives a probit probability, with standardized linear predictor $(x^\mathsf{T}\beta-c)/\sigma$. Under the conventional threshold-zero and unit-variance normalization, this becomes $\Phi(x^\mathsf{T}\beta)$.

Result: If $Y^*\mid x\sim N(x^\mathsf{T}\beta,\sigma^2)$, $\sigma>0$, and $Y=1$ exactly when $Y^*>c$, then $$ P(Y=1\mid x)=\Phi\left(\frac{x^\mathsf{T}\beta-c}{\sigma}\right). $$ In particular, the standard probit specification $c=0,\ \sigma^2=1$ has $P(Y=1\mid x)=\Phi(x^\mathsf{T}\beta)$.

Worked Example: A Zero-Threshold Model

Suppose $Y^*\mid x\sim N(-0.4+0.2x,1)$, and $Y=1$ when $Y^*>0$. Find the probability of $Y=1$ at $x=2$.

At $x=2$, the latent mean is $-0.4+0.2(2)=0$. The threshold is zero and the standard deviation is one, so standardization gives $$ P(Y=1\mid x=2)=\Phi\left(\frac{0-0}{1}\right)=\Phi(0)=0.5. $$

The result also follows directly from symmetry: a normal variable centered at zero has half its probability above zero. This checks both the threshold interpretation and the direction of the tail in the derivation.

Worked Example: A Nonzero Threshold

Suppose $Y^*\mid x\sim N(1+0.25x,4)$, and $Y=1$ when $Y^*>2$. Find the response probability at $x=2$.

The conditional mean at $x=2$ is $1+0.25(2)=1.5$, and the standard deviation is $\sqrt{4}=2$. The distance between the mean and threshold, measured in standard deviations, is $$ \frac{x^\mathsf{T}\beta-c}{\sigma} =\frac{1.5-2}{2}=-0.25. $$ Therefore $$ P(Y=1\mid x=2)=\Phi(-0.25)\approx 0.4013. $$

A probability below one half is sensible: the latent mean, 1.5, is below the threshold, 2. The variance matters because it determines how far that gap is in standard-deviation units.

Why the Latent Error Variance Must Be Fixed

The preceding probability depends on the latent model through the standardized quantity $(x^\mathsf{T}\beta-c)/\sigma$, not through the raw coefficient vector, threshold, and standard deviation separately. The next result makes the scale ambiguity explicit.

Theorem: For any $a>0$, replace the latent response $Y^*$, its conditional mean $x^\mathsf{T}\beta$, its threshold $c$, and its error standard deviation $\sigma$ by $aY^*$, $a x^\mathsf{T}\beta$, $ac$, and $a\sigma$, respectively. The resulting binary outcome is unchanged, and its conditional response probability is unchanged.

Proof. Let the original model be $Y^*=x^\mathsf{T}\beta+\varepsilon$, where $\varepsilon\mid x\sim N(0,\sigma^2)$. Define $Y_a^*=aY^*$. Then $$ Y_a^*=x^\mathsf{T}(a\beta)+a\varepsilon, $$ and $a\varepsilon\mid x\sim N(0,a^2\sigma^2)$. Since $a>0$, multiplication preserves the direction of the threshold inequality: $$ Y^*>c \quad\Longleftrightarrow\quad aY^*>ac \quad\Longleftrightarrow\quad Y_a^*>ac. $$ Thus both latent specifications assign the same binary value for each realization. Their probability formulas agree as well, because $$ \frac{x^\mathsf{T}(a\beta)-ac}{a\sigma} =\frac{a(x^\mathsf{T}\beta-c)}{a\sigma} =\frac{x^\mathsf{T}\beta-c}{\sigma}. $$ Consequently, the conditional probability of $Y=1$ is identical under the two specifications. $\square$

This equivalence means that the binary data cannot distinguish one unit of latent response from, for example, two units, if the coefficients, threshold, and error scale are all changed together. In the probability model, only the normalized coefficients $\beta/\sigma$ and normalized threshold $c/\sigma$ enter. The standard probit convention sets $\sigma=1$, fixing the latent scale and expressing coefficients in units of the latent error standard deviation. It does not mean that the data have directly measured the latent error variance to be one.

Worked Example: Two Latent Scales, the Same Probability

Consider a model with threshold zero, latent mean $-0.6+0.4x$, and error variance $4$. Compare it with the model obtained by multiplying the entire latent response by $0.5$.

The first model has standard deviation $2$. Its response probability at $x=1$ is $$ P(Y=1\mid x=1)=\Phi\left(\frac{-0.6+0.4(1)}{2}\right) =\Phi(-0.1)\approx 0.4602. $$

After multiplying the latent response by $0.5$, the mean becomes $-0.3+0.2x$, the threshold remains zero, and the error standard deviation becomes $1$, so its variance becomes $1$. The transformed model gives $$ P(Y=1\mid x=1)=\Phi\left(\frac{-0.3+0.2(1)}{1}\right) =\Phi(-0.1)\approx 0.4602. $$

The agreement is exact because the numerator and standard deviation were both multiplied by $0.5$. The two latent responses use different units, but they imply the same probability for the observed binary outcome.

Threshold Location and the Intercept

A threshold other than zero can be represented using a zero threshold by shifting the latent response. Define $Z^*=Y^*-c$. Then $Y=1$ exactly when $Z^*>0$, and, conditional on $x$, $$ Z^*\mid x\sim N(x^\mathsf{T}\beta-c,\sigma^2). $$ If the covariate vector includes an intercept, the subtraction of $c$ can be absorbed into that intercept. For example, if $x^\mathsf{T}\beta=\beta_0+\beta_1x_1+\cdots+\beta_px_p$, then the shifted mean is $(\beta_0-c)+\beta_1x_1+\cdots+\beta_px_p$. A separate threshold and intercept are therefore not separately determined by this probability specification: only their difference appears.

This location convention and the scale convention solve different ambiguities. Shifting the latent origin places the threshold at zero; fixing the error variance at one chooses the latent unit. Under both conventions, the model is written $P(Y=1\mid x)=\Phi(x^\mathsf{T}\beta)$. If the latent error variance were instead left free, dividing the linear predictor by its standard deviation would still be necessary to obtain the response probability.

What the Construction Means

The threshold construction is a coherent latent-variable interpretation of probit regression, not a claim that every binary outcome is literally generated by a measurable Gaussian quantity. The observed data identify conditional probabilities; the latent response is a modeling device that produces those probabilities. Its scale is not directly observed, which is why coefficient magnitudes require the unit-variance convention for interpretation.

A positive latent coefficient shifts the conditional Gaussian distribution toward the event $Y^*>c$, and therefore raises the response probability, all else equal. But the coefficient is a change in latent standard-deviation units under the standard normalization; it is not itself a probability change. To obtain probabilities, the linear predictor must be passed through $\Phi$.

Common Pitfalls

Using the lower tail for the event above the threshold. Writing $P(Y=1\mid x)=\Phi((c-x^\mathsf{T}\beta)/\sigma)$ reverses the event. For $Y=1$ when $Y^*>c$, the correct expression is $1-\Phi((c-x^\mathsf{T}\beta)/\sigma)=\Phi((x^\mathsf{T}\beta-c)/\sigma)$.
Dividing by the variance instead of the standard deviation. Standardization divides by $\sigma$, not $\sigma^2$. If the error variance is $4$, the standard deviation used in the probability formula is $2$.
Treating unit variance as empirically discovered. The observed binary outcome is unchanged by common positive rescaling of the latent quantities. Fixing the variance at one sets a scale convention; it does not estimate an observable latent unit.
Separating an intercept and threshold without a constraint. With an intercept, the probability formula contains their difference, not each quantity independently. One may fix the threshold at zero and use an intercept, or use another explicit location convention.

Key takeaway: A Gaussian latent response crossing a threshold gives a probit probability after standardization. The probit index is the latent mean minus the threshold, divided by the latent error standard deviation; setting the threshold to zero and the variance to one fixes location and scale conventions.

Check Your Understanding

Use the threshold rule and standardization before evaluating any normal probabilities.

If $Y^*\mid x\sim N(2,9)$ and $Y=1$ when $Y^*>1$, what is $P(Y=1\mid x)$ in terms of $\Phi$?
Explain why the probability for a zero-threshold latent model is $\Phi(x^\mathsf{T}\beta/\sigma)$, rather than $\Phi(-x^\mathsf{T}\beta/\sigma)$.
What happens to the conditional binary probabilities if every latent quantity, including the error standard deviation, is multiplied by a positive constant?
Why does a standard probit model fix the latent error variance rather than estimate it freely?
If a model includes an intercept and a nonzero threshold, which combination of the intercept and threshold appears in the response probability?