The Logit Link for Bernoulli Means

Why Transform a Bernoulli Mean?

A Bernoulli response $Y_i$ records one of two outcomes, conventionally coded $1$ and $0$. Its conditional mean is also its success probability: if the covariates for observation $i$ are $x_i$, write $\mu_i=E(Y_i\mid x_i)=P(Y_i=1\mid x_i)$. This mean must be between zero and one. A linear predictor such as $\eta_i=x_i^\mathsf{T}\beta$, however, can be any real number. The logit link connects these scales by transforming a probability to the logarithm of its odds.

This transformation is useful because it allows a model to have an unrestricted real-valued predictor while its inverse always returns a probability strictly between zero and one. The focus here is the transformation itself: defining it, deriving its Bernoulli canonical form, and calculating probabilities from given predictor values.

Definition: For a Bernoulli mean $0<\mu<1$, the logit link is $$ g(\mu)=\operatorname{logit}(\mu)=\log\left(\frac{\mu}{1-\mu}\right). $$ The ratio $\mu/(1-\mu)$ is the odds of the event $Y=1$. Thus the logit is the log odds. Its inverse, the inverse logit or logistic function, is $$ g^{-1}(\eta)=\frac{\exp(\eta)}{1+\exp(\eta)}=\frac{1}{1+\exp(-\eta)}, \qquad -\infty<\eta<\infty. $$

The logit maps the open probability interval $(0,1)$ onto the entire real line. If $\mu=1/2$, the odds equal one and the logit is zero. If $\mu>1/2$, the odds exceed one and the logit is positive. If $\mu<1/2$, the odds are below one and the logit is negative. These statements concern odds, not probabilities: for example, a logit of zero means probability one-half, not probability zero.

Why the Logit Is the Bernoulli Canonical Link

The logit is not merely a convenient transformation. It is the canonical link for a Bernoulli response. To see this, let $Y$ have a Bernoulli distribution with success probability $\mu$, so that $P(Y=y)=\mu^y(1-\mu)^{1-y}$ for $y\in\{0,1\}$. Taking logarithms and rearranging gives the exponential-family expression below.

\begin{aligned} \log P(Y=y) &=y\log(\mu)+(1-y)\log(1-\mu)\\ &=y\log\left(\frac{\mu}{1-\mu}\right)+\log(1-\mu)\\ &=y\theta-A(\theta), \end{aligned}

Here the natural parameter is $\theta=\log(\mu/(1-\mu))$, and $A(\theta)=-\log(1-\mu)$. The canonical link sets the linear predictor equal to the natural parameter. Consequently, $\eta=\theta=\operatorname{logit}(\mu)$, which is exactly the logit link. This algebra identifies the canonical link; it does not mean that every Bernoulli model must use it.

Key result: In a Bernoulli GLM using the logit link, the conditional mean and predictor satisfy $$ \eta_i=\log\left(\frac{\mu_i}{1-\mu_i}\right) \qquad\Longleftrightarrow\qquad \mu_i=\frac{\exp(\eta_i)}{1+\exp(\eta_i)}. $$ For a finite real-valued $\eta_i$, the resulting $\mu_i$ is strictly between zero and one.

From Probability to Log Odds

To calculate a logit, first compute the probability of the event and the probability of its complement. Divide the former by the latter to get the odds, then take the natural logarithm. The order of the ratio matters: reversing it changes the sign. Probabilities must be entered as proportions, not as percentages such as 20.

Worked Example: Converting a Probability to a Logit

Suppose a hypothetical Bernoulli outcome has success probability $\mu=0.20$. Find its odds and logit.

The probability of failure is $1-\mu=1-0.20=0.80$. Therefore the odds of success are $$ \frac{\mu}{1-\mu}=\frac{0.20}{0.80}=0.25. $$

Taking the natural logarithm gives $$ \operatorname{logit}(0.20)=\log(0.25)\approx -1.386. $$

The negative result is consistent with a probability below one-half: success is less likely than failure, so the success odds are less than one. The value $-1.386$ is on the log-odds scale, not the probability scale.

The endpoint probabilities require care. At $\mu=0$, the odds are zero and their logarithm is not finite; at $\mu=1$, the denominator $1-\mu$ is zero. The logit is therefore defined for $0<\mu<1$, not at the endpoints. In contrast, every finite predictor has a valid inverse-logit probability in that open interval. Exact zero and one can be approached as the predictor tends toward negative and positive infinity, respectively, but no finite predictor produces either endpoint.

From a Linear Predictor to a Probability

When a model specifies $\eta_i$, the response-scale mean is found by applying the inverse logit. There are two equivalent calculation routes: substitute $\eta_i$ into $\exp(\eta_i)/(1+\exp(\eta_i))$, or first calculate the odds $\exp(\eta_i)$ and then convert odds $o$ to probability $o/(1+o)$. The second route often helps make the arithmetic easier to interpret.

Worked Example: Inverting a Positive Predictor

For a hypothetical observation, suppose the specified linear predictor is $\eta=1.2$. Calculate the Bernoulli mean.

First calculate the odds: $$ \exp(\eta)=\exp(1.2)\approx 3.320. $$

Convert these odds to a probability: $$ \mu=\frac{\exp(\eta)}{1+\exp(\eta)} =\frac{3.320}{1+3.320} =\frac{3.320}{4.320} \approx 0.7685. $$

The model-implied success probability is approximately $0.7685$, or $76.85\%$. The positive predictor corresponds to odds greater than one and thus to a probability greater than one-half, as a check on the result.

A predictor may itself be calculated from covariates. If a model specifies $\eta_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}$, calculate that linear expression first, and then apply the inverse logit to the resulting number. Do not apply the inverse logit separately to each term and combine the resulting probabilities.

Worked Example: Calculating a Probability From Covariates

Consider a hypothetical model with predictor $\eta=-0.70+0.45x$. For an observation with $x=2$, calculate the linear predictor and the model-implied probability.

Substitute the covariate value into the predictor: $$ \eta=-0.70+0.45(2)=-0.70+0.90=0.20. $$

Now apply the inverse logit: $$ \mu=\frac{\exp(0.20)}{1+\exp(0.20)} =\frac{1.2214}{1+1.2214} =\frac{1.2214}{2.2214} \approx 0.5498. $$

At $x=2$, the model therefore gives a success probability of about $0.5498$, or $54.98\%$. The predictor is slightly above zero, so the probability should be slightly above one-half; the computed value passes that check.

Assumptions, Checks, and Failure Modes

The link calculation relies on a limited set of conditions. The response is Bernoulli-coded, so its conditional mean is the conditional success probability. The probability used as an input to the logit must lie strictly between zero and one; a predictor supplied to the inverse logit must be a finite real number. For a GLM, the stated model relates the conditional mean to the predictor through $\operatorname{logit}(\mu_i)=\eta_i$. These facts are sufficient for the pointwise transformation. Conditional independence of observations is a separate assumption commonly used to form a product likelihood; it is not needed to define or invert the logit for one observation.

Several quick checks catch common errors. A calculated inverse-logit value must be between zero and one. A zero predictor must give $0.5$, and changing the sign of a predictor must exchange $\mu$ and $1-\mu$. A very large positive predictor should give a probability close to one; a very negative one should give a probability close to zero. These checks verify the transformation arithmetic, but they do not establish that a fitted model adequately represents the data.

Check the scale.
Confirm whether the supplied number is a probability $\mu$, a logit, or a linear predictor $\eta$. Do not use the probability formula on a number already on the predictor scale.

Use the matching direction.
For a probability, calculate $\log(\mu/(1-\mu))$. For a predictor, calculate $\exp(\eta)/(1+\exp(\eta))$.

Check the result.
Verify the sign against whether the probability is above or below one-half, and verify that an inverse-logit result lies in $(0,1)$.

A common mistake is to treat the logit as $\log(\mu)$, or to calculate odds as $(1-\mu)/\mu$. The first expression omits the probability of failure; the second is the odds of failure rather than success. For $\mu=0.20$, for example, $\log(0.20)\approx-1.609$, whereas the success logit is $\log(0.20/0.80)\approx-1.386$. Another mistake is to report a predictor such as $1.2$ as a probability of $120\%$. A predictor is not a probability; apply the inverse logit to obtain the response-scale mean.

For extremely large absolute predictor values, directly evaluating $\exp(\eta)$ can overflow in finite-precision software. The equivalent expression $1/(1+\exp(-\eta))$ is often stable for nonnegative $\eta$; for negative $\eta$, computing $\exp(\eta)/(1+\exp(\eta))$ avoids exponentiating a large positive number. This is a numerical issue, not a different definition of the inverse link.

How the Logit Compares With Other Link Choices

The identity link, covered in the earlier material on Gaussian means, uses $\eta=\mu$. It is direct to interpret on the mean scale, but an unrestricted linear predictor can fall outside the probability interval when used for Bernoulli means. The logit instead transforms the bounded mean before equating it to the predictor, and its inverse guarantees a value in $(0,1)$ for any finite predictor. Other links for binary outcomes also connect a real-valued predictor with a probability, but use different transformations. Their detailed properties and comparisons are separate topics.

The logit's main practical limitation is also its defining boundary behavior: probabilities exactly equal to zero or one have no finite logit. Moreover, a valid probability from the inverse link is not by itself evidence that the model's assumptions or its relationship to the data are adequate. The transformation guarantees the range of a prediction; it does not guarantee that the prediction is well calibrated or substantively appropriate.

Key takeaway: The logit is $\log(\mu/(1-\mu))$, and the inverse logit is $\exp(\eta)/(1+\exp(\eta))$. Keep probability-scale quantities and predictor-scale quantities distinct, and check that each calculation uses the correct direction.

Check Your Understanding

For each question, try the calculation before reading its solution.

Find the logit of a hypothetical success probability $\mu=0.75$. What does the sign tell you?
A model gives a predictor $\eta=0$. What is the inverse-logit probability?
For a hypothetical observation, $\eta=-1.00$. Calculate its model-implied probability to three significant figures.
Why is the logit undefined at $\mu=0$ and $\mu=1$, while the inverse logit of any finite predictor is valid?
A student calculates $\log(0.30)$ as the logit of a probability $0.30$. Identify the error and give the correct logit.

Solutions. (1) $\operatorname{logit}(0.75)=\log(0.75/0.25)=\log(3)\approx1.099$. It is positive because the success probability exceeds one-half. (2) $\exp(0)/(1+\exp(0))=1/2=0.5$. (3) $\mu=\exp(-1)/(1+\exp(-1))\approx0.3679/1.3679\approx0.269$. (4) At zero the odds are zero, whose logarithm is not finite; at one the odds have a zero denominator. For finite $\eta$, $\exp(\eta)>0$, so $\exp(\eta)/(1+\exp(\eta))$ is strictly between zero and one. (5) The logit uses success odds, including the failure probability in its denominator: $\log(0.30/0.70)=\log(3/7)\approx-0.8473$, not $\log(0.30)$.