The Logit Link and Odds-Based Probability Model

From a Linear Index to a Probability

For a binary outcome, the quantity of interest is often the conditional probability that the outcome equals one, given observed characteristics. A linear expression in those characteristics is useful for summarizing their combined contribution, but a linear expression itself can take any real value. It therefore cannot, without a transformation, serve as a probability, which must lie between zero and one. The logit model resolves this mismatch by applying a nonlinear inverse link to a linear index.

Let $Y_i$ be a binary outcome for observation $i$, taking values zero or one, and let $x_i$ be a vector of observed covariates that includes an intercept. Suppose the conditional distribution is Bernoulli with success probability $p_i=P(Y_i=1\mid x_i)$. The logit specification assumes that the log-odds of this conditional probability are linear in the covariates.

Definition: The logit link maps a probability $p$, with $0<p<1$, to its log-odds: $$ \operatorname{logit}(p)=\log\left(\frac{p}{1-p}\right). $$ A logit response model specifies $\operatorname{logit}(p_i)=\eta_i=x_i^\top\beta$, where $\eta_i$ is the linear index and $\beta$ is a vector of parameters. Equivalently, $Y_i\mid x_i$ has a Bernoulli distribution with probability $p_i$ given by the inverse-logit transformation.

Here $p_i$ is the conditional mean of the binary outcome, because $E[Y_i\mid x_i]=P(Y_i=1\mid x_i)$. The index $\eta_i$ is not itself a probability: it can be any real number. The logit link maps the probability scale to the entire real line, and the inverse link maps the real line back to probabilities.

Deriving the Logistic Response Function

Begin with the model equation $\log\left(p_i/(1-p_i)\right)=\eta_i$. Exponentiating both sides gives the odds as a function of the index. Solving that equation for $p_i$ gives the model’s probability formula.

\begin{aligned} \log\left(\frac{p_i}{1-p_i}\right)&=\eta_i\\ \frac{p_i}{1-p_i}&=\exp(\eta_i)\\ p_i&=\exp(\eta_i)(1-p_i)\\ p_i\left(1+\exp(\eta_i)\right)&=\exp(\eta_i)\\ p_i&=\frac{\exp(\eta_i)}{1+\exp(\eta_i)} =\frac{1}{1+\exp(-\eta_i)}. \end{aligned}

Thus a model with $\eta_i=x_i^\top\beta$ has response probability $p_i=1/\left(1+\exp(-x_i^\top\beta)\right)$. The expression $\exp(\eta_i)$ is the odds, $p_i/(1-p_i)$, while $p_i$ is the probability. Keeping those two quantities distinct prevents a common error in interpreting model output.

Key result: The logistic response function is $p_i=1/\left(1+\exp(-\eta_i)\right)$. For every finite real index $\eta_i$, it produces $0<p_i<1$. The model therefore respects the probability range without imposing bounds on the linear index.

Worked Probabilities and Log-Odds

The following examples use hypothetical model specifications. They demonstrate calculations implied by the stated model; they do not establish that the logit form is appropriate for any real population. In each case, the outcome is binary, the conditional distribution is stipulated to be Bernoulli, and the listed covariates and parameter values define a finite index.

Worked Example: Predicting a Probability From Study Time

Suppose a hypothetical model predicts whether a student passes an assessment, with $Y_i=1$ for a pass. Let $h_i$ be study time in hours and suppose the specified index is $\eta_i=-1.2+0.8h_i$. Calculate the predicted probability for a student who studies 2.5 hours.

The outcome is binary and the prediction is conditional on the stated study time. Under the stipulated Bernoulli-logit model, first substitute $h_i=2.5$ into the index:

\eta_i=-1.2+0.8(2.5)=-1.2+2.0=0.8.

The corresponding odds are $\exp(0.8)\approx 2.2255$. Converting those odds to a probability gives

p_i=\frac{\exp(0.8)}{1+\exp(0.8)} =\frac{2.2255}{1+2.2255} \approx 0.6900.

The model’s predicted probability of passing for a student with 2.5 study hours is approximately 0.690, or 69.0%. The index value 0.8 is not an 80% probability; it is the input to the inverse-logit transformation.

Worked Example: Converting a Probability to Its Index

A hypothetical binary-outcome model assigns a conditional probability of $p_i=0.72$ to a specified covariate profile. Find the odds and the logit index corresponding to that probability.

For a Bernoulli outcome, the odds of $Y_i=1$ rather than $Y_i=0$ are $p_i/(1-p_i)$. Substituting $p_i=0.72$:

\text{odds}=\frac{0.72}{1-0.72} =\frac{0.72}{0.28} \approx 2.571.

The event is therefore assigned about 2.571 to 1 odds relative to its complement. The corresponding log-odds, which equal the index in a logit model, are

\eta_i=\operatorname{logit}(0.72) =\log\left(\frac{0.72}{0.28}\right) =\log(2.5714) \approx 0.9445.

Checking by applying the inverse link,

\frac{1}{1+\exp(-0.9445)}\approx 0.7200.

The conditional outcome is binary, and the given probability is strictly between zero and one, so both the odds and logit are defined. This conversion gives the index value matching the stated probability; it does not by itself identify which covariate values or parameter vector generated that index.

Worked Example: Probability Limits at Large Positive and Negative Indices

Consider a hypothetical binary-outcome model with a finite index $\eta_i$ for each finite covariate profile. Calculate the response probabilities at $\eta_i=5$ and $\eta_i=-5$, and compare them with the limits.

At index 5, the odds are $\exp(5)\approx148.413$, giving

p_i=\frac{148.413}{1+148.413} =\frac{148.413}{149.413} \approx 0.9933.

At index $-5$, the odds are $\exp(-5)\approx0.006738$, and the probability is

p_i=\frac{0.006738}{1+0.006738} \approx 0.006693.

Both calculations use the stipulated Bernoulli-logit form and finite index values. As the index grows without bound, $\exp(-\eta_i)$ tends to zero, so $p_i=1/(1+\exp(-\eta_i))$ tends to one. As the index decreases without bound, $\exp(\eta_i)$ tends to zero, so $p_i=\exp(\eta_i)/(1+\exp(\eta_i))$ tends to zero. The computed probabilities are close to, but not equal to, these limits.

How the Link Shapes Probability Predictions

The logistic response function is increasing: larger index values produce larger predicted probabilities. Its midpoint is $\eta_i=0$, where $\exp(0)=1$, the odds equal one, and the probability equals one half. Positive indices correspond to probabilities above one half; negative indices correspond to probabilities below one half. These facts follow directly from the inverse-logit formula.

The probability scale compresses extreme index values toward its endpoints. An index can become very large or very negative without making the predicted probability exceed one or fall below zero. For a finite index, $\exp(\eta_i)$ is positive and finite, so $\exp(\eta_i)/(1+\exp(\eta_i))$ is strictly between zero and one. The values zero and one are limits, not attainable predictions at finite indices.

The index is linear in the included covariates and parameters, but the response probability is generally nonlinear in them. This distinction matters whenever the model is used to describe or predict outcomes: a change on the index scale is not itself a change of the same size on the probability scale. Translating changes in covariates into probability effects requires additional calculations, which are separate from defining the logit link.

Specify the conditional outcome model.
For each covariate vector $x_i$, treat $Y_i$ as Bernoulli with conditional probability $p_i=P(Y_i=1\mid x_i)$.

Form the linear index.
Calculate $\eta_i=x_i^\top\beta$, including the intercept and using the stated units and coding of the covariates.

Apply the inverse logit.
Calculate $p_i=1/(1+\exp(-\eta_i))$. Report this result as a conditional probability, not as an index or an odds value.

Why the Odds Form Is Useful—and a Common Pitfall

Odds provide an alternative scale for expressing a probability. If the probability of an event is $p$, its odds are $p/(1-p)$; if its odds are $o$, its probability is $o/(1+o)$. The logit uses the logarithm of the odds, which can take any real value and therefore can be modeled with an unrestricted linear index. In this tutorial, odds and log-odds are used to define the link and recover probabilities. They should not be confused with the probability itself.

A common mistake is to calculate an index such as $\eta_i=0.8$ and report it as a predicted probability of 0.8, or 80%. That is wrong because the model defines the probability by applying the inverse-logit transformation. For $\eta_i=0.8$, the predicted probability is $1/(1+\exp(-0.8))\approx0.690$, not 0.8. A related error is to treat $\exp(\eta_i)$ as a probability: it is the odds, and it can exceed one.

The logit specification is a model for a conditional probability, not by itself a claim that covariates cause changes in the outcome. The interpretation depends on the research design and assumptions supporting the analysis. The link guarantees that model predictions fall in the probability range; it does not guarantee that the functional form is appropriate or that a fitted model is well calibrated.

Key takeaway: In a logit response model, the linear index equals the log-odds, not the probability. The inverse-logit converts that index to a valid conditional probability, approaching zero or one only as the index tends to negative or positive infinity.

Check Your Understanding

Use the logit and inverse-logit definitions to answer each question.

If a model index is zero, what are the corresponding odds and conditional probability?
For a conditional probability of 0.25, calculate the odds and logit index.
Why can the linear index take a negative value even though a probability cannot?
Does a finite index produce a predicted probability exactly equal to zero or one? Explain using the logistic response formula.
A model gives an index of 1.5. Explain why 1.5 is not the predicted probability and write the formula needed to obtain that probability.