From Logistic Probabilities to Class Labels
A logistic regression model estimates a probability for each combination of predictor values. A classification rule takes that probability one step further by assigning a label, such as 0 or 1. The distinction matters: a fitted probability is a model estimate, while the label depends on a threshold chosen for the classification task.
For example, a hypothetical insurer might use a logistic model to estimate whether a policy will lapse during the coming year. The model can return an estimated lapse probability of 0.62. That probability alone does not say whether the policy should be labelled “likely to lapse.” The answer depends on the rule used to turn probabilities into labels.
The rule does not change the fitted probability. It only converts the probability into a label. For instance, if \(\hat p(x)=0.62\), the rule assigns class 1 at threshold \(c=0.60\), but class 0 at \(c=0.70\). A different threshold can therefore produce a different label for the same observation even when the fitted model is unchanged.
The Logistic Model and the Threshold on the Log-Odds Scale
For a logistic regression model, the fitted probability has the form \(\hat p(x)=\exp(\hat\eta(x))/(1+\exp(\hat\eta(x)))\), where \(\hat\eta(x)\) is the fitted linear predictor. With predictors \(x_1,\ldots,x_k\), the usual form is \(\hat\eta(x)=\hat\beta_0+\hat\beta_1x_1+\cdots+\hat\beta_kx_k\). The model treats the response, conditional on the predictors, as Bernoulli with probability \(p(x)=P(Y=1\mid X=x)\); logistic regression specifies that the logit of this probability is linear in the predictors. The classification rule is applied to the fitted probability, not to an observed response.
This equivalence is useful because the fitted linear predictor is often easier to work with than the probability formula. It also shows directly how a threshold changes the decision boundary: the coefficients determine how \(\hat\eta(x)\) varies across predictor values, while \(c\) determines the level of \(\hat\eta(x)\) that separates the two labels.
Substitute the observation’s predictor values into \(\hat\eta(x)\).
Either calculate \(\hat p(x)\) and compare it with \(c\), or compare \(\hat\eta(x)\) with \(\log(c/(1-c))\).
With the convention here, equality to the threshold gives class 1.
Worked Examples
Worked Example: Classifying an Individual Policy
Method. Calculate the fitted probability and apply the stated threshold. A hypothetical insurer models whether a policy will lapse within one year using \(Y=1\) for a lapse and \(Y=0\) otherwise. Suppose the fitted model is \(\hat\eta(x)=-1.2+0.6x\), where \(x\) is a policyholder’s recorded engagement score. For a policyholder with \(x=4\), classify the policy using threshold \(c=0.70\). The score is measured in the units used to fit the model; no interest or mortality assumptions are involved.
Step 1: Calculate the linear predictor. Substituting \(x=4\) gives \(\hat\eta(4)=-1.2+0.6(4)=-1.2+2.4=1.2\).
Step 2: Calculate the fitted probability. Thus \(\hat p(4)=\exp(1.2)/(1+\exp(1.2))\). Using \(\exp(1.2)\approx3.3201\), \(\hat p(4)\approx3.3201/4.3201\approx0.7685\).
Step 3: Apply the threshold. Since \(0.7685\ge0.70\), the policy is assigned class 1, meaning the model-based rule labels it as a lapse.
Check and common mistake avoided. The equivalent linear-predictor cutoff is \(\log(0.70/0.30)=\log(7/3)\approx0.8473\). Since \(1.2\ge0.8473\), this gives the same label. A common mistake is to call the fitted probability itself the classification rule; the label requires both the probability and the specified threshold.
Worked Example: One Model, Two Thresholds
Method. Hold the fitted coefficients fixed and apply two different thresholds. In a hypothetical study of whether a customer renews a service contract, suppose \(Y=1\) denotes renewal and the fitted model is \(\hat\eta(x)=-1+0.5x\), where \(x\) is a customer activity score. Compare classifications at \(c=0.50\) and \(c=0.80\) for scores \(x=1,2,3,4,5\).
Step 1: Calculate the predictors and probabilities. For \(x=1,2,3,4,5\), the linear predictors are respectively \(-0.5,0,0.5,1,1.5\). Applying the logistic function gives fitted probabilities approximately \(0.3775,0.5000,0.6225,0.7311,0.8176\), respectively.
Step 2: Classify at \(c=0.50\). The cutoff on the linear-predictor scale is \(\log(0.50/0.50)=0\). Therefore class 1 is assigned when \(-1+0.5x\ge0\), or \(x\ge2\). Among the listed scores, \(x=2,3,4,5\) receive class 1; \(x=1\) receives class 0.
Step 3: Classify at \(c=0.80\). The linear-predictor cutoff is \(\log(0.80/0.20)=\log(4)\approx1.3863\). The rule becomes \(-1+0.5x\ge1.3863\), so \(0.5x\ge2.3863\), and \(x\ge4.7726\). Of the listed scores, only \(x=5\) receives class 1.
Check and common mistake avoided. At \(x=4\), the probability is about \(0.7311\), so it is class 1 under the 0.50 threshold but class 0 under the 0.80 threshold. The coefficients and fitted probabilities have not changed; only the threshold has. It would be incorrect to claim that the fitted coefficients alone determine the labels.
Worked Example: Finding the First Integer Score Above a Threshold
Method. Convert the probability threshold to a log-odds cutoff, solve the inequality, and then respect the predictor’s integer scale. A hypothetical model estimates whether a device needs service within 30 days, with \(Y=1\) denoting service. Let \(x\) be the number of warning signals recorded, an integer, and suppose \(\hat\eta(x)=-2+0.8x\). Find the smallest integer score classified as class 1 at threshold \(c=0.60\).
Step 1: Convert the threshold. The log-odds cutoff is \(\log(0.60/0.40)=\log(1.5)\approx0.405465\).
Step 2: Solve the inequality. Class 1 requires \(-2+0.8x\ge0.405465\). Adding 2 to both sides gives \(0.8x\ge2.405465\). Dividing by 0.8 gives \(x\ge2.405465/0.8\approx3.006831\).
Step 3: Apply the integer restriction. Since \(x\) must be an integer and must be at least \(3.006831\), the smallest qualifying value is \(x=4\).
Step 4: Check the adjacent integer values. At \(x=3\), \(\hat\eta=-2+0.8(3)=0.4\), so \(\hat p=\exp(0.4)/(1+\exp(0.4))\approx0.5987\), which is below 0.60. At \(x=4\), \(\hat\eta=-2+0.8(4)=1.2\), so \(\hat p\approx0.7685\), which is at least 0.60. Thus 4 is indeed the smallest integer score classified as class 1.
Common mistake avoided. Do not round the continuous cutoff to the nearest integer. The requirement is \(x\ge3.006831\), so the correct integer operation is to take the next integer above it, not to round it down to 3. Also, \(2.405465/0.8\approx3.006831\), not \(3.0081\).
Why the Threshold Is Not Estimated by the Logistic Coefficients
The fitted coefficients describe how the log-odds of \(Y=1\) vary with the predictors. In the usual logistic model, coefficient estimation uses the observed binary responses and predictor values to fit the conditional probabilities under the model. Once coefficients have been fitted, they produce a probability for each predictor combination. The threshold is a separate part of a later classification rule.
To see why a threshold cannot be read uniquely from the coefficients, consider the same fitted probability \(\hat p=0.62\). It is assigned class 1 for any threshold \(c\le0.62\), and class 0 for any threshold \(c>0.62\). The coefficient estimates determine the value 0.62 for that case, but they do not select one particular \(c\). More generally, any \(c\) strictly between 0 and 1 defines a mathematical threshold rule; the application must specify which rule is to be used.
The monotonicity argument behind the log-odds conversion is short but important. For \(0<c<1\), start with the threshold condition and use the fact that the logistic function is strictly increasing:
The steps are valid because \(1-c>0\), and the logarithm is increasing on positive values. The last inequality is therefore equivalent to the original probability rule, not an approximation. If the fitted linear predictor has one predictor, \(\hat\eta(x)=\hat\beta_0+\hat\beta_1x\), the boundary can be written as a predictor cutoff when \(\hat\beta_1\ne0\). If \(\hat\beta_1>0\), then \(x\ge\{\log(c/(1-c))-\hat\beta_0\}/\hat\beta_1\); if \(\hat\beta_1<0\), division reverses the inequality. If \(\hat\beta_1=0\), the probability does not vary with \(x\), so there is no predictor cutoff of this form.
Interpretation and Common Traps
A class label is a threshold-dependent summary of a fitted probability, not a statement of certainty about the outcome. A policy assigned class 1 can still have \(Y=0\), and a policy assigned class 0 can still have \(Y=1\). Likewise, two observations with different probabilities can receive the same label if they fall on the same side of the threshold.
Keep the direction of the event consistent throughout the calculation. If \(Y=1\) means “lapse,” then the model probability and class 1 refer to lapse. If the event coded as 1 changes, the probability being thresholded changes too. Do not silently treat \(P(Y=1\mid X=x)\) as the probability of class 0; because \(Y\) is binary, the latter is \(1-\hat p(x)\).
- Confusing probability with label. A fitted probability such as 0.62 is not itself class 0 or class 1 until a threshold is specified.
- Assuming the threshold is always 0.50. A threshold of 0.50 is one possible rule, not a value implied by logistic regression.
- Rounding too early. Retain the threshold and intermediate cutoff at adequate precision, especially when converting a continuous boundary to an integer score.
- Ignoring the coefficient sign. When solving for a predictor cutoff, division by a negative coefficient reverses the inequality.
- Changing the event definition mid-calculation. Verify what \(Y=1\) represents before interpreting the resulting class.
Check Your Understanding
For each problem, apply the stated threshold and use the convention that equality is classified as class 1.
- A hypothetical model has \(\hat\eta=-0.4+0.8x\). For \(x=1\) and threshold \(c=0.50\), find the fitted probability to four decimals and the assigned class. (Answer: \(\hat\eta=0.4\), \(\hat p=0.5987\), class 1.)
- A logistic model has \(\hat\eta=0.3-0.5x\). At \(c=0.50\), solve for the predictor values classified as class 1. State why the inequality direction is as it is. (Answer: \(0.3-0.5x\ge0\), so \(x\le0.6\); division by the negative coefficient reverses the inequality.)
- A model has \(\hat\eta=-1+0.4x\), where \(x\) is a nonnegative integer. At threshold \(c=0.70\), find the smallest integer \(x\) classified as class 1. (Answer: cutoff \(\log(0.70/0.30)\approx0.8473\), so \(x\ge(1+0.8473)/0.4\approx4.6183\); the smallest integer is \(5\).)
- For one observation, a fitted logistic model gives \(\hat p=0.43\). State its label at thresholds 0.40 and 0.50, and explain what changed between the two classifications. (Answer: class 1 at 0.40 and class 0 at 0.50; the threshold changed, not the fitted probability.)
- A model has \(\hat\eta=-2+0.8x\), with integer-valued \(x\). At threshold \(c=0.60\), an analyst reports the smallest class-1 score as \(x=3\) after rounding the continuous cutoff \(3.0068\) to the nearest integer. Identify the error and give the correct score. (Answer: the inequality requires an integer at least \(3.0068\), so rounding to the nearest integer is invalid; the correct smallest score is \(x=4\).)