Why Screen a Numeric Predictor Against a Binary Outcome?
In a binary-outcome problem, the response records whether an event occurred: for example, whether a hypothetical policyholder filed a claim, or whether a hypothetical borrower defaulted. A univariate screen examines one predictor at a time to assess whether its observed values are associated with the two outcome groups. The result can help prioritize predictors for further modeling, but it does not establish that changing the predictor would change the outcome.
For a numeric predictor \(X\) and binary outcome \(Y\), two useful summaries are the difference in average \(X\) between the event and non-event groups, and the slope from a logistic regression using \(X\) as its sole predictor. The first is a group comparison; the second describes how the modeled event odds vary with \(X\). They answer related but different questions, and neither automatically measures a predictor’s unique contribution when other predictors are considered.
Group Comparisons as a First Screen
The mean difference is direct and keeps the predictor in its original units. If the predictor is annual mileage, for example, a difference of 2,000 means that the event group averages 2,000 more miles than the non-event group in the sample. Always state which outcome group is subtracted from which; reversing the order reverses the sign.
A raw difference can be difficult to compare across predictors measured on different scales. One descriptive option is the standardized mean difference. For two groups with sample sizes \(n_1,n_0\) and sample standard deviations \(s_1,s_0\), define the pooled within-group standard deviation as \(s_p\), and standardize the difference by dividing by \(s_p\). This describes the difference in pooled standard deviation units. It is a scale-free summary, not a test of significance or a guarantee of predictive value.
Worked Example: Comparing Average Predictor Values
A hypothetical insurer screens annual service calls as a predictor of whether a household files a specified type of claim. Among 40 households without the claim, the mean number of calls is 12 and the sample standard deviation is 4. Among 20 households with the claim, the corresponding mean is 15 and the sample standard deviation is 3. Calculate the mean difference and standardized mean difference, taking the event group minus the non-event group.
Method. First calculate the raw group difference. Then use the pooled within-group standard deviation to express that difference in standard deviation units. This comparison avoids treating a difference of three calls as automatically large or small without considering the variation in calls.
Step 1: Mean difference. The event-group mean is 15 and the non-event-group mean is 12, so \(15-12=3\) calls. The event group has three more calls per household on average in this sample.
Step 2: Pooled variance. The pooled variance is \(\big((20-1)3^2+(40-1)4^2\big)/(20+40-2)\). The numerator is \(19(9)+39(16)=171+624=795\), and the denominator is \(58\). Thus the pooled variance is \(795/58=13.7069\), to four decimal places.
Step 3: Pooled standard deviation and standardized difference. The pooled standard deviation is \(\sqrt{13.7069}=3.7023\), to four decimal places. Therefore, \(d=3/3.7023=0.8103\), to four decimal places.
Conclusion and check. The event-group mean is higher by 3 calls, or about 0.8103 pooled standard deviations. The sign is positive because the calculation used event minus non-event. A common mistake is to report the difference as \(-3\) by silently reversing the group order; the stated order confirms that \(+3\) is correct. This is a descriptive screen, not evidence that additional calls cause claims.
Simple Logistic Regression as a Marginal Screen
A simple logistic regression models the probability of the event as a function of one numeric predictor. Write \(p(x)=P(Y=1\mid X=x)\). The logit of that probability is the logarithm of the event odds, and the simple logistic model assumes that this log-odds quantity is linear in \(x\):
Here, \(\beta_0\) is the modeled log odds when \(x=0\), and \(\beta_1\) is the change in modeled log odds for a one-unit increase in \(x\). Exponentiating the slope gives the odds ratio for a one-unit increase: \(\exp(\beta_1)\). If \(\beta_1\) is positive, the modeled event odds increase with \(x\); if it is negative, they decrease. An odds ratio is a ratio of odds, not a change in probability.
The model treats the responses as Bernoulli outcomes conditional on the predictor: each \(Y_i\) is either 0 or 1, with event probability \(p(x_i)\). For the usual likelihood-based interpretation, observations are treated as independent and the log odds are assumed to follow the stated linear form. The fitted slope is a marginal, one-predictor association: it does not adjust for other predictors. If the log-odds relationship is not linear, this simple model may be an inadequate summary.
Worked Example: Obtaining a Logistic Slope From Two Predictor Values
In a hypothetical screening data set, 40 observations have \(X=2\), of which 10 have the event. Another 40 observations have \(X=5\), of which 20 have the event. Fit the simple logistic relationship represented by these two observed event proportions, and calculate its slope and odds ratio per one-unit increase in \(X\).
Method. With only two distinct predictor values, calculate the event proportion and log odds at each value. The line through these two log-odds points gives the logistic slope. This calculation avoids confusing a change in event probability with a change in log odds.
Step 1: Event proportions. At \(X=2\), the event proportion is \(10/40=0.25\). At \(X=5\), it is \(20/40=0.50\).
Step 2: Log odds. At \(X=2\), the odds are \(0.25/(1-0.25)=0.25/0.75=1/3\), so the log odds are \(\log(1/3)=-\log(3)\). At \(X=5\), the odds are \(0.50/0.50=1\), so the log odds are \(\log(1)=0\).
Step 3: Slope and intercept. The log odds increase by \(\log(3)\) over three predictor units. Thus \(\beta_1=\log(3)/(5-2)=\log(3)/3=0.3662\), to four decimal places. Substituting \(X=2\) gives \(\beta_0=-\log(3)-2\log(3)/3=-(5/3)\log(3)=-1.8310\), to four decimal places.
Step 4: Odds ratio and check. The odds ratio per unit is \(\exp(\beta_1)=\exp(\log(3)/3)=3^{1/3}=1.4422\), to four decimal places. Across three units, the odds ratio is \(1.4422^3\), approximately 3, matching the observed odds change from \(1/3\) to 1. The fitted model reproduces both proportions: at \(X=2\), its log odds are \(-\log(3)\), giving probability 0.25; at \(X=5\), its log odds are 0, giving probability 0.50. The estimated association is positive, but this two-point fit alone does not show that increasing \(X\) causes the event.
From a Slope to Predictions and Interpretation
A slope should be interpreted in the predictor’s stated units. When one unit is too small to be meaningful, a change of \(c\) units has odds ratio \(\exp(c\beta_1)\). Predicted probabilities require both the intercept and slope: for a specified \(x\), calculate the linear predictor \(\eta=\beta_0+\beta_1x\), then convert log odds to probability using \(p(x)=1/(1+\exp(-\eta))\).
Worked Example: Interpreting a Ten-Unit Change
A hypothetical model for an event has fitted equation \(\log(p(x)/(1-p(x)))=-3+0.05x\), where \(X\) is a numeric score recorded in points. Calculate the modeled event probabilities at scores 20 and 30, and the odds ratio for a ten-point increase.
Method. Substitute each score into the linear predictor, convert each result to probability, and exponentiate the slope multiplied by the ten-unit change. This method avoids reading the one-point slope as a probability increase.
Step 1: Score 20. The linear predictor is \(-3+0.05(20)=-3+1=-2\). The probability is \(1/(1+\exp(2))=0.1192\), to four decimal places.
Step 2: Score 30. The linear predictor is \(-3+0.05(30)=-3+1.5=-1.5\). The probability is \(1/(1+\exp(1.5))=0.1824\), to four decimal places.
Step 3: Ten-point odds ratio. The one-point slope is 0.05, so a ten-point increase has odds ratio \(\exp(10(0.05))=\exp(0.5)=1.6487\), to four decimal places. The modeled odds at score 30 are therefore 1.6487 times those at score 20.
Conclusion and check. The modeled event probability rises from 0.1192 to 0.1824, while the odds rise by a factor of 1.6487. The probability difference is 0.0632, not a 64.87 percentage-point increase. The odds-ratio calculation is consistent with the model’s ten-unit change; the probability change is not equal to the odds ratio minus one.
Reading a Screen With Appropriate Caution
A univariate screen is useful for describing whether a predictor carries a visible marginal signal and for comparing candidates on a consistent basis. It is not a complete model-building decision. A predictor can have a small group mean difference but still relate to the event in another way; conversely, a large difference in this sample need not persist in new data. A single logistic slope summarizes a linear relationship in log odds and can miss patterns that do not follow that form.
Also, a strong marginal association need not remain strong after other predictors are included. Predictors can overlap in the information they carry, and a predictor associated with the event may be acting as a proxy for another characteristic. For these reasons, interpret the screen as evidence about the observed predictor-outcome relationship, not as a causal effect or a final judgment about a predictor’s usefulness in a multivariable model.
- Check outcome coding. If \(Y=1\) is changed from the event to the non-event, the direction of the mean difference and logistic slope reverses.
- State the units. A slope per dollar, year, or point can sound very different. Rescaling \(X\) changes the numerical slope but not the underlying fitted relationship.
- Do not equate odds and probability. An odds ratio of 1.5 means 50% higher odds, not 50 percentage points higher probability.
- Do not infer causation. A univariate association may reflect confounding, selection, or other features of the observed data.
- Do not rank by a single number without context. Consider the predictor’s scale, distribution, data quality, and the purpose of the screen.
Check Your Understanding
For each problem, identify the relevant group comparison or logistic summary and interpret it in context.
- A hypothetical claims study reports a mean predictor value of 8.4 for observations with the event and 7.1 for observations without it. What is the event-minus-non-event mean difference, and what does its sign indicate? (Answer: 1.3 predictor units; the event group has the higher sample mean.)
- A numeric screening variable has event-group mean 18, standard deviation 2, and sample size 25; the non-event group has mean 15, standard deviation 4, and sample size 75. Calculate the pooled standardized mean difference, rounding to four decimals. (Answer: \(s_p=\sqrt{1280/98}=3.6140\), so \(d=3/3.6140=0.8301\).)
- A simple logistic model has slope \(0.12\) per one-unit increase in \(X\). What is the odds ratio for a five-unit increase, rounded to four decimals? (Answer: \(\exp(5(0.12))=\exp(0.60)=1.8221\).)
- A fitted model is \(\log(p(x)/(1-p(x)))=-2.2+0.04x\). Calculate the modeled probability at \(x=10\) and the odds ratio for a ten-unit increase. (Answer: \(\eta=-1.8\), so \(p=1/(1+\exp(1.8))=0.1419\); the ten-unit odds ratio is \(\exp(0.4)=1.4918\).)
- In a hypothetical data set, the event proportions are 0.10 at \(X=1\) and 0.40 at \(X=4\). For the logistic line through those two log-odds points, calculate the slope per unit and its odds ratio per unit, rounding to four decimals. (Answer: \(\beta_1=(\log(2/3)-\log(1/9))/3=\log(6)/3=0.5973\); \(\exp(\beta_1)=6^{1/3}=1.8171\).)