Why Screen Categorical Predictors by Outcome?
A categorical predictor partitions observations into levels: for example, a policy channel might be “direct,” “broker,” or “partner.” To screen it against an outcome, first compare the outcome across those levels. That comparison can reveal a potentially useful predictor, but it is only a marginal screen: it does not establish that the predictor will add value after other predictors are considered, nor that the relationship is causal.
The outcome determines the appropriate comparison. For a binary outcome, outcome-stratified counts and a contingency-table association test are natural starting points. For a numeric outcome, compare group summaries and, when its assumptions are reasonable, use an omnibus group-comparison model such as one-way analysis of variance (ANOVA). A model-based comparison can also ask whether adding the categorical predictor improves a logistic regression model.
Outcome-stratified summaries are useful even when a formal test is not. They show the direction and practical size of differences, make unusual levels visible, and help diagnose whether a test result depends on very few observations. Report the number of observations in each level alongside its outcome summary; percentages without counts can disguise sparse data.
Binary Outcomes: Counts, Expected Counts, and Association
For a categorical predictor with \(r\) levels and a binary outcome, arrange the observations in an \(r\)-by-2 contingency table. Each cell contains the number of observations with one predictor level and one outcome value. The null hypothesis for Pearson’s chi-square test is that predictor level and outcome are independent in the population.
The expected count formula follows directly from the independence assumption. If the row proportion is \(R_i/n\) and the column proportion is \(C_j/n\), independence implies that the expected joint proportion is their product. Multiplying that joint proportion by \(n\) gives the expected cell count:
The chi-square statistic adds the squared discrepancy between observed and expected counts, scaled by the expected count. Larger discrepancies therefore contribute more, while a fixed discrepancy matters more in a cell with a smaller expectation. The test is an omnibus screen: a large statistic says that some distributional difference is present, but does not by itself identify which levels differ or quantify the practical importance.
Worked Example: Screening a Three-Level Predictor Against a Binary Outcome
In a hypothetical portfolio review, analysts classify 90 applications by submission channel and whether an application required additional documentation. The counts are shown below. Test for an association and describe its strength.
| Channel | Additional documentation | No additional documentation | Total |
|---|---|---|---|
| Direct | 6 | 24 | 30 |
| Broker | 15 | 15 | 30 |
| Partner | 24 | 6 | 30 |
| Total | 45 | 45 | 90 |
Method. Use Pearson’s chi-square test of independence, then Cramér’s \(V\) as a descriptive measure of association. The row totals are all 30 and both column totals are 45. Thus, each expected count is \(30(45)/90=15\).
| Channel | Observed counts | Expected counts | Contribution to \(\chi^2\) |
|---|---|---|---|
| Direct | 6, 24 | 15, 15 | \(81/15+81/15=10.8\) |
| Broker | 15, 15 | 15, 15 | 0 |
| Partner | 24, 6 | 15, 15 | \(81/15+81/15=10.8\) |
Therefore, \(\chi^2=10.8+0+10.8=21.6000\), with \((3-1)(2-1)=2\) degrees of freedom. The p-value is approximately \(0.0000\) to four decimal places (less than \(0.0001\)), so the table gives strong evidence of an association in this hypothetical sample. Cramér’s \(V\) is
The value \(0.4899\) describes a substantial association in this table; it is not a probability or a causal effect. The observed documentation rates are \(6/30=0.20\) for Direct, \(15/30=0.50\) for Broker, and \(24/30=0.80\) for Partner. Common mistake avoided: comparing raw counts without accounting for level totals. Here the totals happen to be equal, but in general the relevant comparison is the within-level outcome rate, not the number of outcomes alone. The calculated statistic is checked by summing the unrounded cell contributions, not by rounding each contribution first.
For a \(2\)-by-\(2\) table, a test based on Pearson’s chi-square is closely related to a test of two proportions. With more than two levels, the omnibus statistic does not say which level contrasts account for the association. Follow-up comparisons need a separate plan; inspecting many contrasts and reporting only the most striking one can exaggerate the evidence.
The usual chi-square approximation also depends on having adequate expected counts. A widely used rule of thumb is that no expected count should be below 1 and at least 80% of expected counts should be at least 5. These are guidelines for the approximation, not a guarantee that a screen is useful. When counts are sparse, consider an exact or simulation-based test appropriate to the table, or describe the uncertainty without relying on the approximate p-value. Combine levels only when the combination is substantively defensible, not simply because it produces a more favorable result.
Numeric Outcomes: Compare Outcome Distributions Across Levels
When the response is numeric, begin with the sample size, mean, and spread of the outcome within each predictor level. A plot can help reveal skewness or unusual observations, but the screen should still show group counts: an apparently extreme mean from a level with very few observations may be unstable.
One-way ANOVA compares the group means under a model with independent observations, approximately normal residuals within groups, and a common residual variance across groups. Its F statistic compares variation between group means with variation within groups. If those assumptions are seriously implausible, the usual F reference distribution may not be reliable; use summaries and an alternative method suited to the outcome and design rather than treating the ANOVA result as definitive.
Worked Example: Comparing a Numeric Outcome Across Three Levels
A hypothetical analyst compares processing times, in days, for three service routes. The observed times are Route A: 10, 12, 14, 16; Route B: 14, 16, 18, 20; and Route C: 18, 20, 22, 24. Use one-way ANOVA to screen for differences in mean processing time.
Method. Each group has \(n_g=4\). The group means are \(13\), \(17\), and \(21\) days, and the overall mean is \((52+68+84)/12=204/12=17\) days.
The between-group sum of squares is \(SS_B=4[(13-17)^2+(17-17)^2+(21-17)^2]=4(16+0+16)=128\). For Route A, the deviations from its mean are \(-3,-1,1,3\), so its within-group sum of squares is \(9+1+1+9=20\). The deviations for Routes B and C have the same values, so each also contributes 20. Therefore, \(SS_W=20+20+20=60\).
The degrees of freedom are \(k-1=2\) between groups and \(n-k=12-3=9\) within groups. Hence \(MS_B=128/2=64\), \(MS_W=60/9=6.6667\), and \(F=64/(60/9)=9.6000\). For an \(F\) distribution with 2 and 9 degrees of freedom, the p-value is approximately \(0.0059\). The screen suggests differences in mean processing time across routes, subject to the stated assumptions. The fraction of observed variation associated with differences between these group means is \(\eta^2=SS_B/(SS_B+SS_W)=128/188\approx0.6809\). Common mistake avoided: using the overall mean rather than each group mean to calculate within-group variation; that would incorrectly include between-group differences in \(SS_W\).
Model-Based Comparison for a Binary Outcome
A logistic regression model gives another way to screen a categorical predictor against a binary outcome. With an intercept-only model, every observation has the same modeled event probability. Adding a categorical predictor allows the modeled probability to vary by level. For a predictor with \(k\) levels, a reference-level encoding typically adds \(k-1\) parameters; the overall comparison tests the predictor as a factor, rather than treating its level labels as a numeric trend.
For nested models fitted to the same observations, the likelihood-ratio statistic is the reduction in deviance when the larger model is fitted. Under the null hypothesis and regularity conditions, it is approximately chi-square distributed with degrees of freedom equal to the difference in the number of estimated parameters. The approximation can be poor when cells are sparse or estimates are extreme; a very small level can also produce unstable coefficient estimates.
Worked Example: Testing a Categorical Predictor in Logistic Regression
In a hypothetical review of 60 claims, an analyst records service region and whether a claim required manual review. Region North has 4 manual reviews among 20 claims, Region South has 12 among 20, and Region Central has 8 among 20. Compare an intercept-only logistic model with a model containing region as a categorical predictor.
Method. Under the intercept-only model, the fitted review probability is the overall rate, \(24/60=0.4\). Thus each group of 20 has expected counts 8 reviews and 12 claims without review. In the region model, the fitted probabilities are the observed group rates: \(4/20=0.2\), \(12/20=0.6\), and \(8/20=0.4\). This factor adds \(3-1=2\) parameters.
For grouped binary data, the likelihood-ratio statistic can be calculated as twice the sum of observed counts times the natural logarithm of observed-to-expected count ratios. Cells with observed count zero contribute zero by the limiting convention. The six contributions inside the sum are:
Therefore, \(G^2=2(3.452185)=6.904370\), or \(6.9044\) to four decimal places. With 2 degrees of freedom, the approximate p-value is \(e^{-6.904370/2}\approx0.0317\). This comparison suggests that region improves the fit relative to a constant event probability in this hypothetical sample. The model comparison does not show that region has a causal effect, nor does it adjust for other predictors. Common mistake avoided: assigning one degree of freedom because there is one named predictor. A three-level factor requires two parameters relative to an intercept, so the omnibus comparison uses two degrees of freedom.
Interpreting a Screen When Levels Are Rare
A large association statistic or a small p-value is not enough to decide that a categorical predictor is useful. Review the level counts, outcome rates or means, and the practical magnitude of differences. A rare level may show an extreme rate because only one or two observations determine it; even when the test’s approximation is technically usable, that estimate may not be dependable in new data.
- Show counts with rates. A rate of 100% based on 2 observations does not carry the same evidential weight as 100% based on 200.
- Do not impose an artificial order. Unless a meaningful order and trend are justified, a factor with labels such as “North,” “South,” and “Central” should be compared as a set of levels, not assigned arbitrary numeric codes.
- Distinguish screening from prediction and explanation. A marginal association can disappear after adjustment, and a weak marginal association can still matter in combination with other predictors.
- Use defensible level definitions. Combining rare levels can improve interpretability or stabilize summaries, but the grouping should make sense in context and should not be selected solely to strengthen the observed association.
Use a contingency table for a binary outcome, group summaries and a suitable mean comparison for a numeric outcome, or a model comparison aligned with the outcome model.
Check whether a dramatic rate or mean is based on very few observations, and report counts alongside summaries.
Use a test whose assumptions fit the data, including expected-count or model-regularity conditions where relevant.
Describe the observed association and its uncertainty without claiming causality or implying that a marginal screen proves adjusted predictive value.
Check Your Understanding
Work through these original screening problems, checking both the calculation and whether the conclusion is supported by the available counts.
- A hypothetical two-level predictor has outcome counts A: 8 events and 12 non-events; B: 12 events and 8 non-events. Find the expected counts, Pearson’s chi-square statistic, and Cramér’s \(V\). (Answer: Expected counts are 10 in each cell; \(\chi^2=1.6000\) with 1 degree of freedom; \(V=0.2000\).)
- A numeric outcome has three groups, each with three observations: Group A: 3, 5, 7; Group B: 5, 7, 9; Group C: 7, 9, 11. Calculate \(SS_B\), \(SS_W\), and the ANOVA F statistic. (Answer: \(SS_B=24\), \(SS_W=24\), and \(F=3.0000\) on 2 and 6 degrees of freedom.)
- A binary outcome table has a rare level with 2 observations, 1 event and 1 non-event; the other levels have 38 and 60 observations. The overall event total is 50 out of 100. What are the rare level’s expected counts under independence, and what should you conclude about a routine chi-square approximation? (Answer: Expected counts are 1 event and 1 non-event; both are below 5, so the usual chi-square approximation is questionable and the rare level’s rate is unstable.)
- In a hypothetical logistic model comparison, adding a two-parameter categorical predictor reduces deviance from 84.0 to 77.2. Find the likelihood-ratio statistic and degrees of freedom, and state the approximate p-value for a chi-square reference distribution. (Answer: \(G^2=6.8\), 2 degrees of freedom, approximate \(p=0.0334\).)
- A four-level categorical predictor has group event rates of 0.10, 0.12, 0.11, and 0.60, based on level counts of 200, 180, 190, and 5, respectively. Which rate deserves the strongest stability warning, and why does a significant omnibus test not resolve that concern? (Answer: The 0.60 rate based on 5 observations deserves the strongest warning; an omnibus association test does not make the rare-level estimate stable or establish that the difference will persist in new data.)