Why Make Screening a Workflow?
Predictor screening is not a single statistic. It is a sequence of decisions: identify the kinds of variables in the data, select summaries that suit those kinds, identify possible redundancy or outcome association, and decide which findings deserve review. In an actuarial analysis, a screening table can help focus later modeling work, but a large association or a strong correlation is not, by itself, a reason to delete a predictor.
A reproducible workflow makes those decisions visible. It records the data split and screening settings, retains sample sizes alongside summaries, and separates flags for investigation from final modeling decisions. Crucially, any step that uses the outcome—or learns a value from the data, such as an imputation value—must be fitted using training data only. This is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Start With Variable Types and a Split
The same summary is not appropriate for every pair of variables. For numeric predictors, an absolute Pearson correlation can identify strong linear redundancy. For categorical predictors, Cramér’s V summarizes association between their categories. For a categorical predictor and numeric variable, an \(R^2\)-type measure such as eta-squared can summarize how much of the numeric variable’s variation lies between categories. These are descriptive summaries: each answers a particular question and does not establish that one variable causes another.
For outcome association, the measure must also match the predictor and outcome types. For example, a numeric predictor and a binary outcome can be summarized by a point-biserial correlation, which is the Pearson correlation when the binary outcome is coded 0 and 1. A categorical predictor and a categorical outcome can be summarized with Cramér’s V. The screen should record its measure, the number of usable observations, and the coding or missing-value rule used.
Before any outcome-based screen, partition the observations. In R, a reproducible random split can be made with `set.seed(417); id_train <- sample.int(nrow(dat), floor(0.70 * nrow(dat))); train <- dat[id_train, , drop = FALSE]; test <- dat[-id_train, , drop = FALSE]`. The seed makes the random selection reproducible in the same R environment and with the same input ordering. Keep the outcome in `test` for later evaluation, but do not use it to choose predictors, choose transformations, or estimate preprocessing values.
For each predictor, record its name, class, missing count, and number of observed distinct values. Resolve ambiguous types deliberately; for instance, a numeric-looking code may represent categories rather than a measured quantity.
Make the split once and retain the row indices. Fit supervised screens and data-dependent operations such as imputation or scaling using training data only.
For each predictor pair and each predictor-outcome pair, record the measure, usable sample size, and value. Do not silently compare summaries based on different samples.
Apply pre-stated rules to mark unusually high redundancy, strong outcome association, or data-quality concerns. Use subject-matter knowledge and the modeling objective before deciding what to do next.
In base R, a simple inventory can be assembled with `data.frame(variable = names(train), class = vapply(train, function(x) paste(class(x), collapse = "/"), character(1)), missing = vapply(train, function(x) sum(is.na(x)), integer(1)), distinct = vapply(train, function(x) length(unique(x[!is.na(x)])), integer(1)))`. This is an inventory, not a license to infer meaning from storage class alone. In particular, character columns may need deliberate conversion to factors, while integer identifiers may need exclusion from quantitative summaries.
Choose and Interpret the Summaries
For numeric variables \(X\) and \(Z\), Pearson correlation is \(\operatorname{Corr}(X,Z)=\operatorname{Cov}(X,Z)/(\sigma_X\sigma_Z)\), when both standard deviations are positive. It ranges from \(-1\) to \(1\) and describes linear association. A small correlation does not rule out a nonlinear relationship. The earlier material on nonlinear marginal associations and proxy predictors is a reminder that a screen is a summary, not a complete account of a predictor’s possible value.
For categorical variables, Cramér’s V uses the Pearson chi-square statistic for their contingency table. If the table has \(r\) rows, \(c\) columns, and total count \(n\), let \(O_{jk}\) be the observed count and \(E_{jk}\) the expected count under independence. With positive expected counts, calculate \(\chi^2=\sum_{j=1}^{r}\sum_{k=1}^{c}(O_{jk}-E_{jk})^2/E_{jk}\), then \(V=\sqrt{\chi^2/[n\min(r-1,c-1)]}\). Its value is between 0 and 1. Sparse tables can make inferential chi-square approximations unreliable; a descriptive V does not remove that concern or justify a significance claim.
The expected count formula follows from independence: the estimated probability of row \(j\) is its row proportion, and the estimated probability of column \(k\) is its column proportion. Their product is the estimated joint proportion. Multiplying that proportion by \(n\) gives the expected cell count. The denominator in V scales the chi-square measure by the largest possible association for the table’s dimensions, so V is on a 0-to-1 scale.
For a categorical predictor \(G\) and numeric outcome \(Y\), one possible descriptive measure is eta-squared, \(\eta^2=SS_{\text{between}}/SS_{\text{total}}\), provided the total sum of squares is positive. Here \(SS_{\text{total}}=\sum_i(Y_i-\bar{Y})^2\) and \(SS_{\text{between}}=\sum_g n_g(\bar{Y}_g-\bar{Y})^2\). This measures the fraction of observed variation associated with differences between group means. It does not establish a causal effect or account for other predictors.
In R, numeric redundancy can be summarized with `cor(train[numeric_names], use = "pairwise.complete.obs")`. Because pairwise complete observations can differ from pair to pair, record the pairwise sample sizes as well as the correlations. For Cramér’s V, form a contingency table after applying a stated missing-value rule. A compact base R calculation for a two-variable table `tab` is `expected <- outer(rowSums(tab), colSums(tab)) / sum(tab); chi2 <- sum((tab - expected)^2 / expected); V <- sqrt(chi2 / (sum(tab) * min(nrow(tab) - 1, ncol(tab) - 1)))`. Drop unused levels and check that each dimension has at least two observed categories before applying this calculation.
Worked Example: A Numeric Predictor and a Binary Outcome
A hypothetical training sample contains four records. A numeric predictor \(X\) has values 1, 2, 3, and 4, and a binary outcome \(Y\) has values 0, 0, 1, and 1 in that order. Calculate the training-sample point-biserial correlation and decide whether it meets a review rule of absolute correlation at least 0.80.
Method. Use Pearson correlation with \(Y\) coded 0 and 1. The means are \(\bar{X}=(1+2+3+4)/4=2.5\) and \(\bar{Y}=(0+0+1+1)/4=0.5\). The deviations for \(X\) are \(-1.5,-0.5,0.5,1.5\), and those for \(Y\) are \(-0.5,-0.5,0.5,0.5\).
The sum of cross-products is \(0.75+0.25+0.25+0.75=2\). The sum of squared \(X\) deviations is \(2.25+0.25+0.25+2.25=5\), and the sum of squared \(Y\) deviations is \(0.25+0.25+0.25+0.25=1\). Thus the correlation is \(2/\sqrt{5\cdot1}=0.8944\), rounded to four decimal places.
Decision and check. Since \(0.8944\geq0.80\), flag this predictor-outcome association for review under the stated rule. It is a strong positive linear association in this four-record sample; it is not evidence of a causal relationship, and the small sample makes it unsuitable as a standalone basis for a modeling decision. The common mistake avoided is treating a screening threshold as an automatic predictor-retention rule.
Worked Example: Cramér’s V for Two Categorical Predictors
In a hypothetical training sample, two categorical predictors each have two observed levels. Their joint table has 6 observations in each diagonal cell and 0 in each off-diagonal cell. Calculate Cramér’s V and state what the result means as a redundancy flag.
Method. The row totals and column totals are each 6 and 6, and \(n=12\). Under independence, every expected cell count is \((6)(6)/12=3\). The four chi-square contributions are \((6-3)^2/3=3\), \((0-3)^2/3=3\), \((0-3)^2/3=3\), and \((6-3)^2/3=3\). Therefore \(\chi^2=3+3+3+3=12\).
For a \(2\times2\) table, \(\min(r-1,c-1)=1\). Hence \(V=\sqrt{12/(12\cdot1)}=\sqrt{1}=1.0000\), to four decimal places. The observed categories match perfectly in this sample, so the pair should be flagged as potentially redundant. The calculation is descriptive; with only 12 observations and zero off-diagonal counts, do not interpret it as a reliable inferential test. The common mistake avoided is using an incorrect cell count or omitting one or more of the four cell contributions.
Make Flags Reproducible, Not Automatic
A workflow needs explicit rules for its flags. For example, an analysis might flag an absolute numeric correlation of at least 0.90, a Cramér’s V of at least 0.80, or a predictor with more than 20% missing values. These are illustrative review thresholds, not universal standards. A threshold should be chosen for the analysis and recorded before interpreting a large collection of results. Multiple-testing concerns and screen stability also matter; a high-ranked association can be sample-specific.
A useful review table contains the two variable names, their classes, the summary measure, its value, the usable sample size, the threshold, and the flag decision. For an outcome screen, include the outcome definition and coding. For redundancy, remember that a high pairwise association does not prove that either predictor is unnecessary: each may carry distinct conditional information, and subject-matter considerations can matter.
For a factor predictor and numeric outcome, eta-squared can be calculated by comparing the total variation with the variation explained by group means. For example, if a two-group training sample has group sizes 3 and 3, group means 2 and 6, and overall mean 4, then \(SS_{\text{between}}=3(2-4)^2+3(6-4)^2=24\). To complete the calculation, one would also need the individual outcomes to compute \(SS_{\text{total}}\); group means alone do not determine eta-squared. This illustrates why a workflow should retain the underlying data summaries needed to verify a measure, not just its final value.
A reproducible analysis also records the random seed, split indices, variable lists, missing-data rule, thresholds, and software environment. Preserve the screening table rather than only the selected names. If a decision is revised after review, record the reason. This makes it possible to distinguish a change in the data from a change in the analysis rule.
Worked Example: Keep Outcome Screening Inside Training Data
A hypothetical dataset is split into six training records and two test records. For one numeric predictor \(X\), training values for outcome \(Y=0\) are 1, 2, and 3; training values for \(Y=1\) are 4, 5, and 100. The two test records have \((X,Y)=(1000,0)\) and \((-100,1)\). Calculate the training-only difference in group means, compare it with the prohibited full-data difference, and identify the correct screen.
Method. Define the contrast as mean \(X\) for \(Y=1\) minus mean \(X\) for \(Y=0\). In training, the mean for \(Y=0\) is \((1+2+3)/3=2\). The mean for \(Y=1\) is \((4+5+100)/3=109/3=36.3333\). The training-only contrast is \(109/3-2=103/3=34.3333\).
If the test outcomes are improperly used, the full-data mean for \(Y=0\) becomes \((1+2+3+1000)/4=1006/4=251.5\). The full-data mean for \(Y=1\) becomes \((4+5+100-100)/4=9/4=2.25\). The full-data contrast is \(2.25-251.5=-249.25\), which even reverses sign. The permitted screen is the training-only contrast, \(34.3333\), because the test outcomes must not influence predictor screening.
Decision and check. Report the training result with its sample sizes (three observations per outcome group) and do not use the full-data result to choose the predictor or its treatment. The test outcomes are reserved for evaluation after model development. The common mistake avoided is screening once on all labeled data and then treating a later test-set score as an independent evaluation.
Keep Preprocessing and Screening in the Training Boundary
A supervised screen uses the outcome, so its calculation and any resulting variable selection must use training data only. The same discipline applies to steps that estimate values from observations. For example, estimate a numeric imputation value from the training predictor and apply that stored value to missing training and test entries. Do not recompute a test-set median. If a transformation, category grouping, or scaling rule is learned from data, fit it on training data and apply the fitted rule to test data.
Factor levels need deliberate handling as well. A test record may contain a level not seen in training. A workflow should specify how such a level is represented, such as mapping it to a pre-defined “other” category when appropriate, rather than silently allowing the test distribution to alter a supervised selection rule. Predictor classes can be recorded before the split as a description of the raw input schema, but choices informed by observed data or the outcome must respect the training boundary.
A concise R workflow can use a saved index and named objects: `set.seed(417); id_train <- sample.int(nrow(dat), floor(0.70 * nrow(dat))); train <- dat[id_train, , drop = FALSE]; test <- dat[-id_train, , drop = FALSE]`. Then compute the inventory and summaries from `train`, save thresholds and results, and leave `test` untouched until evaluation. For correlation summaries, the use of `pairwise.complete.obs` should be paired with recorded pairwise counts; for contingency tables, record how missing values were handled. Re-running with the same input ordering and settings should reproduce the same split and review table.
Check Your Understanding
For each problem, state the measure or workflow decision and show the key calculation.
- A training sample has numeric predictor values 2, 4, 6, 8 and binary outcomes 0, 0, 1, 1. What is the point-biserial correlation, rounded to four decimals? (Answer: 0.8944)
- A \(2\times2\) table has observed counts 5 and 5 on the diagonal and 0 and 0 off the diagonal. Calculate Cramér’s V. (Answer: \(\chi^2=10\), \(V=1.0000\))
- A screening workflow imputes missing numeric values with the median. Which data should determine the median, and what is done with the test data? (Answer: Estimate the median from training data only and apply that stored value to missing test entries.)
- A numeric predictor has absolute correlation 0.93 with another predictor, above a pre-stated review threshold of 0.90. What is the immediate workflow action? (Answer: Flag the pair for review with its sample size and measure; do not automatically remove either predictor.)
- Training group means for a numeric predictor are 3 for \(Y=0\) and 9 for \(Y=1\). The test set has means 100 for \(Y=0\) and 0 for \(Y=1\). What contrast is used for supervised screening, defined as mean for \(Y=1\) minus mean for \(Y=0\)? (Answer: The training-only contrast, \(9-3=6\); test outcomes are not used for screening.)