Why Screen Predictors One at a Time?
A data set may contain many candidate predictors, while the outcome of interest is a numeric quantity such as claim cost, account balance, or time to resolution. A univariate screen summarizes the relationship between the outcome and each predictor considered separately. It can help prioritize variables for closer analysis, but it is not a substitute for fitting and evaluating a model that uses predictors together.
For a numeric predictor, Pearson correlation and the slope from a simple linear regression are common screening summaries. They describe linear marginal association: the observed relationship between one predictor and the outcome without adjusting for other predictors. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Correlation and Simple-Regression Summaries
For \(n\) observations with values \(x_i\) and \(y_i\), define their sample means as \(\bar{x}\) and \(\bar{y}\). The centered sums of squares and cross-products are \(SS_X=\sum_{i=1}^n(x_i-\bar{x})^2\), \(SS_Y=\sum_{i=1}^n(y_i-\bar{y})^2\), and \(SP_{XY}=\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})\). If both variables have nonzero sample variation, their sample Pearson correlation is:
The correlation lies between \(-1\) and \(1\). Its sign gives the direction of the linear association, while its absolute value measures the strength of that association in the sample. It has no measurement units, so it can be compared across numeric predictors measured on different scales.
A separate ordinary least squares regression of \(Y\) on \(X\), with an intercept, has fitted equation \(\hat{y}=b_0+b_1x\). Provided \(SS_X>0\), its slope and intercept are:
The slope describes the fitted change in the outcome for a one-unit increase in the predictor. Unlike correlation, it depends on the units used for both variables. The coefficient of determination for this simple regression, \(R^2\), is the proportion of the sample variation in \(Y\) explained by the fitted linear relationship with \(X\).
Worked Example: Ranking Three Numeric Predictors
A hypothetical portfolio team has five records. The numeric outcome \(Y\) is a service-cost index, and \(X_A\), \(X_B\), and \(X_C\) are three numeric candidate predictors. Rank the predictors by the absolute value of their marginal Pearson correlation with \(Y\), and calculate their simple-regression slopes.
| Record | Y | X_A | X_B | X_C |
|---|---|---|---|---|
| 1 | 2 | 1 | 1 | 2 |
| 2 | 4 | 2 | 1 | 1 |
| 3 | 5 | 3 | 3 | 4 |
| 4 | 4 | 4 | 5 | 3 |
| 5 | 5 | 5 | 5 | 5 |
Step 1: Center the outcome. Its mean is \(\bar{y}=20/5=4\), so its deviations are \((-2,0,1,0,1)\) and \(SS_Y=4+0+1+0+1=6\).
Step 2: Calculate the summaries for each predictor. Each predictor has mean 3. For \(X_A\), the deviations are \((-2,-1,0,1,2)\), giving \(SS_{X_A}=10\) and \(SP_{X_AY}=4+0+0+0+2=6\). For \(X_B\), the deviations are \((-2,-2,0,2,2)\), giving \(SS_{X_B}=16\) and \(SP_{X_BY}=4+0+0+0+2=6\). For \(X_C\), the deviations are \((-1,-2,1,0,2)\), giving \(SS_{X_C}=10\) and \(SP_{X_CY}=2+0+1+0+2=5\).
Step 3: Calculate correlations and slopes. Using \(r=SP/\sqrt{SS_XSS_Y}\) and \(b_1=SP/SS_X\):
- \(X_A\): \(r=6/\sqrt{10(6)}=0.7746\), and \(b_1=6/10=0.6000\).
- \(X_B\): \(r=6/\sqrt{16(6)}=0.6124\), and \(b_1=6/16=0.3750\).
- \(X_C\): \(r=5/\sqrt{10(6)}=0.6455\), and \(b_1=5/10=0.5000\).
Conclusion and check. By absolute correlation, the ranking is \(X_A\), \(X_C\), then \(X_B\). The corresponding simple-regression \(R^2\) values are \(0.7746^2=0.6000\), \(0.6455^2\approx0.4167\), and \(0.6124^2=0.3750\). These agree with \(SP^2/(SS_XSS_Y)\) for each predictor. The slopes are in outcome-index units per predictor unit; comparing them directly would not be the same as ranking scale-free association.
Method and mistake avoided. This is a separate correlation and simple-regression screen for each predictor. It avoids treating the largest raw slope as automatically the strongest association when predictor scales differ.
Why the Correlation and Slope Are Linked
The relationship between correlation and slope follows directly from their definitions. Assume the predictor and outcome have nonzero sample variation, and fit an ordinary least squares line with an intercept to the same complete set of observations. From the definitions:
The square-root factor rescales the dimensionless correlation into outcome units per predictor unit. Thus, a positive correlation gives a positive slope and a negative correlation gives a negative slope, but the slope magnitude also reflects the variables’ scales.
For the \(R^2\) identity, the fitted values in a simple regression with an intercept are \(\hat{y}_i=\bar{y}+b_1(x_i-\bar{x})\). Their centered sum of squares is therefore \(\sum(\hat{y}_i-\bar{y})^2=b_1^2SS_X\). Substituting \(b_1=SP_{XY}/SS_X\) gives \(SP_{XY}^2/SS_X\). Dividing by \(SS_Y\), the total centered sum of squares of the outcome, gives \(SP_{XY}^2/(SS_XSS_Y)=r_{XY}^2\). The identity relies on the simple fitted line including an intercept; it is not a general formula for a regression with multiple predictors or a model without an intercept.
Worked Example: A Weak Marginal Screen Can Miss Joint Value
In a hypothetical four-record process study, two numeric predictors are \(X_1=(5,-5,5,-5)\) and \(X_2=(6,-4,4,-6)\). The numeric outcome is \(Y=(-1,-1,1,1)\). Determine the marginal correlations, then show what happens when both predictors are used in a linear model with an intercept.
Step 1: Check the means and sums of squares. Each column has mean zero. Thus \(SS_{X_1}=25+25+25+25=100\), \(SS_{X_2}=36+16+16+36=104\), and \(SS_Y=1+1+1+1=4\).
Step 2: Calculate the cross-products. The cross-product for \(X_1\) and \(Y\) is \((-5)+5+5+(-5)=0\). The cross-product for \(X_2\) and \(Y\) is \((-6)+4+4+(-6)=-4\). Therefore \(r_{X_1Y}=0/\sqrt{100(4)}=0\), while \(r_{X_2Y}=-4/\sqrt{104(4)}\approx-0.1961\). On a marginal correlation screen, \(X_1\) appears unrelated to the outcome and \(X_2\) has only a weak negative linear association.
Step 3: Fit the two-predictor relationship. Row by row, \(X_1-X_2=(-1,-1,1,1)=Y\). Consequently, the model with intercept zero and coefficients 1 for \(X_1\) and \(-1\) for \(X_2\) reproduces all four outcomes exactly: \(\hat{y}=X_1-X_2\).
Step 4: Check predictor redundancy. Their cross-product is \(5(6)+(-5)(-4)+5(4)+(-5)(-6)=100\), so their correlation is \(100/\sqrt{100(104)}\approx0.9806\). They are highly correlated with one another, even though neither marginal association with the outcome is strong.
Conclusion and check. Both predictors contribute to the exact joint relationship in this constructed data, but marginal screening does not reveal that contribution. The predictor-predictor correlation is consistent with substantial redundancy. This example shows why marginal rank is a prioritization summary, not a test of conditional usefulness.
Method and mistake avoided. The method compares marginal correlations first and then checks the joint linear identity directly. It avoids concluding that a near-zero marginal correlation proves a predictor has no value when other predictors are present.
Interpreting a Ranking in Practice
A ranking depends on the chosen summary and the purpose of the screen. If the goal is to rank the strength of linear marginal association across numeric predictors, absolute correlation is a natural scale-free choice. Retain the sign as well: it communicates whether larger predictor values tend to accompany larger or smaller outcome values. If the goal is to interpret the fitted change per unit of a predictor, the simple-regression slope is relevant, but its magnitude is not directly comparable across differently scaled predictors.
With the same observations and a simple regression containing an intercept, ranking by absolute correlation and by \(R^2\) gives the same order because \(R^2=r^2\). This does not mean the summaries answer every question about predictive performance. They do not account for other predictors, establish causation, or describe a relationship that is not well summarized by a straight line.
When using R, calculate each summary on a consistent set of observations. For example, after defining a data frame with one numeric outcome and numeric candidate predictors, a call such as `cor(dat$X_A, dat$Y)` gives their Pearson correlation when the paired values are complete. A separate call to `lm(Y ~ X_A, data = dat)` fits the simple regression with an intercept; `coef` returns its intercept and slope, and `summary` reports \(R^2\). Repeat for each candidate predictor. If different predictors use different subsets because of missing values, their reported associations may not be directly comparable. Select and document a consistent analysis set rather than treating a change in sample as a change in predictor quality.
Worked Example: Why Predictor Units Change the Slope
A hypothetical analyst compares two numeric predictors of a numeric outcome \(Y\), measured in thousands of dollars. Both simple regressions include an intercept and use the same observations. The sample standard deviation of \(Y\) is 4. Predictor \(X_A\), measured in single units, has sample standard deviation 2 and correlation \(0.60\) with \(Y\). Predictor \(X_B\), measured in a different unit, has sample standard deviation 10 and correlation \(0.45\). The means are \(\bar{y}=12\), \(\bar{x}_A=5\), and \(\bar{x}_B=20\). Calculate each slope and intercept, and rank the predictors by absolute correlation.
Step 1: Calculate slopes. The slope identity is \(b_1=r(s_Y/s_X)\). Thus for \(X_A\), \(b_{1A}=0.60(4/2)=1.20\) thousand dollars per \(X_A\) unit. For \(X_B\), \(b_{1B}=0.45(4/10)=0.18\) thousand dollars per \(X_B\) unit.
Step 2: Calculate intercepts. For \(X_A\), \(b_{0A}=12-(1.20)(5)=6\). For \(X_B\), \(b_{0B}=12-(0.18)(20)=8.4\). The fitted lines are \(\hat{y}_A=6+1.20x_A\) and \(\hat{y}_B=8.4+0.18x_B\), with the outcome expressed in thousands of dollars.
Step 3: Rank and check. By absolute correlation, \(X_A\) ranks first because \(0.60>0.45\). Their simple-regression \(R^2\) values are \(0.60^2=0.36\) and \(0.45^2=0.2025\). The slopes differ in both magnitude and units; the larger slope for \(X_A\) is not, by itself, the reason it ranks higher. Correlation and \(R^2\) give the stated ranking.
Method and mistake avoided. Use correlation for scale-free ranking and slopes for unit-specific fitted changes. This avoids treating raw slope magnitude as a universal measure of association.
Common Screening Traps
- Ranking signed correlations as if they measured strength. If one predictor has \(r=-0.90\) and another has \(r=0.40\), ranking raw values gives \(-0.90<0.40\) and might incorrectly place the first predictor lower. For strength of linear association, compare \(|-0.90|=0.90\) with \(|0.40|=0.40\); the negative sign indicates direction, not weakness.
- Calling a marginal ranking a conditional result. A predictor’s correlation with the outcome ignores the other predictors. As the four-record example shows, correlated predictors can have important joint roles despite weak individual correlations.
- Comparing slopes across incompatible units. Rescaling a predictor changes its slope even though its correlation with the outcome is unchanged. State the units before interpreting a slope.
- Treating correlation as a complete description of association. Pearson correlation summarizes linear association. A value near zero does not prove that no relationship of any kind exists.
- Comparing summaries calculated on different records. If one correlation uses one subset and another uses a different subset, the ranking may partly reflect the changed observations. Check the analysis sample and report any exclusions.
A univariate screen is most useful as a first-pass organizing tool: it helps identify clear marginal signals and gives a transparent summary of direction and linear strength. It should not be used alone to discard every weakly ranked predictor or to claim that the top-ranked variable will remain most useful in a multivariable model. The next modeling and validation steps must address how predictors work together and how well a proposed model predicts new outcomes.
Check Your Understanding
For each problem, use the stated marginal or simple-regression information only; distinguish a screening summary from a claim about conditional usefulness.
- A numeric outcome has sample standard deviation 3. A numeric predictor has sample standard deviation 6 and sample correlation \(-0.50\) with the outcome. The sample means are 20 for the outcome and 40 for the predictor. Find the simple-regression slope and intercept. (Answer: slope \(-0.25\); intercept \(30\).)
- Two numeric predictors have correlations \(0.70\) and \(-0.80\) with the same numeric outcome, calculated on the same records. Which ranks higher by absolute marginal correlation, and what are their simple-regression \(R^2\) values? (Answer: the predictor with correlation \(-0.80\); \(R^2=0.49\) and \(0.64\), respectively.)
- For a predictor and numeric outcome, \(SS_X=20\), \(SS_Y=45\), and \(SP_{XY}=-15\). Calculate the sample correlation, simple-regression slope, and \(R^2\). (Answer: \(r=-0.5000\), slope \(-0.75\), and \(R^2=0.25\).)
- In a four-record data set, \(X_1=(5,-5,5,-5)\), \(X_2=(6,-4,4,-6)\), and \(Y=(-1,-1,1,1)\). Find the two marginal correlations and state whether the weak marginal associations rule out joint usefulness. (Answer: \(r_{X_1Y}=0\), \(r_{X_2Y}\approx-0.1961\); no, because \(Y=X_1-X_2\) exactly.)
- A numeric predictor takes values \((-2,-1,0,1,2)\), and the numeric outcome is its square on each record. What is the sample Pearson correlation, and what does that value establish about a linear marginal screen? (Answer: \(r=0\); the screen finds no linear marginal association, but that does not establish the absence of every kind of relationship.)