Why Correlation Cutoffs Can Mislead

Why a Single Cutoff Is Not a Redundancy Test

A screening rule such as “remove one predictor whenever the absolute correlation exceeds 0.8” is easy to apply, but its simplicity can be misleading. A pairwise correlation summarizes one kind of relationship between two variables; it does not establish that one variable is unnecessary for every model, nor does a low correlation establish that two variables contain different information.

This matters in predictive analytics because a screening decision is made to support a modeling goal. A pair of predictors may have a nonlinear relationship, several predictors may be redundant only as a group, or two highly correlated predictors may carry a useful contrast for predicting the outcome. Correlation is a diagnostic, not an automatic deletion rule. This topic is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Definition: For paired observations $(x_i,z_i)$, $i=1,\ldots,n$, the sample Pearson correlation is $$ r_{XZ}= \frac{\sum_{i=1}^n(x_i-\bar{x})(z_i-\bar{z})} {\sqrt{\sum_{i=1}^n(x_i-\bar{x})^2\sum_{i=1}^n(z_i-\bar{z})^2}}. $$ It is defined when both variables have nonzero sample variance and summarizes the strength and direction of their linear association. It does not, by itself, describe every possible dependence between them.

The cutoff $c$ in a rule based on $|r_{XZ}|>c$ is a convention chosen for a particular screening process, not a universal statistical boundary. A cutoff can identify pairs worth investigating, but the result depends on the variables included, the data used to calculate the correlations, and the modeling objective. The formula is descriptive; no assumption of normality is required to calculate it. Statistical inference about a population correlation would require additional assumptions and is a separate question.

A Correlation of Zero Can Hide Exact Dependence

Pearson correlation detects linear association. A relationship can be perfectly determined by one variable and still have correlation zero if its shape is nonlinear. The following calculation shows why: when the observed $x$-values are symmetric around zero, positive and negative contributions to the covariance of $x$ and $x^2$ cancel.

Worked Example: A Nonlinear Relationship Below the Cutoff

A hypothetical pricing analysis records four centered exposure scores $X$ and a second feature $Z$, where $Z=X^2$. An analyst uses $|r|>0.8$ to flag pairs and concludes that pairs below the cutoff are not redundant. Calculate the sample correlation and assess the conclusion.

Method. Calculate the means, centered cross-products, and correlation. Then check whether the data show a relationship that Pearson correlation does not summarize.

The observations are $X=(-2,-1,1,2)$ and $Z=(4,1,1,4)$. Their means are $\bar{x}=(-2-1+1+2)/4=0$ and $\bar{z}=(4+1+1+4)/4=2.5$.

The centered cross-products sum to $(-2)(1.5)+(-1)(-1.5)+(1)(-1.5)+(2)(1.5)=-3+1.5-1.5+3=0$. The sums of centered squares are $\sum(x_i-\bar{x})^2=4+1+1+4=10$ and $\sum(z_i-\bar{z})^2=1.5^2+(-1.5)^2+(-1.5)^2+1.5^2=9$. Therefore,

r_{XZ}=\frac{0}{\sqrt{10\cdot 9}}=0.

The cutoff does not flag the pair. But every observation satisfies $Z=X^2$, so $Z$ is completely determined by $X$ in this data set. A linear correlation of zero is not evidence of independence or distinct information here.

Check. The centered cross-products cancel in positive-negative pairs, giving a zero numerator, while both sums of squares are positive. The computed correlation is therefore zero even though the relationship is exact.

Common mistake avoided: Treating $|r|=0$ as proof that the variables are unrelated. The correction is to interpret Pearson correlation as a measure of linear association and inspect whether the modeling process can represent a nonlinear feature relationship.

More generally, the cancellation follows from symmetry. If paired values occur as $(x,x^2)$ and $(-x,(-x)^2)$, their contributions to the centered cross-product have opposite signs when the mean of $X$ is zero and the paired squared values are equal. Thus the numerator can vanish even though $Z$ is a deterministic function of $X$. A scatterplot or a subject-matter check can reveal structure that the single correlation coefficient cannot.

The practical implication is not that every nonlinear relationship should be added to a model. It is that a low pairwise Pearson correlation cannot, on its own, justify treating two predictors as unrelated or interchangeable. Whether the relationship matters depends on the features and model being considered.

Several Predictors Can Be Redundant Together

A pairwise cutoff checks predictors two at a time. It can therefore miss a dependency involving three or more predictors. For example, one column may be an exact linear combination of two others even though neither pairwise correlation with it reaches the chosen threshold. In a model, that joint dependency can still cause unstable coefficient estimation or make the design matrix singular.

Worked Example: Pairwise Correlations Miss a Joint Dependency

A hypothetical portfolio model has three centered candidate predictors measured on four records. Let $A=(1,-1,1,-1)$, $B=(1,1,-1,-1)$, and $C=A+B$. A screening rule flags pairs only when $|r|>0.8$. Calculate the three pairwise correlations and determine whether the predictors are jointly redundant.

Method. Use the centered vectors directly in the correlation formula. Since each vector has mean zero, each numerator is a dot product and each sum of squares is its squared length.

The vectors are $A=(1,-1,1,-1)$, $B=(1,1,-1,-1)$, and $C=(2,0,0,-2)$. Their squared lengths are $A\cdot A=4$, $B\cdot B=4$, and $C\cdot C=8$. Also, $A\cdot B=1-1-1+1=0$, $A\cdot C=2+0+0+2=4$, and $B\cdot C=2+0+0+2=4$.

Thus $r_{AB}=0/\sqrt{4\cdot4}=0$, while $r_{AC}=4/\sqrt{4\cdot8}=1/\sqrt{2}\approx0.7071$ and $r_{BC}=4/\sqrt{4\cdot8}=1/\sqrt{2}\approx0.7071$.

None of the pairs exceeds 0.8, so the rule flags none. Nevertheless, $C=A+B$ exactly. The three columns are jointly linearly dependent: a model containing all three cannot uniquely estimate three separate coefficients from these data.

Check. The two correlations with $C$ are both about 0.7071, below the cutoff, but the displayed identity $C=A+B$ verifies exact joint redundancy.

Common mistake avoided: Concluding that no redundancy exists because every pair falls below the threshold. Pairwise screening cannot rule out a dependency among a set of predictors. The earlier material on variance inflation factors describes a diagnostic for linear dependence of a predictor on the other predictors collectively.

This example also shows why the word “redundant” needs context. A pairwise table describes pairwise relationships; a regression model uses all included columns together. A screening summary should not be mistaken for a complete assessment of the predictor matrix.

Predictor Similarity Is Not the Same as Predictive Usefulness

Correlation among predictors is an unsupervised property: it is calculated without considering the outcome. Predictive usefulness is outcome- and model-specific. A highly correlated pair may be individually similar but still contain a small contrast that is strongly related to the outcome. Deleting a variable solely because it crosses a predictor-correlation cutoff can remove that contrast.

Worked Example: A Useful Contrast Between Highly Correlated Predictors

A hypothetical claims model has four records. Let $A=(1,-1,1,-1)$, $B=(1,1,-1,-1)$, $X_1=A$, $X_2=A+0.1B$, and outcome $Y=B$. A pairwise rule flags predictor pairs above 0.8 in absolute correlation. Calculate the predictor correlation and compare the information in a model using both predictors with models using either one alone.

Method. Calculate the centered sums of squares and cross-products. Then use the exact relationship between the predictors and outcome to check what information is lost by deleting one predictor.

All three vectors have mean zero. Since $A\cdot A=B\cdot B=4$ and $A\cdot B=0$, the predictor cross-product is $X_1\cdot X_2=A\cdot(A+0.1B)=4$. The sums of squares are $X_1\cdot X_1=4$ and $X_2\cdot X_2=(A+0.1B)\cdot(A+0.1B)=4+0+0.01(4)=4.04$.

Therefore,

r_{X_1X_2}=\frac{4}{\sqrt{4(4.04)}}\approx0.9950.

The pair is flagged. Yet $X_2-X_1=0.1B$, so $Y=B=10(X_2-X_1)$. A linear model using both predictors can represent the outcome exactly: $Y=-10X_1+10X_2$, with zero intercept.

If $X_1$ alone is used, its cross-product with $Y$ is $A\cdot B=0$, so its fitted linear slope is zero. For $X_2$ alone, $X_2\cdot Y=(A+0.1B)\cdot B=0.4$; its fitted slope is $0.4/4.04\approx0.0990$, and its squared correlation with $Y$ is $0.4^2/(4.04\cdot4)\approx0.0099$. The one-predictor fit captures little of the outcome pattern, while the two-predictor contrast captures it exactly.

Check. The predictor correlation is greater than 0.8, but the outcome is exactly ten times their difference. The screening flag alone cannot establish that either predictor can be deleted without loss for this modeling goal.

Common mistake avoided: Treating a high predictor-predictor correlation as proof that one predictor cannot help predict the outcome. The relevant question is whether the modeling goal and validation results support the proposed simplification.

This example is deliberately constructed to isolate the issue; real data will not generally produce exact relationships. In practice, the predictive consequences of a screening decision should be assessed using the intended model and an appropriate validation design. A pairwise threshold does not measure out-of-sample performance, coefficient stability, interpretability, or the cost of a particular prediction error.

Use Cutoffs as Flags, Not Automatic Deletion Rules

A fixed threshold can still be useful. It gives a consistent way to flag pairs for review, especially when a data set has many predictors. But a conclusion based on that flag should be limited to what was actually measured: the chosen correlation, calculated on the available observations, using the chosen variables and method. For Pearson correlation, the flag concerns linear association.

Before acting on a flagged or unflagged pair, consider the following checks:

Check the relationship’s form. A scatterplot or subject-matter reasoning can reveal a curved or otherwise structured relationship that a linear coefficient summarizes poorly.
Check predictors collectively. Several predictors may share a dependency even when no individual pair passes the cutoff. A pairwise matrix alone cannot exclude this.
Keep the outcome and objective in view. Similar predictors can contribute a useful contrast, and the best screening decision may differ depending on the prediction task.
Check the data used. Missing-value handling, unusual observations, and small samples can affect estimated correlations. State which observations were used and be cautious when a correlation is unstable.
Validate a proposed simplification. Compare the resulting models using a validation approach appropriate to the task rather than assuming that a smaller predictor set must perform equally well.

A common exam trap is to read a threshold as a theorem: “below 0.8 means safe to retain” or “above 0.8 means one variable must be removed.” Neither statement follows from the correlation formula. A correct interpretation describes what the observed coefficient says, identifies what it does not establish, and links any screening action to the model’s purpose.

Key takeaway: A pairwise correlation cutoff is a screening convention, not a universal test of redundancy. It can miss nonlinear or joint dependence, and a highly correlated pair can still contain outcome-relevant information. Interpret the flag in the context of the intended model and validate consequential decisions.

Check Your Understanding

For each problem, distinguish what the correlation calculation establishes from what it cannot establish.

A feature $Z$ is recorded as $X^2$ for $X=(-3,-1,1,3)$. Explain why their sample Pearson correlation is zero and state whether $Z$ is determined by $X$. (Answer: $r=0$ by symmetry, but $Z=X^2$ exactly, so it is determined by $X$.)
Centered predictors $A$ and $B$ are orthogonal, each with squared length 4, and $C=A+B$. With a cutoff of 0.8, calculate $r_{AC}$ and say whether the three predictors are jointly independent columns. (Answer: $r_{AC}=1/\sqrt{2}\approx0.7071$, below the cutoff; the columns are jointly dependent because $C=A+B$.)
Two predictors have sample correlation 0.91. What does this establish, and what additional question must be answered before deleting one for a prediction model? (Answer: It establishes strong positive sample linear association; assess whether deleting one harms the intended model’s validated predictive performance or other stated objective.)
Suppose $X_2=X_1+0.2B$, where centered $X_1$ and $B$ are orthogonal, each with squared length 25. Calculate $r_{X_1X_2}$. (Answer: $25/\sqrt{25(25+0.04\cdot25)}=1/\sqrt{1.04}\approx0.9806$.)
In the preceding setup, let the centered outcome be $Y=B$. Explain why the high predictor correlation does not alone justify removing $X_2$. (Answer: $X_2-X_1=0.2B$, so $Y=5(X_2-X_1)$; the two-predictor contrast represents the outcome exactly, while a predictor-only cutoff does not assess this outcome-specific information.)

What You'll Learn