Why Marginal Screens Miss Interactions

Why a Predictor Can Matter Only in Combination

A univariate screen evaluates predictors one at a time. It might rank variables by correlation, a difference in average outcomes, or another measure of marginal association. This is useful when a predictor has a clear standalone relationship with the outcome. But some signals depend on a combination: one predictor’s relationship with the outcome changes according to the value of another predictor.

In that situation, averaging over the second predictor can cancel the pattern. Each predictor may appear weak marginally even though the pair strongly separates outcomes. This matters when screening is used to reduce a large candidate set: removing both variables before considering their combination may remove information that a later model could have used.

Definition: An interaction occurs when the relationship between an outcome and one predictor depends on the value of another predictor. A marginal association describes the relationship between the outcome and one predictor after averaging over, or ignoring, the other predictors. A weak marginal association does not establish that a predictor has little value jointly.

Interaction is scale-dependent: a pattern may be present on one outcome scale and not another. Here, the examples state the scale and measure being examined. A screen based on correlation, for instance, should not be treated as a universal test for all forms of predictive value.

A Balanced Binary Pattern: Marginal Cancellation

Suppose two binary predictors, \(X_1\) and \(X_2\), are equally represented in all four combinations. Let the binary outcome \(Y\) have probability 0.9 when the predictors match and probability 0.1 when they differ. Conditional on the predictors, outcomes are Bernoulli with the stated probabilities. This hypothetical setup has a strong joint pattern: matching predictors indicate high risk, and differing predictors indicate low risk.

P(Y=1\mid X_1,X_2)= \begin{cases} 0.9, & X_1=X_2,\\ 0.1, & X_1\ne X_2. \end{cases}

Yet the marginal risk is the same at each value of either predictor. Given \(X_1=0\), for example, \(X_2\) is 0 or 1 with equal probability, so the conditional risk averages a matching cell and a differing cell. The same cancellation occurs for \(X_1=1\), and for each value of \(X_2\).

Worked Example: A Joint Risk Pattern With No Marginal Risk Difference

A hypothetical insurer studies 400 otherwise comparable exposures. There are 100 exposures in each combination of two binary indicators, \(X_1\) and \(X_2\). The outcome \(Y=1\) is a claim. Within each cell, claims follow a Bernoulli distribution: the claim probability is 0.9 if the indicators match and 0.1 if they differ. Calculate the marginal claim risks and the accuracy of predicting a claim whenever the indicators match.

Method. Average the cell-specific probabilities over the other predictor to obtain each marginal risk. Then calculate the expected number of correct predictions using the joint rule.

For \(X_1=0\), 100 exposures are in the matching cell \((0,0)\) and 100 are in the differing cell \((0,1)\). Thus:

P(Y=1\mid X_1=0)=\frac{0.9+0.1}{2}=0.5.

For \(X_1=1\), the cells are \((1,1)\) and \((1,0)\), again with probabilities 0.9 and 0.1:

P(Y=1\mid X_1=1)=\frac{0.9+0.1}{2}=0.5.

By the same calculation, \(P(Y=1\mid X_2=0)=P(Y=1\mid X_2=1)=0.5\). Each marginal risk difference is therefore \(0.5-0.5=0\). A screen using these risk differences would find no marginal signal.

Now predict \(Y=1\) for matching indicators and \(Y=0\) otherwise. In each of the 200 matching exposures, the probability of a correct prediction is 0.9. In each of the 200 differing exposures, it is also 0.9, because the rule predicts no claim and the claim probability is 0.1. The expected number correctly classified is \(200(0.9)+200(0.9)=360\), so expected accuracy is \(360/400=0.9\), or 90%.

As a check, the expected number of claims is \(100(0.9)+100(0.1)+100(0.1)+100(0.9)=200\), consistent with an overall claim probability of 0.5. The joint rule’s expected accuracy is therefore 90%, despite zero marginal risk differences. Common mistake avoided: treating the 0.9 probability in the matching cell as the marginal risk given \(X_1=0\); that marginal risk must average over both values of \(X_2\).

Why the Cancellation Happens

The balanced binary example can be expressed compactly using values coded as \(-1\) and \(+1\). Let \(A\) and \(B\) be independent, each taking \(-1\) and \(+1\) with probability one half. In this coding, the predictors match when \(AB=+1\). The conditional mean in the example is:

E[Y\mid A,B]=0.5+0.4AB.

This formula follows directly from the two cell probabilities: if \(AB=+1\), the mean is \(0.5+0.4=0.9\); if \(AB=-1\), it is \(0.5-0.4=0.1\). To find the marginal mean given \(A\), average over \(B\). Since \(B\) has mean zero and is independent of \(A\), \(E[AB\mid A]=A E[B]=0\). Thus \(E[Y\mid A]=0.5\) for either value of \(A\). The same argument gives \(E[Y\mid B]=0.5\). The joint term survives, while averaging over either predictor removes it.

This is the central reason a marginal screen can miss an interaction: the screen averages over contexts in which the predictor’s effect points in opposite directions. It is not that the predictors contain no information. Rather, the information is arranged jointly, not as a standalone average relationship.

A Product Feature Can Carry the Signal

The same issue arises with quantitative predictors. A linear marginal screen may look for a relationship between each predictor and the outcome separately, while the relevant signal is a product of the two predictors. The next example derives the population quantities under explicit distributional assumptions.

Worked Example: Zero Marginal Covariances but a Predictive Product

In a hypothetical severity study, let \(X\) and \(Z\) be independent standard normal predictors, so \(X\sim N(0,1)\) and \(Z\sim N(0,1)\). Let the outcome be \(Y=2XZ+\varepsilon\), where \(\varepsilon\sim N(0,1)\) and is independent of both predictors. Find the marginal covariances of \(X\) and \(Z\) with \(Y\), then calculate the population least-squares slope and \(R^2\) when using the constructed feature \(W=XZ\) alone.

Method. Use independence and the standard normal moments \(E[X]=E[Z]=0\), \(E[X^2]=E[Z^2]=1\), and \(E[\varepsilon]=0\). First calculate the marginal covariances, then the moments needed for regression on \(W\).

Since \(E[Y]=0\), the covariance of \(X\) with \(Y\) is:

\operatorname{Cov}(X,Y)=E[XY] =2E[X^2Z]+E[X\varepsilon] =2E[X^2]E[Z]+0=0.

Similarly, \(\operatorname{Cov}(Z,Y)=2E[X]E[Z^2]+E[Z\varepsilon]=0\). Thus both predictors have zero marginal linear covariance with the outcome.

For the product feature, \(W=XZ\) has mean zero and variance \(E[X^2Z^2]=E[X^2]E[Z^2]=1\). Also, \(Y=2W+\varepsilon\), so:

\operatorname{Cov}(W,Y)=E[W(2W+\varepsilon)] =2E[W^2]+E[W\varepsilon]=2.

The population least-squares slope using \(W\) is \(\operatorname{Cov}(W,Y)/\operatorname{Var}(W)=2/1=2\), with intercept zero. To calculate \(R^2\), first note that \(\operatorname{Var}(Y)=4\operatorname{Var}(W)+\operatorname{Var}(\varepsilon)=4+1=5\). Therefore:

R^2=\frac{\operatorname{Cov}(W,Y)^2} {\operatorname{Var}(W)\operatorname{Var}(Y)} =\frac{2^2}{1(5)}=0.8.

The check is that the product explains variance 4 out of total variance 5, leaving the independent noise variance 1. Common mistake avoided: concluding that \(X\) and \(Z\) have no predictive value because each has zero marginal covariance. The result applies to the stated linear marginal screen; it does not say that every possible marginal screen will fail or that \(X\) and \(Z\) are independent of the outcome jointly.

Rare Combinations Can Be Diluted Marginally

Cancellation is not the only problem. A joint pattern can be concentrated in a small subgroup. When a predictor is screened alone, its comparison group mixes that subgroup with many exposures that do not share the joint pattern. The marginal difference may then be modest even though the subgroup itself has a very different outcome rate.

Worked Example: A Rare High-Risk Combination

Consider two independent binary indicators \(A\) and \(B\) for a hypothetical portfolio. Each equals 1 with probability 0.1. The outcome \(Y\) is Bernoulli conditional on the indicators, with claim probability 0.80 when both indicators equal 1 and 0.05 in each of the other three cells. Calculate the marginal claim risks for \(A\) and \(B\), the overall claim probability, and the risk difference for the joint indicator \(H=AB\).

Method. Average the conditional claim probabilities using the predictor distribution. Independence gives \(P(A=1,B=1)=0.1(0.1)=0.01\), so the high-risk combination is only 1% of exposures.

Given \(A=1\), the probability that \(B=1\) is 0.1. Therefore:

P(Y=1\mid A=1)=0.1(0.80)+0.9(0.05)=0.08+0.045=0.125.

Given \(A=0\), the claim probability is 0.05 for either value of \(B\), so \(P(Y=1\mid A=0)=0.05\). The marginal risk difference for \(A\) is \(0.125-0.05=0.075\). By symmetry, the risk difference for \(B\) is also 0.075.

The overall claim probability is the probability-weighted average:

$$ P(Y=1)=0.01(0.80)+0.99(0.05)=0.008+0.0495=0.0575. $$

For the joint indicator \(H\), the risk is 0.80 when \(H=1\) and 0.05 when \(H=0\). Its risk difference is \(0.80-0.05=0.75\), much larger than either marginal difference. As a count check, in 10,000 exposures the joint subgroup has an expected 100 exposures and 80 claims; the remaining 9,900 exposures have an expected 495 claims. The expected total is 575 claims, or \(575/10{,}000=0.0575\). Common mistake avoided: comparing the joint subgroup’s 0.80 risk directly with the overall 0.0575 rate and calling that a marginal effect of \(A\). The marginal comparison for \(A\) must include the other \(A=1\) exposures, for which \(B=0\).

Practical Implications for Screening

A marginal screen answers a limited question: does this predictor show a sufficiently strong relationship with the outcome by itself, according to the chosen measure? It does not answer whether that predictor could contribute as part of a combination. When interactions are plausible, screening decisions should reflect that limitation.

Check the screening measure. Zero linear correlation rules out a linear marginal association, not every nonlinear or joint pattern. A screen based on group averages may miss a pattern that changes direction across groups.
Use context to nominate combinations. Subject-matter reasoning can identify pairs whose joint values might reasonably change risk, cost, or behavior. This does not prove an interaction, but it can identify features that a purely marginal ranking would overlook.
Avoid irreversible pruning on a single weak score. If a predictor has a plausible joint role, retain it or assess a candidate combination before removing it solely because of a small marginal association.
Evaluate the combination as a feature. A product, indicator for a joint category, or other explicitly constructed feature can represent the combined pattern. The construction should match the predictor types and the question being studied.
Do not infer that every pair matters. Adding many combinations can create complexity and unstable estimates. A weak marginal score is a reason to ask what the screen can detect, not a reason to include every possible interaction.

A related exam trap is to interpret “not selected by a univariate screen” as “has no predictive value.” Those statements are not equivalent. The screen has assessed a predictor in isolation using a particular criterion. To make a broader claim about predictive value, one would need to consider the relevant joint structure and evaluate it appropriately.

Key takeaway: Marginal screening averages over other predictors. When the outcome pattern depends on a combination, that averaging can cancel or dilute the signal. Recognize plausible interactions before discarding predictors, and distinguish a weak standalone association from a weak joint contribution.

Check Your Understanding

For each problem, calculate the requested marginal and joint quantities before checking the final answer.

Two balanced binary predictors \(C\) and \(D\) have a Bernoulli outcome with probability 0.85 when they match and 0.15 when they differ. Find the risk given each value of \(C\), and the accuracy of predicting an outcome of 1 for matching pairs. (Answer: Both marginal risks are 0.5000; joint-rule accuracy is 0.8500.)
Let independent predictors \(U,V\sim N(0,1)\), and let \(T=3UV+\eta\), where \(\eta\sim N(0,4)\) is independent. Find \(\operatorname{Cov}(U,T)\), \(\operatorname{Cov}(V,T)\), and the population \(R^2\) using \(UV\) alone. (Answer: Both covariances are 0; \(R^2=9/13=0.6923\).)
Independent binary predictors \(G,H\) have probabilities \(P(G=1)=0.2\) and \(P(H=1)=0.05\). The Bernoulli outcome risk is 0.90 if both equal 1 and 0.04 otherwise. Find the risk difference for \(G\) alone and for the joint indicator \(GH\). (Answer: \(P(Y=1\mid G=1)=0.0830\), \(P(Y=1\mid G=0)=0.0400\), marginal difference \(0.0430\); joint difference \(0.8600\).)
Two balanced binary predictors have Bernoulli outcome probabilities 0.12 in the matching cells and 0.28 in the differing cells. Find the marginal risk for either value of the first predictor and the difference between differing-cell and matching-cell risks. (Answer: Each marginal risk is 0.2000; differing minus matching risk is 0.1600.)
A candidate predictor has zero marginal correlation with the outcome, but subject-matter reasoning suggests that its effect may reverse across two groups defined by another predictor. What does the zero correlation establish, and what should be examined before removing the predictor? (Answer: It establishes zero linear marginal covariance under the data and coding used; examine the group-conditional pattern or a justified joint feature before deciding it has no predictive value.)

What You'll Learn