Two Different Questions About a Variable
A variable can help predict an outcome without causing it. For example, a hypothetical insurer might find that a certain account flag is associated with a higher claim rate. The flag could still be useful for forecasting even if changing the flag itself would not change anyone’s claim risk. The association may instead reflect other characteristics that the flag happens to identify.
This distinction matters because prediction and causal analysis ask different questions. A predictive question asks what outcome is likely for cases with observed information. A causal question asks how outcomes would differ if the same target population received one intervention rather than another. A strong predictive relationship alone does not answer the second question.
The conditional probability \(P(Y=1\mid X=x)\) describes the outcome rate among cases with observed value \(X=x\). It can support a forecast if the relationship is stable in the population where the forecast will be used. It does not, by itself, describe what would happen if someone changed \(X\).
Potential Outcomes: Defining an Intervention Effect
For a binary intervention, let \(A=1\) denote receiving the intervention and \(A=0\) denote not receiving it. For each individual, define \(Y(1)\) as the outcome that would occur under the intervention and \(Y(0)\) as the outcome that would occur without it. For example, if \(Y\) records whether a loss occurs during a fixed period, both potential outcomes are zero or one.
This definition specifies a contrast between two intervention conditions. It is not the same quantity as the observed difference \(E[Y\mid A=1]-E[Y\mid A=0]\). That observed difference compares people who did and did not receive the intervention; it equals the ATE only under suitable conditions, such as random assignment or adequate control for confounding.
A causal question also requires a clearly specified intervention and outcome. “What is the effect of a safety review?” is incomplete if the review’s content, timing, target population, or outcome period is unclear. The intervention conditions must be sufficiently well defined for \(Y(1)\) and \(Y(0)\) to have interpretable meanings.
Worked Example: A Predictive Flag With No Within-Group Association
A hypothetical insurer reviews 200 accounts. A flag indicates that an account has a certain recorded feature. The accounts also fall into high- and low-baseline-risk groups. During a defined future period, the following outcomes are observed. Assume the flag is a marker and is not itself an intervention.
| Baseline risk | Flagged: losses / accounts | Not flagged: losses / accounts |
|---|---|---|
| High | 32 / 80 | 8 / 20 |
| Low | 2 / 20 | 8 / 80 |
Method. First calculate the overall predictive association, then compare rates within each baseline-risk group.
Among flagged accounts, there are \(32+2=34\) losses among \(80+20=100\) accounts. The observed loss rate is \(34/100=0.34\). Among unflagged accounts, there are \(8+8=16\) losses among \(20+80=100\) accounts, giving \(16/100=0.16\). The crude difference is \(0.34-0.16=0.18\), or 18 percentage points. The flag is predictive in this observed data.
Now compare within baseline-risk groups. In the high-risk group, the flagged rate is \(32/80=0.40\) and the unflagged rate is \(8/20=0.40\). In the low-risk group, the flagged rate is \(2/20=0.10\) and the unflagged rate is \(8/80=0.10\). Within each group, the observed rates are equal.
The flagged accounts contain a larger proportion of high-risk accounts: \(80/100=0.80\), compared with \(20/100=0.20\) among unflagged accounts. The overall association arises because baseline risk is distributed differently between the groups. It does not show that changing the flag would change losses. Common mistake: calling the 18-point crude difference the causal effect of the flag. That calculation compares differently composed groups; the within-group comparisons show why the causal claim is not supported.
Check. The four row totals are 200 accounts, and the loss counts sum to \(32+8+2+8=50\). The computed overall rates use 100 accounts in each flag group, as required by the table.
Why Random Assignment Supports a Causal Comparison
A randomized experiment assigns the intervention independently of individuals’ potential outcomes. Under random assignment, the treated and untreated groups are comparable in expectation with respect to those potential outcomes. Two additional conditions clarify what is being estimated: consistency, meaning the observed outcome equals the potential outcome under the intervention actually received, and no interference, meaning one individual’s outcome is not changed by another individual’s assignment. Together these conditions allow the group outcome averages to estimate the corresponding potential-outcome averages.
The justification can be written directly. If \(A\) is randomly assigned, then it is independent of \(Y(1)\) and \(Y(0)\). Consistency gives \(Y=Y(1)\) when \(A=1\), and \(Y=Y(0)\) when \(A=0\). Therefore:
The last equality follows from linearity of expectation. In a finite experiment, observed rates will generally differ from the population quantities because of random variation. Random assignment supports a causal interpretation of the comparison; it does not guarantee that a particular sample difference is exactly the true effect.
Worked Example: Estimating an Effect in a Randomized Trial
A hypothetical organization randomly assigns 200 eligible accounts, 100 to a new inspection procedure and 100 to the existing procedure. The outcome is whether an account has a recorded loss during the next six months. There are 12 losses in the new-procedure group and 20 in the existing-procedure group. Assume assignment was implemented as specified, outcome recording is complete, the procedures are well defined, and one account’s assignment does not affect another account’s outcome.
Method. Estimate the average effect on the binary outcome by subtracting the randomized groups’ observed loss rates, in the order new procedure minus existing procedure.
The new-procedure rate is \(12/100=0.12\). The existing-procedure rate is \(20/100=0.20\). The estimated difference is \(0.12-0.20=-0.08\). Thus, the observed estimate is an 8-percentage-point reduction in the six-month loss rate for assignment to the new procedure, compared with assignment to the existing procedure.
Common mistake: reporting \(20/12\), or a percentage change in the number of losses, as the requested difference in risk. The groups have equal sizes, but the causal contrast requested here is a difference in rates: \(12/100-20/100\), not a ratio of loss counts. Also, the estimate applies to the stated outcome window and trial population; it does not automatically establish the effect in a different population or period.
Check. The groups contain \(100+100=200\) accounts, and \(12+20=32\) accounts had losses. The difference is negative because the observed rate is lower in the new-procedure group.
Confounding and the Limits of Adjustment
A confounder is a variable related to both intervention receipt and the outcome that can make their observed association differ from the causal effect. In the flag example, baseline risk was associated with flag status and with losses. Even when a variable is useful for prediction, its relationship with the outcome may be due partly or wholly to such differences in case mix.
One way to make an adjusted comparison is to calculate outcome rates within groups defined by measured baseline characteristics, then average those rates using the same target-population weights for both intervention conditions. This is standardization. It can address differences represented by the measured groups, but a causal interpretation still requires assumptions: the relevant confounders must be measured and appropriately controlled; each relevant group must have a meaningful chance of receiving either condition (positivity); and intervention definitions and outcomes must be consistent. Adjustment cannot rule out bias from unmeasured confounding merely because the adjusted results look plausible.
Worked Example: Crude and Standardized Program Rates
In a hypothetical observational evaluation, accounts either participated in a prevention program or did not. High- and low-baseline-risk status was measured before participation. The outcome is a loss during the same fixed period for both groups. The target population for standardization is defined to be 50% high-risk and 50% low-risk. Assume, for the purpose of interpreting the calculation, that within each risk group there is no unmeasured confounding, both participation options are represented, and the program and outcome are well defined. These assumptions are substantial; the table alone cannot verify them.
| Baseline risk | Program participants: losses / accounts | Nonparticipants: losses / accounts |
|---|---|---|
| High | 32 / 80 | 10 / 20 |
| Low | 2 / 20 | 4 / 80 |
Method. Compare crude rates first, then calculate each condition’s rate under the same 50%/50% target-population mix.
There are \(34\) losses among \(100\) participants, so the crude participant rate is \(34/100=0.34\). There are \(14\) losses among \(100\) nonparticipants, so the crude nonparticipant rate is \(14/100=0.14\). The crude difference is \(0.34-0.14=0.20\), which is 20 percentage points higher among participants.
Within the high-risk group, the participant rate is \(32/80=0.40\), while the nonparticipant rate is \(10/20=0.50\). Within the low-risk group, the participant rate is \(2/20=0.10\), while the nonparticipant rate is \(4/80=0.05\).
For participants, the standardized rate is \(0.50(0.40)+0.50(0.10)=0.20+0.05=0.25\). For nonparticipants, it is \(0.50(0.50)+0.50(0.05)=0.25+0.025=0.275\). The standardized difference is \(0.25-0.275=-0.025\), or 2.5 percentage points lower for participants under the stated target mix and assumptions.
The reversal occurs because 80% of participants are high-risk, compared with 20% of nonparticipants. Common mistake: concluding from the crude 20-point increase that participation raises losses. The crude comparison uses different risk-group mixes; standardization uses the same specified mix. The adjusted result is not automatically causal: its causal interpretation depends on the assumptions stated above. Check. Each standardized rate uses weights summing to \(0.50+0.50=1\), and their difference is \(0.250-0.275=-0.025\).
What Prediction Can and Cannot Establish
A predictor does not have to be a cause. A location, an account-history flag, or another recorded feature may help forecast an outcome because it summarizes information about underlying risk. Removing the feature from a data table, changing its recorded value, or intervening on it are distinct actions; none is automatically equivalent to changing the underlying risk factors.
Conversely, a variable can have a causal effect but add little predictive value in a particular dataset. Its effect may be small relative to other sources of variation, or the variable may barely vary across the cases being forecast. Prediction concerns how well observed information forecasts outcomes in a specified setting. Causal analysis concerns a contrast between interventions in a specified population. Neither question substitutes for the other.
When reviewing a result, state which quantity it estimates. A conditional outcome rate is descriptive or predictive. A crude difference between intervention groups is an observed association. An adjusted or randomized comparison may estimate an intervention effect only when its design and assumptions support that interpretation. This distinction is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
- Ask whether the task is forecasting an observed outcome or estimating the effect of a specified intervention.
- For observed group comparisons, check whether the groups differ in baseline characteristics related to the outcome.
- For adjusted estimates, identify the target population, adjustment variables, and assumptions behind the causal interpretation.
- Do not infer that changing a predictive feature will change the outcome merely because the feature improves prediction.
Check Your Understanding
For each problem, distinguish the observed predictive or descriptive comparison from any causal conclusion it can support.
- A hypothetical portfolio has 60 accounts with a renewal-risk marker, of which 18 have a claim in the following year. Of 80 accounts without the marker, 8 have a claim. Calculate both claim rates and their difference, marker group minus non-marker group. Does this establish a causal effect of the marker? (Answer: \(18/60=0.3000\), \(8/80=0.1000\), difference \(=0.2000\); no causal effect is established.)
- A randomized pilot assigns 75 accounts to a revised review process and 75 to the standard process. There are 9 losses in the revised group and 15 in the standard group. Calculate the observed risk difference, revised minus standard, and state what supports a causal interpretation. (Answer: \(9/75-15/75=0.12-0.20=-0.0800\); random assignment, consistency, and no interference support interpreting it as an estimated effect.)
- In an observational study, a program group has rates \(0.30\) among high-risk accounts and \(0.10\) among low-risk accounts. A comparison group has rates \(0.40\) and \(0.05\), respectively. For a target population that is 40% high-risk and 60% low-risk, calculate each standardized rate and the difference, program minus comparison. (Answer: program \(=0.40(0.30)+0.60(0.10)=0.18\); comparison \(=0.40(0.40)+0.60(0.05)=0.19\); difference \(=-0.0100\), subject to causal assumptions.)
- A score predicts higher claim rates for accounts with an old address on file. No intervention or randomized change to the address is studied. Which conclusion is supported: prediction, causal effect, or both? Name one additional concern if the score is used in a population with a different mix of account types. (Answer: The observed relationship may support prediction, but it does not establish a causal effect; the score’s predictive relationship may not transport to a population with a different case mix.)