When Predictions Change What Happens Next
A predictive model is often used to decide what to do: contact a customer, inspect a claim, offer a retention incentive, or take no action. Once the decision is made, the outcome may differ from what would have happened without it. The resulting data are therefore not always a neutral record of the original population. They can reflect both the underlying risks and the policy applied to those risks.
This creates a feedback loop. A model identifies a group as high risk; a policy directs an intervention to that group; the intervention changes behavior or the chance an outcome is observed; and later data are used to assess or retrain the model. A lower claim rate among the targeted group, for example, could mean the original prediction was too high, the intervention prevented claims, or both. Separating those explanations matters when interpreting results.
The key distinction is between predicting an outcome under the conditions represented in the data and predicting what would happen under a specified action. For causal comparisons, let \(Y(1)\) denote an individual’s outcome if action 1 is taken and \(Y(0)\) the outcome if action 0 is taken. Only one of these outcomes is observed for a given individual. They are called potential outcomes. For a claim indicator, for example, \(Y(a)=1\) can mean that a claim occurs under action \(a\).
An action policy may assign an action based on predictors \(X\). A policy can be deterministic, such as “offer a review when the risk score exceeds a threshold,” or stochastic, assigning action \(a\) with probability \(\pi(a\mid X)\). The expected outcome under that policy is a policy-specific quantity, not automatically the same as the outcome rate observed before deployment.
Calculating an Outcome Under a Policy
The policy mean depends on action-specific potential outcomes. For discrete predictors and actions, if the policy’s action is independent of the potential outcomes conditional on \(X\), then:
Here \(\pi(a\mid x)\) is the probability that the policy assigns action \(a\) to cases with predictor value \(x\). For this identity, assume consistency—when action \(A=a\) is actually assigned, the observed outcome equals \(Y(a)\)—and assume policy assignment is independent of the potential outcomes conditional on \(X\). This conditional independence holds by design in a suitable randomized assignment. It should not be assumed merely because the action and predictors were recorded.
To estimate the action-specific means from observational data, further conditions matter. In particular, there must be adequate positivity: for predictor values where the policy may assign an action, the data must contain relevant observations under that action. If every high-risk case historically received an intervention, those data alone cannot reveal what the high-risk cases’ untreated outcome mean would have been. A randomized pilot or another justified source of comparison may be needed.
The expression is not valid if policy assignment remains associated with potential outcomes after conditioning on \(X\). For example, if staff give an intervention to cases they privately judge to be most likely to worsen, \(X\) may not capture that judgment. Weighting the marginal \(E[Y(a)\mid X=x]\) by the policy’s action probabilities would then generally fail to give the policy mean. One would need additional assumptions or data to address that association.
Why the Policy-Mean Formula Holds
Under the stated conditional independence and consistency assumptions, first condition on the predictor value and then on the action assigned by the policy. By the law of total expectation:
Consistency gives \(Y=Y(a)\) for cases assigned action \(a\). Conditional independence lets us replace the mean among those assigned cases with the action-specific potential-outcome mean for the predictor group:
Since \(P(A=a\mid X=x)=\pi(a\mid x)\), substitution gives the policy-mean formula. Without conditional independence, the second equality is not guaranteed: cases receiving an action may differ in their potential outcomes from other cases with the same recorded \(X\).
Worked Example: Estimating Claims Under a Loss-Prevention Policy
A hypothetical insurer considers a prevention call for policyholders classified as high risk. A randomized pilot within each risk group estimates the following annual claim probabilities. There are 200 high-risk and 800 low-risk policyholders. The proposed policy calls every high-risk policyholder and makes no calls to low-risk policyholders. Treat the pilot probabilities as applicable to this population, and define the action as receiving the call.
| Risk group | Count | Claim probability without call | Claim probability with call |
|---|---|---|---|
| High risk | 200 | 0.20 | 0.10 |
| Low risk | 800 | 0.05 | 0.04 |
Method. Apply the policy to each group and multiply its count by the claim probability under the assigned action. The randomized pilot supports using the action-specific means for this calculation.
High-risk expected claims under the policy: \(200(0.10)=20\). Low-risk expected claims: \(800(0.05)=40\), because low-risk policyholders receive no call. Total expected claims are \(20+40=60\), so the policy’s expected claim rate is \(60/1{,}000=0.060\), or 6.0%.
Without calls, expected claims would be \(200(0.20)+800(0.05)=40+40=80\), for a rate of \(80/1{,}000=0.080\), or 8.0%. Under the stated assumptions, the policy reduces expected claims by \(80-60=20\), or \(20/80=25\%\) relative to the no-call expectation.
Check. The group contributions under the policy sum to \(20+40=60\), and the group counts sum to \(200+800=1{,}000\). The calculated rate is consistent with the total.
Common mistake avoided. Using the no-call rate for the high-risk group after the calls are introduced gives \(200(0.20)+800(0.05)=80\). That calculation describes the no-call scenario, not the policy scenario. Conversely, the hypothetical pilot estimates are not evidence that every deployed policy will achieve exactly these outcomes; the calculation depends on their applicability to the population and on the policy actually being implemented as described.
Targeting Can Change Which Labels Are Observed
A policy can change data even when it does not directly change the underlying behavior: it may determine which cases receive an investigation, measurement, or verified outcome. If unreviewed cases are recorded as having no positive finding, a model can learn the review policy as well as the underlying event. If unreviewed cases are instead marked missing, the available labels are still selected by the policy.
Worked Example: Targeted Claim Reviews and Confirmed Fraud Labels
A hypothetical insurer receives 1,000 claims: 200 are in a high-score group and 800 in a low-score group. Assume true fraud prevalence is 30% in the high-score group and 2% in the low-score group. The insurer reviews 80% of high-score claims and 10% of low-score claims. A review detects 90% of fraudulent claims, and assume no nonfraudulent claim is incorrectly confirmed as fraud. Unreviewed claims have no confirmed fraud label.
Method. Calculate the expected true fraud count first, then the expected reviewed fraud count and detections. The expected counts may be fractional because they are averages over repeated populations with these rates.
High-score true fraud claims: \(200(0.30)=60\). Low-score true fraud claims: \(800(0.02)=16\). Thus, expected true fraud claims in the full population are \(60+16=76\), a prevalence of \(76/1{,}000=0.076\), or 7.6%.
The review counts are \(200(0.80)=160\) high-score claims and \(800(0.10)=80\) low-score claims, for \(160+80=240\) reviewed claims. Expected fraudulent claims among those reviewed are \(60(0.80)=48\) and \(16(0.10)=1.6\), for \(48+1.6=49.6\). At 90% sensitivity, expected confirmed positive labels are \(49.6(0.90)=44.64\).
Among reviewed claims, the expected confirmed-positive proportion is \(44.64/240=0.186\), or 18.6%. If unreviewed claims are incorrectly treated as confirmed negatives, the recorded positive proportion across all claims is \(44.64/1{,}000=0.04464\), or 4.464%. Neither figure is the underlying 7.6% prevalence: the first is a rate among selected, reviewed claims, and the second also counts unreviewed claims as if their status were known.
Check. The expected reviewed fraud count, 49.6, is less than the 76 true fraud claims in the population; the 90% sensitivity reduces expected confirmations further to 44.64. Both checks are consistent with the assumptions.
Common mistake avoided. Calling 4.464% the fraud prevalence treats every unreviewed claim as nonfraud. The correction is to distinguish confirmed labels from the underlying outcome: here the assumed underlying prevalence is 7.6%, while the policy determines which cases can receive a confirmed label. Estimating prevalence from this review process requires accounting for selection and imperfect detection, or collecting labels through an appropriate validation process.
Customer Outcomes Can Also Change
Interventions directed at customers can change the outcome that a model is later trained to predict. This is not necessarily a defect: reducing churn or claims may be exactly the policy’s purpose. The important point is to describe what the label means. “Churn under the current retention policy” differs from “churn if no retention offer were made.”
Worked Example: Retention Offers and the Future Churn Rate
A hypothetical service provider identifies 200 customers as high predicted churn risk and 800 as low risk. In a randomized pilot, high-risk customers offered a retention benefit have a 25% churn rate; comparable high-risk customers not offered it have a 50% rate. Low-risk customers, who are not offered the benefit under the proposed policy, have a 10% churn rate. Assume these rates apply to the 1,000 customers and that the policy offers the benefit to every high-risk customer.
Method. Calculate churn under the policy using each group’s rate for its assigned action, then compare with the no-offer scenario.
Under the policy, expected high-risk churns are \(200(0.25)=50\). Expected low-risk churns are \(800(0.10)=80\). The total is \(50+80=130\), giving a policy churn rate of \(130/1{,}000=0.13\), or 13%.
If no customers received an offer, expected churns would be \(200(0.50)+800(0.10)=100+80=180\), or 18%. The policy scenario has \(180-130=50\) fewer expected churns. This difference is an expected comparison based on the pilot rates, not a guarantee for an individual customer.
After deployment, a model trained on these outcomes may observe a 25% churn rate among high-risk customers under the offer policy. That rate does not establish that their untreated churn risk is 25%; the pilot’s no-offer rate was 50%. If the target is future churn under the same offer policy, the 25% rate may be relevant. If the target is churn without an offer, the post-policy labels do not directly answer the question.
Check. The policy total, \(50+80=130\), is lower than the no-offer total, \(100+80=180\), by exactly 50 expected churns. The group counts still total 1,000.
Common mistake avoided. Interpreting the post-offer 25% as the high-risk group’s untreated risk confuses the outcome under action 1 with the outcome under action 0. State the action regime associated with a prediction or comparison.
Practical Interpretation and Exam Traps
When a model influences decisions, record both the prediction and the action that follows it. Otherwise, later analysts may see an outcome without knowing whether it arose under an intervention, a review policy, or no action. The distinctions are useful when explaining results to decision-makers: a lower observed event rate can be a policy success, a change in population, a change in measurement, or evidence of model error.
- Do not equate a prediction with a causal effect. A risk score estimates an outcome under the conditions represented by its target and data; it does not by itself estimate what an intervention will change.
- Do not treat missing labels as negative outcomes. A case that was not reviewed has an unknown status unless the outcome is otherwise established.
- Do not use the policy-mean formula without its assumptions. Conditional independence of action and potential outcomes is essential for replacing action-specific means among assigned cases with the population means \(E[Y(a)\mid X]\).
- Do not overlook lack of action overlap. When historical policy assigns only one action to a predictor group, its data may not identify the mean outcome under the alternative action.
- Do not describe a policy-conditioned rate as an untreated rate. Name the action regime that generated the observed outcome.
Check Your Understanding
For each problem, identify the relevant policy effect or data-selection issue and check the arithmetic.
- A hypothetical prevention program covers 100 high-risk and 300 low-risk accounts. Under the program, their claim probabilities are 0.12 and 0.04, respectively. Calculate expected claims and the overall rate. (Answer: \(100(0.12)+300(0.04)=24\) expected claims; rate \(24/400=0.060\), or 6.0%.)
- A review program examines 50% of 120 high-score claims and 20% of 280 low-score claims. How many claims are reviewed? If unreviewed claims have unknown status, may they be counted as confirmed negatives? (Answer: \(120(0.50)+280(0.20)=60+56=116\) reviewed; no, unreviewed status is unknown.)
- In a randomized pilot, an offer reduces an identified group’s churn rate from 0.30 without the offer to 0.18 with it. For 250 customers all offered the benefit, calculate expected churn under the offer and without it, and the difference. (Answer: \(250(0.18)=45\) under the offer; \(250(0.30)=75\) without it; 30 fewer expected churns under the offer.)
- A population has 100 high-score and 400 low-score claims. Their true fraud prevalences are 20% and 5%. The review rates are 75% and 25%, and review sensitivity is 80%, with no false positives. Calculate expected confirmed positives and explain why that count is not total fraud. (Answer: reviewed fraud \(=100(0.20)(0.75)+400(0.05)(0.25)=15+5=20\); confirmed positives \(=20(0.80)=16\). Total true fraud is \(20+20=40\), so 16 is not total fraud because some fraud claims are unreviewed or missed.)
- A policy assigns an intervention based on a predictor, but staff also use unrecorded judgment related to likely outcomes when choosing cases. Is consistency alone enough to justify weighting the marginal action-specific potential-outcome means by the policy’s action probabilities? (Answer: No. Conditional independence of action and potential outcomes given recorded predictors is also needed; unrecorded judgment can violate it.)