Two Different Questions About an Outcome
A predictive analysis asks what outcome is likely for a case, given information available when a decision must be made. An explanatory analysis asks why an outcome occurred, or what role particular factors played in producing it. These questions can concern the same outcome and use some of the same data, but they are not interchangeable.
For example, an insurer might want to predict whether a policy will have a claim next year using information recorded at renewal. The insurer might instead want to understand why claim frequency increased in one region. A model can help forecast claims without identifying the reason for the regional increase. Keeping the question clear matters: it affects how model results should be described and what conclusions are justified.
A predictor can be useful because it is associated with the target, even if changing that predictor would not change the outcome. It may be a proxy for an unmeasured factor, a consequence of earlier events, or simply a feature that tends to co-occur with the target. Predictive performance answers whether the predictions are useful for the specified task; it does not, on its own, answer why the outcomes occurred.
| Question | Typical output | What the output supports |
|---|---|---|
| Will this policy have a claim during the next year? | A probability for the policy | An estimate of future claim likelihood, given available information |
| Why did claims increase in this region? | An account of factors associated with the observed increase | A description of the pattern; causal conclusions require additional justification |
| What claim amount should be budgeted for this group? | An expected amount or an aggregate forecast | A forecast for the defined group and time period |
Prediction Uses Information; Explanation Interprets It
A prediction is tied to a target, a prediction time, and information available at that time. If a feature is recorded only after the outcome, it cannot be used for a genuine forecast made earlier. Even when a feature is available at the right time, its predictive role should not automatically be described as a reason for the outcome.
Explanation requires careful interpretation of how the data were produced and what question is being asked. A model’s coefficient, a difference between group averages, or a high feature-importance score describes a statistical relationship under the modeling setup. None of these quantities automatically says what would happen if someone changed the feature while other relevant conditions were held appropriately fixed. The distinction between predictive association and causal effect is important; the central point here is that predictive success alone does not resolve it.
Worked Example: Predicting Claims From Length of Stay
A hypothetical health insurer studies 200 completed episodes to develop a preliminary rule for forecasting whether an episode will be followed by a claim during a fixed outcome window. The recorded length of stay is available at the prediction time. Among 80 short-stay episodes, 8 are followed by a claim; among 120 long-stay episodes, 24 are followed by a claim. Calculate the observed claim rates, use them to make a simple forecast for a portfolio with 50 episodes in each group, and state what the results do and do not establish.
Step 1: Calculate the rates. For short stays, the rate is \(8/80=0.10\), or 10%. For long stays, it is \(24/120=0.20\), or 20%. The observed difference is \(0.20-0.10=0.10\), or 10 percentage points.
Step 2: Apply the simple rule to the new portfolio. The forecast for the 50 short-stay episodes is \(50(0.10)=5\) claims. The forecast for the 50 long-stay episodes is \(50(0.20)=10\) claims. The total forecast is \(5+10=15\) claims among 100 episodes, or 15%.
Step 3: Interpret the result. Length-of-stay group is associated with different observed claim rates in these data and could be useful for a forecast if the pattern carries over to future cases. The calculation does not show that a longer stay causes a claim. Severity, treatment, or other features could be related both to length of stay and subsequent claims. The rates also come from the stated data; their usefulness for future cases should not be assumed without appropriate evaluation.
Method and mistake avoided. The method is a group-rate forecast: multiply each group’s predicted rate by its number of cases and add. A common mistake is to interpret the 10-percentage-point difference as the effect of making a stay longer. It is an observed rate difference, not by itself a causal result.
A Prediction Can Be Useful Without Explaining Why
Suppose a model uses the number of customer-service contacts recorded before renewal to predict whether a policy will lapse. The contact count might help identify cases with elevated lapse probabilities, allowing the insurer to forecast expected lapses for planning. But the count need not explain why a particular policy lapses. It could be a signal of dissatisfaction, administrative complexity, or another circumstance. It could even partly reflect an earlier event that is itself related to lapse.
This distinction guides careful wording. “Policies with more recorded contacts had higher predicted lapse probabilities” is a predictive statement. “Additional contacts caused those policies to lapse” is a causal claim and is not justified simply because the contact count helped a prediction. The first statement can be useful to a decision-maker without making the second.
Worked Example: Checking Predictive Performance Without Claiming an Explanation
A hypothetical renewal model assigns lapse probabilities to four policies based on pre-renewal records. The predicted probabilities are 0.8, 0.6, 0.3, and 0.1. The observed lapse indicators are 1, 0, 1, and 0, where 1 means the policy lapsed. Calculate the Brier score and compare it with a constant prediction equal to the observed lapse proportion. Explain the limits of this comparison.
Step 1: Calculate the observed proportion. Two of the four policies lapsed, so the proportion is \(2/4=0.5\). The constant benchmark therefore assigns probability 0.5 to each policy.
Step 2: Calculate squared errors for the model. The four errors squared are \((0.8-1)^2=0.04\), \((0.6-0)^2=0.36\), \((0.3-1)^2=0.49\), and \((0.1-0)^2=0.01\). Their sum is \(0.04+0.36+0.49+0.01=0.90\). The Brier score, the mean squared difference between predicted probabilities and binary outcomes, is \(0.90/4=0.225\).
Step 3: Calculate the benchmark score. Each prediction of 0.5 has squared error \((0.5-1)^2=0.25\) for a lapse and \((0.5-0)^2=0.25\) for a non-lapse. The mean is \((0.25+0.25+0.25+0.25)/4=0.25\). In this four-policy illustration, the model’s score is lower by \(0.25-0.225=0.025\).
Step 4: State the conclusion carefully. On these four cases, the model has a lower Brier score than the constant benchmark. This small comparison does not establish future performance or explain why any policy lapsed. The contact information may help forecast lapse while remaining only a predictive signal.
Method and mistake avoided. The method is to average the squared probability errors and compare like with like. A common mistake is to treat a better score as evidence that a model feature caused the outcomes. The score concerns prediction error, not explanation.
Squared-Error Prediction and the Conditional Mean
For a numerical target, a point prediction often uses squared error as the loss: predicting \(a\) when the outcome is \(Y\) incurs \((Y-a)^2\). At a particular information state \(X=x\), the best constant prediction under this loss is the conditional mean, provided the conditional second moment is finite. A finite conditional mean alone is not enough to justify this result.
To see why the result holds, write \(m=E[Y\mid X=x]\). The finite conditional second moment ensures that the conditional variance and squared-error risk are finite. Add and subtract \(m\), then expand the square:
The middle term is zero because \(E[Y-m\mid X=x]=0\). The variance term does not depend on \(a\), and \((m-a)^2\) is minimized at \(a=m\). This proves the result. If the conditional second moment is infinite, this finite-risk decomposition cannot be used as stated; a finite conditional mean by itself is insufficient.
Worked Example: Choosing a Squared-Error Forecast
For cases with a specified information value \(X=x\), suppose the hypothetical outcome \(Y\), measured in thousands of dollars, has the following conditional distribution: \(Y=0\) with probability 0.25, \(Y=10\) with probability 0.50, and \(Y=20\) with probability 0.25. Find the squared-error-optimal forecast, calculate its conditional risk, and compare it with a forecast of 12. The distribution has finite support, so its conditional second moment is finite.
Step 1: Find the conditional mean. The mean is \(0(0.25)+10(0.50)+20(0.25)=0+5+5=10\). Therefore the squared-error-optimal forecast is 10 thousand dollars.
Step 2: Find the conditional second moment and variance. The second moment is \(0^2(0.25)+10^2(0.50)+20^2(0.25)=0+50+100=150\), in squared thousands of dollars. The variance is \(150-10^2=50\).
Step 3: Calculate the risk for each forecast. For forecast 10, the conditional expected squared error is \(50+(10-10)^2=50\). For forecast 12, it is \(50+(12-10)^2=50+4=54\). Thus 10 has lower conditional squared-error risk.
Step 4: Check the calculation directly. For forecast 10, the errors squared are 100, 0, and 100, giving \(0.25(100)+0.50(0)+0.25(100)=50\). For forecast 12, they are 144, 4, and 64, giving \(0.25(144)+0.50(4)+0.25(64)=36+2+16=54\). The direct calculations agree.
Method and mistake avoided. The method is to use the conditional mean and the variance-plus-squared-bias decomposition. A common mistake is to use this result when only a finite mean is known. The condition needed here is a finite conditional second moment, so the conditional squared-error risk is finite.
Common Exam Trap: Turning Association Into an Explanation
Suppose a hypothetical portfolio has lapse rates of 30% among policies with multiple service contacts and 10% among policies with no recorded contacts. The arithmetic difference is \(0.30-0.10=0.20\), or 20 percentage points.
Wrong conclusion: “Multiple contacts increased the lapse probability by 20 percentage points.” This treats an observed group difference as though it were the effect of changing the contact count.
Correction: “The observed lapse rate was 20 percentage points higher in the multiple-contact group in this data set.” That statement describes the data accurately. The difference may be useful for prediction, but the comparison alone does not establish why the groups differ or what would happen if contact counts were changed.
When answering a question, first identify whether the requested result is a forecast or an explanation. For a forecast, specify the target, prediction time, and available inputs, then interpret performance in that context. For an explanation, do not present predictive association as a sufficient account of why the outcome occurred.
Check Your Understanding
For each problem, identify the appropriate interpretation and complete the requested calculation.
- A hypothetical portfolio has 40 claims among 200 policies in one risk-score group and 18 claims among 150 policies in another. Calculate both observed claim rates and their difference. Does the difference alone show that the score caused claims? (Answer: The rates are 20% and 12%, a difference of 8 percentage points; no, it is an observed association, not proof of causation.)
- A model assigns probabilities 0.7, 0.4, 0.2, and 0.9 to four cases whose binary outcomes are 1, 0, 0, and 1. Calculate the Brier score. (Answer: \((0.09+0.16+0.04+0.01)/4=0.075\).)
- At an information state \(X=x\), \(Y\) equals 2 with probability 0.75 and 6 with probability 0.25. Find the squared-error-optimal forecast and its conditional risk. (Answer: The forecast is \(0.75(2)+0.25(6)=3\); the variance and minimum conditional squared-error risk are \(0.75(2-3)^2+0.25(6-3)^2=3\).)
- A model uses the number of repair inquiries filed before a vehicle policy’s renewal date to predict a claim in the next year. It scores better than a constant-rate benchmark on a test set. State one justified conclusion and one conclusion not justified by that result. (Answer: The inquiry count helps predict claims on that test set relative to the benchmark; the score does not show that filing inquiries causes claims.)
- At a fixed information state, \(E[Y\mid X=x]\) is finite but \(E[Y^2\mid X=x]\) is infinite. Can the conditional-mean squared-error result be applied as stated? (Answer: No; a finite conditional mean alone does not ensure finite conditional squared-error risk or justify the result.)