Why Recalibrate Renewal Predictions?
A renewal model can rank policies well yet systematically overstate or understate their renewal probabilities. For example, policies assigned probabilities near 0.80 might renew less often than that. This is a calibration problem: predicted probabilities do not agree with observed outcome frequencies. Recalibration adjusts the probability scale using outcomes from a recent validation period, while leaving the original predictor model intact.
The validation period should represent the renewal setting where the corrected probabilities will be used. The original predictions must have been generated without using those validation outcomes to fit the underlying model. Recalibrating on a distinct validation period avoids evaluating the correction on the same observations that determined it; a final assessment on new data is still needed to judge how well it generalizes.
Calibration is distinct from ranking. Ranking concerns whether policies with higher predicted probabilities tend to renew more often than policies with lower predicted probabilities. A common recalibration adds the same amount to every predicted log-odds, changing probability levels while preserving their ordering. Segment checks matter because a portfolio-wide correction can conceal overprediction in one group and underprediction in another.
Intercept Adjustment: Correcting the Overall Level
For an intercept adjustment, keep each original prediction’s logit slope fixed at one and add a common intercept \(a\). The logit of a probability \(p\) is \(\log(p/(1-p))\); its inverse, \(\operatorname{expit}(z)=1/(1+e^{-z})\), converts a log-odds value back to a probability.
This is a calibration-in-the-large adjustment: it shifts the overall probability level without changing the model’s relative log-odds contrasts. A positive \(a\) raises every probability; a negative \(a\) lowers every probability. It is not a universal correction for every kind of miscalibration. If high predicted probabilities are too high while low predicted probabilities are too low, changing only the intercept may not be enough.
Calculate the observed renewal proportion, \(\sum_iY_i/n\), and compare it with the average original prediction.
Solve the intercept equation using the original probabilities and the observed renewal count. For unequal predictions, numerical root-finding is generally needed.
Recalculate the adjusted average and inspect calibration within meaningful segments, not only across the full sample.
Why the Intercept Equation Works
Suppose validation outcomes are conditionally independent Bernoulli variables, with adjusted renewal probabilities \(p_i^*(a)\). The Bernoulli log-likelihood for the intercept is \(\ell(a)=\sum_i[Y_i\log p_i^*(a)+(1-Y_i)\log(1-p_i^*(a))]\). Because the derivative of the logistic probability with respect to its linear predictor is \(p_i^*(a)\{1-p_i^*(a)\}\), differentiating the log-likelihood gives \(\ell'(a)=\sum_i\{Y_i-p_i^*(a)\}\). At an interior maximum, this derivative is zero, so \(\sum_i p_i^*(a)=\sum_iY_i\). Thus the intercept maximum-likelihood estimate makes the adjusted average match the observed renewal proportion.
The solution is unique when the validation sample contains both outcomes. Indeed, \(\ell''(a)=-\sum_i p_i^*(a)\{1-p_i^*(a)\}<0\), so the log-likelihood is strictly concave. Equivalently, the sum of adjusted probabilities increases continuously with \(a\), from zero toward \(n\). This also explains why the equation can be solved reliably by one-dimensional numerical methods.
Worked Example: Lowering an Overstated Portfolio Probability
A hypothetical insurer has 1,000 policies with renewal decisions in one annual period. Each original prediction is \(0.8000\), while 760 policies renew. Assume each outcome is Bernoulli with the stated probability, conditional independence for the calibration likelihood, and no interest or mortality assumption: this is a probability exercise, not a present-value calculation. Estimate the intercept adjustment and check the corrected level.
Method. Use an intercept adjustment with slope fixed at one. The observed renewal proportion is \(760/1{,}000=0.7600\). The original log-odds are \(\log(0.8/0.2)=\log 4\). Since all predictions are equal, setting the adjusted probability to the observed proportion solves the intercept equation directly: \(\log 4+a=\log(0.76/0.24)=\log(19/6)\).
Therefore \(a=\log(19/24)\), approximately \(-0.2336\). The adjusted odds are \(4e^a=4(19/24)=19/6\). Converting these odds to a probability gives \((19/6)/(1+19/6)=19/25=0.7600\).
The adjusted average is \(0.7600\), matching \(760/1{,}000\), as required. The adjustment reduces the original probability by 0.0400. Common mistake avoided: subtracting 0.0400 from the log-odds is not the intercept solution. The 0.0400 difference is on the probability scale; the correct log-odds shift is approximately \(-0.2336\).
Recalibration With an Intercept and Slope
An intercept shift assumes the original log-odds contrasts are already appropriately scaled. A more flexible calibration model estimates both a new intercept \(a\) and a slope \(b\): \(\operatorname{logit}\{P(Y_i=1)\}=a+b\operatorname{logit}(p_i)\). The parameters are estimated by logistic regression of the validation outcomes on the original predicted logits. The original renewal model is not refit; this model adjusts its probability scale.
Worked Example: Correcting Overly Extreme Predictions
A hypothetical annual renewal validation sample has two equally sized groups of 100 policies. The first group has original predicted probability \(0.25\) for each policy and 40 renewals; the second has predicted probability \(0.75\) and 60 renewals. Treat individual outcomes as Bernoulli, conditionally independent for fitting the calibration model, with no interest or mortality assumption. Fit the two-parameter logistic recalibration model to these grouped outcomes and check its segment predictions.
Method. The observed rates are \(40/100=0.40\) and \(60/100=0.60\). The original logits are \(\operatorname{logit}(0.25)=-\log3\) and \(\operatorname{logit}(0.75)=\log3\). The recalibration model is \(\operatorname{logit}(q)=a+b\operatorname{logit}(p)\). With two distinct original logits and two parameters, the fitted probabilities match the two observed group rates.
The observed group logits are \(\operatorname{logit}(0.40)=\log(2/3)=-\log1.5\) and \(\operatorname{logit}(0.60)=\log1.5\). The two fitted equations are therefore \(-a-b\log3=-\log1.5\) and \(a+b\log3=\log1.5\). Adding them gives \(2a=0\), so \(a=0\). Substitution gives \(b=\log1.5/\log3\), approximately \(0.3691\).
The recalibrated probabilities are \(\operatorname{expit}(-0.3691\log3)=0.4000\) and \(\operatorname{expit}(0.3691\log3)=0.6000\). They match the observed rates in both groups, and \(b>0\) preserves the ranking. Common mistake avoided: using only an intercept shift cannot match both groups here. The observed rates require different changes to the original logits; the fitted slope below one compresses their overly wide separation.
Check Calibration Across Segments
A global correction answers whether the overall level is right, not whether every part of the portfolio is right. Useful segments are defined before inspecting results and have enough observations to make comparisons interpretable. Depending on the application, they may reflect price-change bands, tenure, product, or another meaningful characteristic. For each segment, compare the mean recalibrated probability with the observed renewal proportion. Look for persistent differences, not just a single small discrepancy that could arise from sampling variation.
If a single global intercept improves the total but leaves meaningful segment gaps, investigate whether a segment-specific correction is justified and supported by enough validation experience. Such corrections add parameters and can overfit small groups. They can also change the relative ordering of policies in different segments, even if each segment’s internal ordering is preserved. A more complex recalibration should therefore be treated as a modeling choice, not an automatic response to every observed difference.
For the intercept-and-slope model, inspect segment performance after applying the fitted transformation. A positive slope preserves the global ordering, but neither that property nor an improved overall average guarantees segment calibration. Conversely, a global adjustment that matches the portfolio-wide renewal rate can still leave one segment overpredicted and another underpredicted, with the errors canceling in aggregate.
Worked Example: A Global Correction That Hides Segment Errors
A hypothetical annual validation sample contains 100 policies in each of two segments. Original predictions are \(0.60\) in Segment A and \(0.40\) in Segment B. There are 50 renewals in A and 30 in B. Assume Bernoulli outcomes and conditional independence for the calibration likelihood; no interest or mortality assumption is relevant. Find the global intercept adjustment and assess whether it fixes each segment.
Method. The total observed renewal count is \(50+30=80\) out of 200, so the observed overall rate is \(0.4000\). At \(a=0\), the total predicted count is \(100(0.60)+100(0.40)=100\). The intercept must lower the combined probabilities to 80.
The solution is \(a=\log(0.5)\). To verify it, the original odds for A are \(0.60/0.40=1.5\); adjusted odds are \(1.5(0.5)=0.75\), giving adjusted probability \(0.75/(1+0.75)=3/7\), or \(0.4286\). For B, the original odds are \(0.40/0.60=2/3\); adjusted odds are \(1/3\), giving probability \(1/4=0.2500\).
The adjusted total is \(100(3/7)+100(1/4)=300/7+25=475/7\), approximately 67.86, not 80. Thus \(a=\log(0.5)\) is not the solution; it would be incorrect to infer an odds multiplier from the overall rate ratio. To obtain the actual intercept, solve \(100\operatorname{expit}\{\operatorname{logit}(0.60)+a\}+100\operatorname{expit}\{\operatorname{logit}(0.40)+a\}=80\).
Because the original probabilities are complementary, the adjusted probabilities also sum to one for every policy pair: if \(p_B=1-p_A\), then \(\operatorname{expit}(\operatorname{logit}(p_B)+a)=1-\operatorname{expit}(\operatorname{logit}(p_A)+a)\). The total predicted count therefore remains 100 for every finite \(a\). No intercept can match 80. This occurs because the two equal-sized segments have complementary original logits, while the observed total is below 100; the one-parameter model cannot change that total. The segment rates, 0.5000 and 0.3000, also show opposite calibration errors. Common mistake avoided: assuming a global intercept must be able to match any observed portfolio rate. Check whether a solution exists before interpreting a failed or unstable fit; segment or slope misspecification may be responsible.
Practical Checks and Exam Traps
- Do not multiply probabilities by an observed-to-predicted ratio and call the result an intercept adjustment. The intercept acts on log-odds, not directly on probabilities; direct multiplication can even produce values above one.
- Do not confuse a level error with a slope error. A common shift can correct an overall offset, but it cannot generally fix probabilities that are too extreme or not extreme enough across their range.
- Do not stop at the portfolio-wide average. Compare observed and adjusted rates by relevant segments; opposing errors can cancel in the aggregate.
- Do not assume a fitted correction will persist. Validation results are empirical, and recalibration can overfit a small sample. Retain a separate assessment period where practical.
- Check feasibility and ordering. Some data patterns cannot be matched by an intercept-only adjustment, and a negative recalibration slope reverses the original ranking.
Check Your Understanding
For each hypothetical annual validation sample, identify the appropriate calculation and check what the corrected rates imply. Treat outcomes as Bernoulli; interest and mortality assumptions are not applicable to these probability-calibration problems.
- There are 500 policies, each originally assigned probability \(0.70\), and 325 renewals. Find the intercept shift and adjusted probability. (Answer: \(a=\log(13/14)\approx-0.0741\); adjusted probability \(0.6500\).)
- Two equal-sized groups of 80 policies have original probabilities \(0.50\) and \(0.80\), and observed renewal counts 24 and 48. Find the common intercept adjustment. (Answer: \(a=\log(0.5)\approx-0.6931\); the adjusted group probabilities are \(1/3\) and \(2/3\).)
- A fitted recalibration model is \(\operatorname{logit}(q)=0.10+0.80\operatorname{logit}(p)\). Find the recalibrated probability for an original \(p=0.75\), and state whether its ordering relative to lower original probabilities is preserved. (Answer: \(q\approx0.7090\); yes, because the slope \(0.80\) is positive.)
- Two equal-sized segments have original probabilities \(0.60\) and \(0.40\), with observed renewal rates \(0.55\) and \(0.35\). Can an intercept adjustment match the overall observed rate? What segment rates should be inspected? (Answer: Yes; the observed overall rate is \(0.4500\), and the common intercept \(a=\log(0.75)\) changes the segment probabilities to \(0.5294\) and \(0.3711\).)
- A recalibration fit on one recent validation period matches its overall observed renewal proportion, but the lower-price-change segment is overpredicted and the higher-price-change segment is underpredicted. What does this imply about the global correction? (Answer: It corrects the aggregate level but does not establish segment calibration; the segment-specific discrepancies require investigation rather than assuming the global shift is adequate.)