Forecasting Retention When the Portfolio Mix Changes
A portfolio’s renewal rate can change for two distinct reasons. Renewal probabilities may shift for policies with comparable characteristics, or the portfolio may contain different proportions of products, tenure groups, channels, or price-change categories. A sound forecast distinguishes these effects rather than treating an aggregate rate as if it described policyholder behavior alone.
The starting point is a set of mutually exclusive, collectively exhaustive segments. For each segment, use its share of eligible policies and its conditional renewal probability. The resulting rate is an average weighted by the portfolio being forecast. This makes the choice of weights central: applying today’s probabilities to tomorrow’s mix answers a different question from applying them to today’s mix.
A segment should be defined using characteristics that support the forecast being made. For example, product, tenure, channel, and price-change category can be combined into joint segments. If two portfolios have the same channel shares but different price-change distributions within each channel, channel totals alone will not represent the change in composition. The model probabilities and mix weights must also refer to the same eligibility definition, renewal period, and segment boundaries.
Separating Probability Changes and Mix Changes
Write \(q_{0j}\) and \(p_{0j}\) for the baseline segment shares and probabilities, and \(q_{1j}\) and \(p_{1j}\) for the forecast shares and probabilities. The baseline rate is \(r_0=\sum_j q_{0j}p_{0j}\); the forecast rate is \(r_1=\sum_j q_{1j}p_{1j}\). A useful sequential decomposition first changes probabilities while holding the baseline mix fixed, then changes the mix while holding forecast probabilities fixed.
This identity is exact. It is also a convention about attribution: the interaction between changing probabilities and changing weights is assigned to the composition effect because the second step uses \(p_{1j}\). An alternative sequence changes the mix first and evaluates probability shifts at the new mix. Either sequence gives the same total change, but can assign different amounts to the two components. State which convention is being used when reporting a decomposition.
Define which policies are exposed to renewal and whether the baseline and forecast counts refer to the same period and coverage of policies.
Check that the segment shares are nonnegative, sum to one, and describe the portfolio whose rate is being calculated.
Multiply each segment’s share by its conditional probability, then add the products to obtain the portfolio rate.
Apply one stated decomposition convention, then multiply the rate by eligible policy count to obtain expected renewals.
Why the Decomposition Holds
The result follows by adding and subtracting the rate that would result from applying forecast probabilities to baseline weights. In each segment, the change in the weighted contribution can be written as a probability change at the baseline weight plus a weight change at the forecast probability.
The expected renewal count then follows from the expected value of a sum. If \(I_k\) equals 1 when eligible policy \(k\) renews and 0 otherwise, then \(R=\sum_k I_k\), so \(E[R]=\sum_k E[I_k]=\sum_k P(I_k=1)\). Grouping policies by segment gives \(E[R]=\sum_j N_jp_j=Nr\), where \(N_j=Nq_j\). Dependence among policy outcomes would matter for the variance of \(R\), but not for this expectation.
Worked Example: A Two-Segment Mix Shift
A hypothetical insurer has 20,000 eligible policies. At baseline, 60% are in Segment A and 40% in Segment B, with renewal probabilities of 90% and 70%, respectively. For the forecast, the shares are 45% and 55%, and the segment probabilities are 92% and 68%. Calculate both rates, expected renewal counts, and the rate-change decomposition using baseline weights for the probability effect.
Method and segment check. The two segments are exhaustive: the baseline shares total \(0.60+0.40=1.00\), and the forecast shares total \(0.45+0.55=1.00\). First calculate each weighted contribution.
Baseline: Segment A contributes \(0.60(0.90)=0.540\), and Segment B contributes \(0.40(0.70)=0.280\). Thus \(r_0=0.540+0.280=0.820\), or 82%.
Forecast: Segment A contributes \(0.45(0.92)=0.414\), and Segment B contributes \(0.55(0.68)=0.374\). Thus \(r_1=0.414+0.374=0.788\), or 78.8%.
Expected renewals are \(20{,}000(0.820)=16{,}400\) at baseline and \(20{,}000(0.788)=15{,}760\) in the forecast, a decrease of 640.
Decomposition. At the baseline mix, the probability effect is \(0.60(0.92-0.90)+0.40(0.68-0.70)=0.012-0.008=0.004\), an increase of 0.4 percentage points. The composition effect, evaluated at forecast probabilities, is \((0.45-0.60)(0.92)+(0.55-0.40)(0.68)=-0.138+0.102=-0.036\), a decrease of 3.6 percentage points. The effects sum to \(0.004-0.036=-0.032\), matching \(0.788-0.820=-0.032\). On 20,000 policies, these components are \(+80\) and \(-720\) expected renewals, totaling \(-640\).
Check and mistake avoided. The forecast rate is lower even though the probability effect is positive: the less favorable composition effect is larger. Applying forecast probabilities to baseline shares would produce 82.4%, a useful intermediate rate, not the actual forecast rate. Treating that intermediate rate as the forecast would use the wrong mix.
Joint Mixes and Conditional Probabilities
The same logic applies when segments combine several characteristics. If the forecast changes the distribution of product, tenure, and channel together, use joint cells when the needed probabilities are conditional on those combinations. Replacing joint shares with separate marginal shares can lose information: in general, a product share and a channel share do not determine the share of each product-channel combination.
Worked Example: Product and Tenure Mix
A hypothetical portfolio has 20,000 eligible policies divided into four joint product-tenure segments. Baseline counts are 4,800, 3,200, 7,200, and 4,800 in the order shown below. Baseline probabilities are 88%, 78%, 82%, and 68%. Forecast shares are 20%, 20%, 35%, and 25%, with forecast probabilities of 86%, 80%, 79%, and 70%. Calculate the rates, expected counts, and sequential decomposition.
| Joint segment | Baseline share | Forecast share | Baseline probability | Forecast probability |
|---|---|---|---|---|
| Product A, newer tenure | 0.24 | 0.20 | 0.88 | 0.86 |
| Product A, longer tenure | 0.16 | 0.20 | 0.78 | 0.80 |
| Product B, newer tenure | 0.36 | 0.35 | 0.82 | 0.79 |
| Product B, longer tenure | 0.24 | 0.25 | 0.68 | 0.70 |
Method and weight check. Baseline shares sum to \(0.24+0.16+0.36+0.24=1.00\); forecast shares sum to \(0.20+0.20+0.35+0.25=1.00\). The stated baseline counts also sum to \(4{,}800+3{,}200+7{,}200+4{,}800=20{,}000\), consistent with the total.
The baseline rate is \(0.24(0.88)+0.16(0.78)+0.36(0.82)+0.24(0.68)=0.2112+0.1248+0.2952+0.1632=0.7944\). The forecast rate is \(0.20(0.86)+0.20(0.80)+0.35(0.79)+0.25(0.70)=0.1720+0.1600+0.2765+0.1750=0.7835\).
Expected renewals are \(20{,}000(0.7944)=15{,}888\) at baseline and \(20{,}000(0.7835)=15{,}670\) in the forecast. The change is \(15{,}670-15{,}888=-218\).
For the probability effect, apply baseline weights to probability changes: \(0.24(-0.02)+0.16(0.02)+0.36(-0.03)+0.24(0.02)=-0.0048+0.0032-0.0108+0.0048=-0.0076\). For the composition effect, apply share changes to forecast probabilities: \((-0.04)(0.86)+(0.04)(0.80)+(-0.01)(0.79)+(0.01)(0.70)=-0.0344+0.0320-0.0079+0.0070=-0.0033\). Their sum is \(-0.0109\), exactly \(0.7835-0.7944\). Multiplying by 20,000 gives \(-152\) and \(-66\) expected renewals, totaling \(-218\).
Check and mistake avoided. The baseline mix is computed from baseline counts, while the forecast mix is supplied separately. Reusing the baseline shares for the final forecast would answer a fixed-mix question and miss the composition change.
When Aggregate Categories Conceal Mix Shifts
A portfolio can have unchanged shares in a broad category yet change substantially within that category. This often occurs when channel totals stay constant but the distribution of price-change categories within each channel shifts. Joint segmentation allows the calculation to reflect that movement. The probability effect and composition effect should be interpreted relative to the chosen segment definitions; a different grouping can change their attribution even when the overall rates remain unchanged.
Worked Example: Stable Channel Shares, Changing Price-Change Mix
A hypothetical portfolio has 10,000 eligible policies in four cells: Direct channel with low or high price change, and Broker channel with low or high price change. Baseline shares are 40%, 10%, 20%, and 30% in that order; forecast shares are 25%, 25%, 35%, and 15%. Baseline probabilities are 94%, 75%, 90%, and 68%; forecast probabilities are 92%, 70%, 88%, and 62%. Calculate the portfolio rates and decompose the change using baseline weights for the probability effect.
Method and marginal-share check. Both sets of four shares sum to 100%. Direct’s total share is \(0.40+0.10=0.50\) at baseline and \(0.25+0.25=0.50\) in the forecast. Broker’s share is also 0.50 in both periods. The joint price-change mix nevertheless differs within each channel.
The baseline rate is \(0.40(0.94)+0.10(0.75)+0.20(0.90)+0.30(0.68)=0.376+0.075+0.180+0.204=0.835\). The forecast rate is \(0.25(0.92)+0.25(0.70)+0.35(0.88)+0.15(0.62)=0.230+0.175+0.308+0.093=0.806\).
Expected renewals are \(10{,}000(0.835)=8{,}350\) at baseline and \(10{,}000(0.806)=8{,}060\) in the forecast, a decrease of 290.
The probability effect is \(0.40(-0.02)+0.10(-0.05)+0.20(-0.02)+0.30(-0.06)=-0.008-0.005-0.004-0.018=-0.035\). The composition effect is \((-0.15)(0.92)+(0.15)(0.70)+(0.15)(0.88)+(-0.15)(0.62)=-0.138+0.105+0.132-0.093=0.006\). Together, \(-0.035+0.006=-0.029=0.806-0.835\), or \(-350+60=-290\) expected renewals.
Check and mistake avoided. Looking only at channel totals would suggest there is no channel composition change, but that overlooks shifts between low- and high-price-change policies within channels. The four-cell calculation captures those shifts. The composition effect is positive here because the net movement across the joint cells favors cells with higher forecast probabilities; it does not mean that every change in mix improves retention.
Common Forecasting Traps
- Using unweighted averages. Averaging segment probabilities equally is correct only when the segment shares are equal. The portfolio rate uses the actual shares.
- Mixing denominators. Shares based on all in-force policies cannot be combined directly with probabilities defined only for policies eligible to renew. Align the populations first.
- Using old weights for the final forecast. This produces a fixed-mix comparison, not a forecast for a portfolio whose composition changes.
- Ignoring joint composition. Stable marginal shares, such as stable channel totals, do not imply stable combinations of channel and price-change category.
- Reporting an unattributed interaction without a convention. The total rate change is unique, but a sequential split assigns the interaction according to the order of the calculation. State the convention and verify that the components sum to the total.
- Confusing a rate with a count. A lower forecast rate can still produce more expected renewals if the eligible portfolio grows enough. Forecast the rate and eligible count separately.
Check Your Understanding
For each problem, use the stated segment shares as weights and check that they sum to one where applicable.
- A hypothetical portfolio has baseline shares of 0.70 and 0.30, with probabilities of 0.95 and 0.65. Forecast shares are 0.50 and 0.50, with probabilities of 0.94 and 0.70. For 2,500 eligible policies, find baseline and forecast expected renewals and the change. (Answer: Baseline rate \(0.860\), or 2,150 expected renewals; forecast rate \(0.820\), or 2,050; change \(-100\) expected renewals.)
- Baseline shares are 0.20 and 0.80, with probabilities 0.80 and 0.60. Forecast shares are 0.40 and 0.60, with probabilities 0.75 and 0.65. Using baseline weights for the probability effect and forecast probabilities for the composition effect, calculate both effects and their total for 8,000 policies. (Answer: Probability effect \(+0.030\); composition effect \(+0.020\); total \(+0.050\), or \(+400\) expected renewals.)
- A portfolio grows from 6,000 to 6,600 eligible policies. Its rate falls from 0.75 to 0.70. Calculate expected renewals at each date and the change. (Answer: 4,500 at baseline and 4,620 in the forecast; an increase of 120 expected renewals despite the rate decline.)
- A portfolio has Direct-low, Direct-high, Broker-low, and Broker-high cells. Baseline shares are \(0.30, 0.20, 0.10, 0.40\); forecast shares are \(0.20, 0.30, 0.30, 0.20\). The probabilities are unchanged at \(0.90, 0.60, 0.85, 0.50\). For 10,000 policies, calculate the rates and explain what channel-only shares would conceal. (Answer: Baseline rate \(0.675\); forecast rate \(0.715\); 6,750 versus 7,150 expected renewals, an increase of 400. Direct and Broker shares each remain 0.50, concealing the within-channel price-change mix shifts.)
- Baseline and forecast probabilities in two segments are both \(0.80\) and \(0.60\). Shares change from \(0.50,0.50\) to \(0.25,0.75\). For 4,000 eligible policies, find the rate change and explain its source. (Answer: The rate falls from \(0.700\) to \(0.650\), a decrease of 0.050 or 200 expected renewals; the change is entirely composition because segment probabilities are unchanged.)