Selection and Churn After Modification Changes

Why Selection Matters After a Modification Change

An experience modification can affect more than the premium or rating factor assigned to an account. After a change, some insureds may renew, some may leave, and new insureds may enter. If those decisions are related to risk characteristics or to the modification itself, the portfolio observed after the change is not simply the original portfolio under a new rule.

This creates a selection problem. A comparison of post-change renewals with the earlier portfolio may combine two effects: a change in outcomes within comparable risks and a change in which risks remain observable. The second effect can make a plan appear to improve or worsen even when the underlying outcomes within risk groups have not changed.

Definition: Selection after a modification change occurs when renewal, nonrenewal, or entry is associated with characteristics that also relate to measured experience, premiums, or modifications. Churn is the movement of insureds into or out of the evaluated portfolio over time. A renewal-only analysis conditions on remaining in the portfolio; it does not automatically represent the original cohort.

The relevant unit is usually the account or policy-year, but the evaluation must state its unit and denominator explicitly. A result per account, per exposure unit, and per dollar of indicated premium can each answer a different question. A renewal rate is also a separate outcome from claim cost or premium adequacy.

Separate Performance From Portfolio Composition

Suppose the evaluator divides risks into groups $g$, such as modification bands or stable risk categories. Let $w_{gt}$ be group $g$'s share of the evaluated portfolio at time $t$, and let $\mu_{gt}$ be its average outcome. The portfolio average is a weighted average:

R_t=\sum_g w_{gt}\mu_{gt}, \qquad \sum_g w_{gt}=1.

An aggregate result can change because group outcomes change, because group shares change, or both. One exact decomposition is obtained by adding and subtracting $\sum_g w_{g0}\mu_{g1}$:

R_1-R_0 =\sum_g w_{g0}(\mu_{g1}-\mu_{g0}) +\sum_g (w_{g1}-w_{g0})\mu_{g1}.

The first term measures within-group changes using the initial group weights. The second measures the effect of changed group shares, valued at the later group outcomes. The decomposition is an accounting identity, not proof that either component is causal. It requires consistent group definitions and comparable outcome measurement at both times.

For an experience-rating evaluation, $\mu$ might be average loss, a loss-to-indicated-premium ratio, or a modification. The interpretation depends on the chosen measure. In particular, a lower loss-to-premium ratio among renewals is not by itself evidence that the modification change reduced losses; renewal selection may have altered the mix.

Worked Example: Renewal-Only Loss-to-Premium Ratio

A hypothetical portfolio is divided into two stable risk groups. Before a modification change, the full cohort has indicated premium of $130,000 and losses of $104,000 in Group L, and indicated premium of $80,000 and losses of $96,000 in Group H. Among the accounts that renew, Group L contributes $89,000 of indicated premium and $71,200 of losses; Group H contributes $9,000 of indicated premium and $10,800 of losses. Calculate the full-cohort and renewal-only loss-to-premium ratios, and assess the difference.

Method. Calculate each aggregate ratio as total losses divided by total indicated premium. Then check the group ratios to see whether the aggregate difference can be attributed to within-group changes.

For the full cohort, total losses are $104{,}000+96{,}000=200{,}000$, and total indicated premium is $130{,}000+80{,}000=210{,}000$. Thus the ratio is $200{,}000/210{,}000=0.95238095$, or 95.2381%.

For renewals, total losses are $71{,}200+10{,}800=82{,}000$, and indicated premium is $89{,}000+9{,}000=98{,}000$. Thus the ratio is $82{,}000/98{,}000=0.83673469$, or 83.6735%.

The exact difference is $95.238095\%-83.673469\%=11.564626$ percentage points. Rounded to two decimal places, the renewal-only ratio is 11.56 percentage points lower. This difference is consistent with a changed mix: Group L's ratio is $104{,}000/130{,}000=80\%$ in the full cohort and $71{,}200/89{,}000=80\%$ among renewals; Group H's ratio is $96{,}000/80{,}000=120\%$ and $10{,}800/9{,}000=120\%$, respectively. The aggregate fell while both group ratios stayed the same.

Check. The full-cohort group ratios reproduce the aggregate because $(130{,}000 \times 0.80)+(80{,}000 \times 1.20)=200{,}000$. The renewal group ratios likewise give $(89{,}000 \times 0.80)+(9{,}000 \times 1.20)=82{,}000$. The change is compositional, not a within-group ratio improvement in these data. The common mistake is subtracting the separately rounded displayed ratios, which gives 11.57 points; round the exact difference instead.

Renewal, Nonrenewal, and Entry Are Different Selection Paths

Renewal selection concerns which existing accounts remain observable. Nonrenewal may be initiated by the insured, the insurer, or both, and those mechanisms need not select the same risks. Entry creates a second source of composition change: new business may have different risk characteristics, coverage, or information quality from the continuing cohort.

A useful evaluation therefore distinguishes at least three populations:

Original cohort: the accounts present before the modification change, followed whether they renew or leave.
Renewing cohort: original accounts observed again after renewal. This is a selected subset if renewal depends on experience or related characteristics.
Post-change portfolio: renewals plus entrants, potentially minus nonrenewals. This describes the business currently written, but it is not a fixed-cohort comparison.

The distinction determines the question. To describe the post-change book, use the post-change portfolio and label it as such. To assess experience among accounts originally exposed to the change, follow the original cohort and report the exposure and outcome information available for both retained and departed accounts. If departed accounts' later outcomes are unavailable, that limitation cannot be repaired merely by calculating a renewal-only average.

Key result: A renewal-only comparison estimates performance conditional on renewal. It represents the original cohort only under conditions that make renewal unrelated to the outcome of interest, or when a justified adjustment accounts for the selection. Without such conditions or an adjustment, interpret it as a selected-cohort description.

Worked Example: Retention Changes the Average Risk

A hypothetical book has 60 Group L accounts with average annual loss of $600 and 40 Group H accounts with average annual loss of $1,500. After a modification change, 90% of Group L and 50% of Group H renew. Assume, for this calculation, that the group-average losses of renewing accounts equal their respective pre-change group averages. Calculate the original and renewal-only averages, then standardize the renewal results to the original group shares.

Method. First calculate the original weighted average. Next apply each group's retention rate to its account count and calculate the renewal average. Finally, use the original 60%/40% group shares to standardize the renewal group averages.

Initially there are 60+40=100 accounts. Total loss is $60 \times 600=36{,}000$ in Group L and $40 \times 1{,}500=60{,}000$ in Group H. The original average is $(36{,}000+60{,}000)/100=\$960$.

Group L has $60 \times 0.90=54$ renewals; Group H has $40 \times 0.50=20$. There are 74 renewals. Their modeled total loss is $54 \times 600=32{,}400$ plus $20 \times 1{,}500=30{,}000$, or $62,400. The renewal-only average is $62{,}400/74=\$843.2432$, or $843.24.

For standardization, apply the original shares to the renewal group averages: $0.60 \times 600+0.40 \times 1{,}500=360+600=\$960$. This returns the original average because the example assumes no within-group change and unchanged group-specific averages. The renewal-only average is lower because Group L, the lower-loss group, has a larger share among renewals.

Check. Renewals are 54/74=72.9730% Group L, compared with 60% in the original cohort. Weighting $600 and $1,500 by those renewal shares gives $843.24. The common mistake is to treat the $116.76 decline from $960 as an improvement in risk outcomes without checking the changed group shares.

Standardization Helps, but Does Not Solve Every Selection Problem

Standardization compares periods using common group weights. If group definitions capture all relevant differences in outcomes and the groups have sufficient observations in both periods, it separates changes in group mix from changes in group averages. The weights might be the original cohort shares, a fixed reference portfolio, or another explicitly chosen distribution. State which weights are used because different reference mixes answer different questions.

Standardization cannot correct selection within a group. For example, if the highest-loss accounts within Group L are more likely not to renew, the average loss among renewing Group L accounts may be below the average for all original Group L accounts. Using the original group share does not restore the missing accounts' outcomes. This is why group definition, available exposure, and reasons for departure matter.

A practical assessment should examine renewal and entry rates by pre-change modification, experience level, and other relevant characteristics that are available consistently. It should also compare the characteristics of renewals and nonrenewals, distinguish insurer-initiated from insured-initiated exits when possible, and report the amount of outcome data missing after departure. These are diagnostics for selection, not automatic proof of its direction or size.

Where an adjustment is considered, it needs explicit assumptions. For example, reweighting renewals by estimated inverse renewal probabilities can represent the original cohort only if renewal probabilities are estimated from relevant pre-renewal variables, the model is adequate, and each relevant type of account has a positive chance of renewal. It also relies on the assumption that, conditional on those variables, renewed and nonrenewed accounts have comparable outcomes for the target analysis. If an important driver of both renewal and outcome is unobserved, reweighting may remain biased.

Worked Example: Separating Churn From New Entry

A hypothetical portfolio initially contains 50 Group L accounts averaging $500 in annual loss and 30 Group H accounts averaging $1,500. After a modification change, 80% of Group L and 40% of Group H renew. The insurer also writes 20 new accounts with an average annual loss of $900. Assume the stated group averages apply to the renewing accounts. Calculate the original average, the renewal-only average, and the average of the post-change portfolio including entrants.

Method. Treat the original cohort, renewals, and entrants as separate populations before combining any totals.

The original total loss is $50 \times 500=25{,}000$ in Group L and $30 \times 1{,}500=45{,}000$ in Group H. The original average is $(25{,}000+45{,}000)/(50+30)=70{,}000/80=\$875$.

There are $50 \times 0.80=40$ Group L renewals and $30 \times 0.40=12$ Group H renewals. Their total loss is $40 \times 500+12 \times 1{,}500=20{,}000+18{,}000=38{,}000$. The renewal-only average is $38{,}000/52=\$730.7692$, or $730.77.

The post-change portfolio adds 20 entrants with total loss $20 \times 900=18{,}000$. Its total is $38{,}000+18{,}000=56{,}000$ across $52+20=72$ accounts, giving $56{,}000/72=\$777.7778$, or $777.78.

Check. The renewal-only group mix is 40/52 Group L and 12/52 Group H, yielding $(40/52 \times 500)+(12/52 \times 1{,}500)=\$730.77$. Entrants have a $900 average, above the renewal average, so adding them raises the portfolio average to $777.78. Neither observed post-change figure alone describes the original 80 accounts as if all had remained. The common mistake is to treat entrants as part of the original cohort or to compare the post-change book directly with the original without separating the population change.

Common Evaluation Traps

Using only those who renewed. This changes the denominator and may change the risk mix. Report renewal rates and the renewal-only population alongside the result.
Calling a portfolio average a within-risk change. An aggregate can move solely because group shares changed. Show group-level results and, where appropriate, standardized results.
Ignoring entrants. A current-book average combines continuing accounts with new business. Keep those components distinct before interpreting a trend.
Assuming standardization removes all bias. It adjusts the recorded groups' weights, not unobserved selection within groups or outcomes missing for departed accounts.
Confusing association with causation. A change in renewal patterns after a modification change is evidence to investigate, but it does not alone establish that the change caused the pattern.

These checks are aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus: define the population and measure, examine how selection changes composition, and qualify conclusions according to the available comparison. The central discipline is to state whether a result describes a fixed cohort, renewals only, or the post-change portfolio.

Check Your Understanding

For each problem, distinguish the population being measured from any composition effect.

A portfolio has 70 Group L accounts averaging $400 and 30 Group H accounts averaging $1,200. Retention is 80% for L and 40% for H. Assuming retained accounts have the stated group averages, calculate the original average and the renewal-only average. (Answer: original average $640; renewal-only average $541.18.)
A book has 40 Group L accounts averaging $300 and 60 Group H accounts averaging $900. Retention is 75% in L and 50% in H. Calculate the renewal-only average and the average standardized to the original group shares. (Answer: renewal-only average $600; standardized average $660.)
An original cohort has 30 Group L accounts averaging $500 and 20 Group H accounts averaging $1,300. Retention is 80% in L and 30% in H. Ten entrants average $900. Calculate the original average, renewal-only average, and post-change portfolio average including entrants. (Answer: original $820; renewals $660; post-change portfolio $720.)
In the preceding problem, explain why standardizing renewal group averages to the original group shares does not necessarily recover the original-cohort outcome if high-loss accounts within each group are less likely to renew. (Answer: the renewal group averages may themselves be selected and differ from the full-group averages; reweighting group shares cannot restore missing within-group outcomes.)
A post-change portfolio has a lower average loss than the pre-change cohort, but the high-loss modification group has a much lower renewal rate and new business has a different risk mix. State two checks needed before attributing the decrease to improved experience. (Answer: compare renewal and nonrenewal rates and characteristics by pre-change risk group; separate entrants from continuing accounts and compare group-level or standardized outcomes.)

What You'll Learn