Data Limitations in Berquist-Sherman Adjustments

Why Data Quality Is Part of the Adjustment

Berquist-Sherman adjustments use observed claim experience to distinguish changes in claim settlement or case-reserving behavior from changes in the underlying loss experience. That distinction depends on the data representing comparable claims at comparable development ages. If records are missing, definitions change, mature cohorts are small, or closure dates are unreliable, a calculated adjustment can be precise in arithmetic but unsupported by the evidence.

The task here is not to repair every defective data set. It is to assess whether the quantities being used are sufficiently complete, consistent, and credible for the proposed adjustment. A data limitation can affect the direction of an adjustment, not merely its precision. For example, omitting many still-open claims can make a cohort appear to settle faster than it did.

Definition: A supportable adjustment input is a claim-history quantity that has a clearly defined population, development age, and measurement basis; is sufficiently complete and reliable for its intended use; and can be reconciled to the underlying records. A quantity that fails these checks should not be treated as a dependable measure of claim handling or reserve adequacy.

Completeness: Which Claims Are Missing?

A basic diagnostic for settlement behavior is the proportion of claims closed by a stated development age. Let $N$ be the number of claims in the defined cohort and $C$ the number closed by that age. The observed closure proportion is $C/N$. Interpreting it as representative requires, among other things, that the cohort’s reported claims and statuses are captured consistently.

The effect of missing records depends on which claims are missing. To see why, suppose $M$ claims are absent from an extract and $M_C$ of those missing claims had closed by the selected age. The true closure proportion is $q=C/N$, while the extract gives $q_{\text{obs}}=(C-M_C)/(N-M)$. Subtracting the true proportion gives:

q_{\text{obs}}-q = \frac{M(q-q_{\text{miss}})}{N-M}, \qquad q_{\text{miss}}=\frac{M_C}{M}.

Thus, if the missing claims have a lower closure proportion than the recorded claims, the extract overstates closure. If the missing claims have a higher closure proportion, the extract understates it. This is why “some records are missing” is not a complete data-quality assessment: the pattern of missingness matters.

Worked Example: Missing Open Claims in a Historical Cohort

A hypothetical insurer is reviewing claims at 18 months of development. A reconciliation identifies 120 claims in the complete cohort, of which 78 are closed. A legacy extract contains only 90 claims, including 72 closed claims. Assess how the extract affects the closure proportion and what the result implies for adjustment work.

Method. Calculate the closure proportion on the complete cohort, then on the extract, and compare the results. The comparison is a completeness diagnostic; it is not, by itself, a universal Berquist-Sherman adjustment formula.

On the complete cohort, $N=120$ and $C=78$, so:

q=\frac{78}{120}=0.650=65.0\%.

The extract gives $N-M=90$ claims and $C-M_C=72$ closed claims. Its closure proportion is:

q_{\text{obs}}=\frac{72}{90}=0.800=80.0\%.

The extract overstates closure by $80.0\%-65.0\%=15.0$ percentage points. The omitted records are $M=120-90=30$, with $M_C=78-72=6$ closed claims. Their closure proportion is $6/30=20.0\%$, far below the complete-cohort proportion. The missingness identity confirms the direction:

q_{\text{obs}}-q = \frac{30(0.650-0.200)}{90} = 0.150.

Conclusion and trap avoided. The extract makes settlement appear faster because it disproportionately omits open claims. Treating the 80.0% as a genuine handling change would confuse a data-coverage break with claim behavior. The missing-claim pattern must be understood and the cohort reconciled before using its closure experience.

Definition Changes: Are the Cohorts Still Comparable?

A claim definition determines what enters the denominator, what counts as closed, and which payments contribute to severity measures. A system migration, a change in coverage coding, or a new rule for including expense-only claims can therefore create a discontinuity even when underlying claim handling has not changed. Comparing two periods without checking these rules can attribute a population change to settlement behavior or reserve adequacy.

A useful audit records the claim unit and inclusion criteria for each period: for example, whether the unit is a claim feature or an occurrence, whether expense-only records are included, and whether reopened claims are treated as closed or open at the measurement date. Where possible, recalculate the quantities on a common definition. If that is not possible, the definition break should be treated as a limitation, not hidden by pooling the periods.

Worked Example: Excluding Expense-Only Claims Changes the Measured Population

A hypothetical portfolio’s older extract includes both indemnity-bearing and expense-only claims. The older cohort has 80 indemnity-bearing claims, of which 60 are closed with total closed paid losses of $600,000. It also has 20 expense-only claims, of which 10 are closed with total closed paid losses of $50,000. A later system extract includes indemnity-bearing claims only. Compare the old mixed-population measures with the later definition’s relevant measures.

Method. Compute closure proportions and average closed paid severity separately by claim group before calculating the mixed result. This exposes the composition effect and avoids assuming that unlike populations are interchangeable.

For indemnity-bearing claims, the closure proportion is $60/80=75.0\%$, and average paid loss per closed claim is:

\frac{\$600{,}000}{60}=\$10{,}000.

For expense-only claims, the closure proportion is $10/20=50.0\%$, and average paid loss per closed claim is:

\frac{\$50{,}000}{10}=\$5{,}000.

Under the old definition, there are $80+20=100$ claims and $60+10=70$ closed claims. The mixed closure proportion is $70/100=70.0\%$. Mixed closed paid losses are $\$600{,}000+\$50{,}000=\$650{,}000$, so mixed average paid loss per closed claim is:

\frac{\$650{,}000}{70}=\$9{,}285.71.

Under the later indemnity-only definition, the comparable figures are 75.0% closed and $10,000 per closed claim. Relative to the old mixed measures, those are increases of 5.0 percentage points and $\$10{,}000-\$9{,}285.71=\$714.29$, respectively.

Conclusion and trap avoided. Neither difference alone demonstrates a change in settlement behavior or severity. The older population includes a group with a lower closure proportion and lower average paid amount. Comparing its mixed results directly with indemnity-only results confounds a definition change with a time trend. The adjustment needs a common claim definition or a clearly quantified limitation.

Sparse Mature Cohorts: A Rate Can Be Highly Unstable

Mature cohorts are often important because they provide observations at later development ages. But a cohort being mature does not make its estimate reliable if it contains very few claims. With small denominators, one claim’s status can move a closure proportion materially and can have an outsized influence on any adjustment built from that proportion.

For a simple uncertainty diagnostic, assume claims in the cohort have independent, identical probabilities of being closed by the selected age. Then the number closed has a binomial distribution with cohort size $n$ and closure probability $p$. The estimated proportion is $c/n$, with approximate standard error $\sqrt{\hat p(1-\hat p)/n}$, where $\hat p=c/n$. This approximation describes sampling variability under the stated model; it does not correct for incomplete records, changing definitions, or dependence between claims.

Worked Example: One Claim Changes a Sparse Cohort’s Closure Rate

A hypothetical mature cohort contains eight claims, six of which are recorded closed by 24 months. Assess the estimated closure proportion, its approximate sampling uncertainty, and the effect of one status changing.

Method. Use the binomial proportion and its approximate standard error, then recompute the proportion after changing one claim’s status. The calculation is a stability check, not proof that the model assumptions hold.

The estimated closure proportion is:

\hat p=\frac{6}{8}=0.750.

Under the independent, identical Bernoulli-trial assumption, the estimated standard error is:

\sqrt{\frac{0.750(1-0.750)}{8}} = \sqrt{\frac{0.1875}{8}} = \sqrt{0.0234375} \approx 0.1531.

If one claim recorded as closed is instead open at 24 months, the estimate becomes $5/8=0.625$. If one recorded as open is instead closed, it becomes $7/8=0.875$. Thus, changing one status moves the estimate by $0.125$, or 12.5 percentage points, in either direction.

Conclusion and trap avoided. The point estimate of 75.0% may look usable, but eight observations leave it highly sensitive to individual claims. The approximate standard error is about 15.3 percentage points, and one status change moves the estimate by 12.5 points. Treating the estimate as a stable settlement pattern because the cohort is mature ignores its sparse volume.

Unreliable Closure Dates: Bound the Consequence

Closure status at a development age depends on the relevant date and on the definition of closure. A date may be missing, overwritten, stored in a different time zone, or refer to a system action rather than the actual settlement event. If the date is uncertain around the development-age cutoff, classifying every affected record as definitively open or closed can create false precision.

A practical diagnostic is to identify records whose dates are uncertain and calculate a range. The lower bound counts only closures that are definitely before the cutoff; the upper bound also counts closures that could have occurred before it. This range is not a replacement for resolving the dates, but it shows how much the uncertainty can change the measured quantity.

Worked Example: Uncertain Dates Around a Development Cutoff

A hypothetical cohort has 40 claims at 12 months. The extract labels 29 as closed by 12 months, but five of those closure dates may have been entered after the cutoff and cannot be verified. Assume the other 24 are definitely closed by the cutoff, and no additional claims are known to have closed with a missing date. Find the closure-proportion range implied by the date uncertainty.

Method. Calculate a conservative lower bound using only verified closures and an upper bound including all five uncertain dates. This avoids treating uncertain timestamps as exact.

The lower bound is $24/40=0.600$, or 60.0%. The upper bound is $(24+5)/40=29/40=0.725$, or 72.5%. The width of the range is $72.5\%-60.0\%=12.5$ percentage points.

Conclusion and trap avoided. The extract’s nominal rate of $29/40=72.5\%$ is only the upper end of the defensible range under the stated facts. Using it as an exact rate would silently assume all five uncertain dates fall before the cutoff. Reconcile the event-date definition and source records, or carry the resulting uncertainty into the assessment of the adjustment input.

A Practical Supportability Review

A disciplined review separates data checks from the calculation of an adjustment. For each quantity to be used, document the population, valuation date, development age, claim-status rule, and source fields. Then test whether those definitions and records remain consistent across the cohorts being compared.

Define the measurement.
State the claim unit, inclusion rules, closure definition, development age, and valuation date. Verify that the numerator and denominator use the same population.

Reconcile coverage.
Compare extract counts and statuses with source systems, prior snapshots, or control totals. Investigate whether missing records are concentrated among open, closed, old, or particular types of claims.

Check comparability and volume.
Identify definition changes and inspect the number of claims behind mature-cohort estimates. Where relevant, quantify how much individual records or reasonable alternative classifications move the result.

Document the consequence.
State which input is affected, the possible direction or range of distortion, and whether reconciliation or a common-basis calculation is available. Do not present an uncertain input as a precise behavioral signal.

These checks matter because an adjusted triangle can appear internally consistent even when its underlying claim history is not comparable. A small change in the measured closure proportion, paid severity, or case-reserve pattern may be caused by record coverage or classification rather than by a real operational change. The appropriate response is to investigate the data and show the consequence of unresolved uncertainty, not to let an adjustment conceal it.

Key takeaway: Before using claim-history measures in a Berquist-Sherman adjustment, establish that the claims are captured, defined, dated, and sufficiently numerous on a comparable basis. If those conditions are not met, quantify the limitation and do not overstate what the resulting adjustment shows.

Check Your Understanding

For each hypothetical situation, identify the relevant data limitation and calculate the requested diagnostic.

A cohort has 150 claims, of which 90 are closed by 18 months. An extract contains 120 claims, including 84 closed claims. Calculate both closure proportions and the difference in percentage points. What pattern of missing claims is consistent with the extract’s result? (Answer: Complete cohort: 90/150 = 60.0%; extract: 84/120 = 70.0%; the extract is 10.0 percentage points higher, consistent with missing claims having a lower closure proportion.)
An older definition includes 50 indemnity claims, 35 closed with $280,000 paid, and 10 expense-only claims, 4 closed with $12,000 paid. A later definition includes indemnity claims only. Calculate the older mixed closure proportion and mean paid per closed claim, then the indemnity-only figures. (Answer: Mixed closure proportion = 39/60 = 65.0%; mixed mean = $292,000/39 = $7,487.18; indemnity-only closure proportion = 35/50 = 70.0%; indemnity-only mean = $280,000/35 = $8,000.)
A mature cohort has 12 claims, 9 closed by the selected age. Calculate the closure proportion and approximate binomial standard error. How much does the proportion change if one closed claim is reclassified as open? (Answer: Proportion = 9/12 = 0.750; standard error = $\sqrt{0.75(0.25)/12}\approx0.1250$; reclassified proportion = 8/12 = 0.6667, a decrease of 0.0833.)
In a cohort of 64 claims, 41 are definitely closed by 15 months and 7 additional closure dates may be either before or after the cutoff. Calculate the defensible closure-proportion range and its width. (Answer: Lower bound = 41/64 = 64.0625%; upper bound = 48/64 = 75.0%; range width = 10.9375 percentage points.)
A data extract shows 96 closed claims out of 120. A reconciliation finds 20 omitted claims, 18 of which were open at the selected age. Calculate the closure proportion after restoring the omitted claims and the extract’s overstatement in percentage points. (Answer: Restored proportion = (96+2)/(120+20) = 98/140 = 70.0%; extract proportion = 80.0%; overstatement = 10.0 percentage points.)

What You'll Learn