Benchmarking Capital Model Results

Why Benchmarking Requires More Than Comparing Numbers

A capital result is meaningful only in relation to a defined loss basis, measurement horizon, risk scope, and capital measure. A peer figure, historical loss, or independent calculation can help assess whether a model result is plausible, but a numerical difference alone does not show that the model is wrong. The benchmark may reflect a different portfolio, definition, data period, or calibration.

Benchmarking is therefore a structured comparison. First determine whether the results measure the same thing. Then adjust for material, supportable differences, quantify the remaining gap, and decide what the gap can and cannot tell you. The purpose is to inform model assessment, not to force different methods to produce identical answers.

Definition: A benchmark is a reference result used to assess a model output. It may come from external peer information, suitably adjusted historical experience, or an alternative calculation. A comparable benchmark measures a sufficiently similar risk, horizon, loss basis, capital measure, and exposure basis for the comparison to be interpretable.

For example, a one-year 99.5% net loss VaR is not directly comparable with a three-year gross loss estimate, even if both are labelled “capital.” A careful analyst documents the comparison basis before calculating a gap.

A Framework for Making Comparisons

A useful benchmark review separates alignment questions from numerical analysis. The first question is not “How large is the difference?” but “What do these two numbers represent?” The following dimensions commonly affect comparability.

Dimension	Questions to resolve	Possible consequence
Risk scope	Are the same risks, entities, and business units included?	A broader benchmark may be larger even if its underlying model is not more conservative.
Loss basis	Are losses gross or net of reinsurance, and are expenses or other items included consistently?	Different bases can create a systematic gap unrelated to calibration.
Measure and horizon	Are both results the same measure, confidence level, and time horizon?	A VaR, tail measure, or multi-year figure cannot be treated as interchangeable without a justified conversion.
Exposure and mix	Do the amounts represent similar portfolio size and risk composition?	Scaling by exposure may address size but not different concentrations or product mix.
Data and calibration	Are the data periods, assumptions, and calibration bases sufficiently alike?	Different periods or calibrations can explain differences without indicating model error.
Method	Does the benchmark use the same aggregation structure and dependence assumptions?	Different methods may produce a useful range rather than a single expected answer.

Some adjustments are straightforward when their assumptions are credible. For example, a proportional exposure adjustment can be made if losses are assumed to scale proportionally with the chosen exposure measure and the risk mix is comparable. Other differences are not safely addressed by a simple arithmetic adjustment. In particular, subtracting a separately reported risk component from an aggregated capital result may be invalid when diversification or nonlinear aggregation is present.

Define the target result.
Record the model’s risk scope, loss basis, horizon, capital measure, confidence level, and valuation date.

Describe the benchmark on the same dimensions.
Identify what is known, what differs, and what is uncertain about its scope, data, and method.

Make only supportable adjustments.
State each adjustment and its assumption. Do not disguise an unquantified difference as a precise normalization.

Quantify and interpret the residual gap.
Report the absolute difference and, where useful, a relative difference with a stated denominator. Explain limitations and follow-up questions.

Translating a Normal-Model Benchmark

A benchmark may use a different confidence level from the model result. One possible translation is available when the loss is modeled as normal and the mean and standard deviation are appropriate for the same scope and horizon. This is a model-based translation, not a universal conversion between confidence levels.

Definition: For a loss $X$, define $\operatorname{VaR}_p(X)$ as the $p$-quantile of its loss distribution. In this tutorial, economic capital at confidence level $p$ is $\operatorname{VaR}_p(X)-E[X]$. If $X\sim N(\mu,\sigma^2)$, then $\operatorname{VaR}_p(X)=\mu+\sigma z_p$, where $z_p$ is the standard normal $p$-quantile.

The formula follows by standardizing the normal variable. If $X\sim N(\mu,\sigma^2)$, then $(X-\mu)/\sigma$ has a standard normal distribution. Thus the event $X\leq \mu+\sigma z_p$ is equivalent to $(X-\mu)/\sigma\leq z_p$, which has probability $p$. Therefore the $p$-quantile is $\mu+\sigma z_p$. Subtracting the mean gives economic capital $\sigma z_p$.

\operatorname{VaR}_p(X)=\mu+\sigma z_p,\qquad K_p=\operatorname{VaR}_p(X)-\mu=\sigma z_p

If a normal benchmark provides capital at level $p_1$, then, provided the same normal model and parameters apply, capital at level $p_2$ can be translated by the ratio of the standard normal quantiles. This works because the same $\sigma$ multiplies each quantile. It does not justify translating a non-normal model, a different loss basis, or a result whose mean and calibration are not comparable.

K_{p_2}=K_{p_1}\frac{z_{p_2}}{z_{p_1}}

Worked Example: Translating a Normal Benchmark to a Higher Confidence Level

A hypothetical benchmark models one-year loss as normal with mean $$1.5$ million. Its 99% VaR is $$10.8158$ million. Using $z_{0.99}=2.3263$ and $z_{0.995}=2.5758$, calculate the 99% economic capital and the translated 99.5% economic capital and VaR. Assume the same normal loss model, mean, and standard deviation apply at both confidence levels.

Method and assumptions. Under the stated normal model, economic capital is VaR less the mean, and capital scales with the standard normal quantile. The comparison is valid only under the shared-model assumptions stated in the problem.

Step 1: Find 99% economic capital. Subtract the mean loss from the 99% VaR:

K_{0.99}=10.8158-1.5000=9.3158\text{ million}

Step 2: Translate capital to 99.5%. Apply the ratio of the quantiles:

K_{0.995}=9.3158\left(\frac{2.5758}{2.3263}\right) =9.3158(1.1073\text{ approximately}) \approx 10.3149\text{ million}

Step 3: Recover 99.5% VaR. Add the unchanged mean:

\operatorname{VaR}_{0.995}=10.3149+1.5000 =11.8149\text{ million}

The 99.5% capital is approximately $$10.3149$ million and the 99.5% VaR is approximately $$11.8149$ million, rounded to four decimal places in millions. As a check, both exceed their 99% counterparts because $z_{0.995}>z_{0.99}$. A common mistake is to call the translated capital the VaR, omitting the mean. Another is to report $$10.3132$ million: the stated product is $9.3158(2.5758/2.3263)\approx10.3149$, not $10.3132$.

Peer Comparisons and Exposure Normalization

Peer information is useful when the definition and portfolio basis are sufficiently clear. A peer amount can be adjusted for portfolio size only under a stated scaling assumption. Proportional scaling is most defensible when the portfolios have similar risk mix, limits, geographic distribution, and relevant underwriting characteristics. It does not correct for differences in those features.

Worked Example: Normalizing a Peer Capital Figure

A hypothetical peer reports $$72$ million of capital for a portfolio with $$900$ million of annual exposure. A company’s comparable portfolio has $$1{,}200$ million of exposure, and its model reports $$102$ million of capital. Assume both results use the same one-year 99.5% measure, loss basis, and risk scope, and assume capital scales proportionally with exposure because the risk mix is comparable. Normalize the peer figure and quantify the company’s difference.

Method and assumptions. Scale the peer amount by the ratio of the company’s exposure to the peer’s exposure. The calculation relies on proportionality; it is not appropriate if material differences in risk mix or exposure quality make that assumption unreasonable.

Step 1: Calculate the exposure ratio.

\frac{1{,}200}{900}=\frac{4}{3}=1.3333\text{ approximately}

Step 2: Scale the peer capital.

72\left(\frac{4}{3}\right)=96\text{ million}

Step 3: Compare the company result with the normalized benchmark. The absolute difference is $102-96=$$6 million. Relative to the normalized peer benchmark, the difference is:

\frac{102-96}{96}=\frac{6}{96}=0.0625=6.25\%

The company’s model result is $$6$ million, or 6.25%, above the normalized peer figure. The calculation is internally consistent: the normalized peer amount corresponds to $$96$ million, and adding the $$6$ million gap gives the company’s $$102$ million. The gap prompts questions about calibration and portfolio details; it does not, by itself, establish that either result is incorrect. A common mistake is to scale by $900/1{,}200$, which would shrink the peer amount even though the company portfolio has more exposure.

Historical Experience as a Reference Point

Historical experience can provide a concrete comparison, especially for assessing whether a modeled scenario or loss magnitude is plausible. But a past loss is not automatically a capital measure or a direct estimate of a tail quantile. The event may have occurred under different exposure levels, prices, policy terms, inflation, or reinsurance arrangements. Adjustments should make those differences visible rather than imply that the past is a complete representation of future risk.

Worked Example: Adjusting a Historical Loss for Inflation and Exposure

A hypothetical catastrophe produced a $$40$ million loss when a severity index was 100 and the relevant exposure index was 80. For a current comparison, the severity index is 110 and the exposure index is 100. Assume the loss scales proportionally with both indices, the event definition and coverage basis are unchanged, and the two adjustments can be applied multiplicatively. A current model scenario for the same event type produces $$62$ million. Adjust the historical loss and calculate the model’s percentage difference from that adjusted amount.

Method and assumptions. Apply the severity adjustment and the exposure adjustment separately. This is a simplifying benchmark calculation; it does not model changes in event frequency, event footprint, or the distribution of possible losses.

Step 1: Adjust for severity. The severity factor is $110/100=1.10$, so the loss becomes:

40(1.10)=44\text{ million}

Step 2: Adjust for exposure. The exposure factor is $100/80=1.25$, so:

44(1.25)=55\text{ million}

Equivalently, the combined factor is $1.10(1.25)=1.375$, and $40(1.375)=55$ million. The model scenario exceeds the adjusted historical loss by $62-55=$$7 million. Relative to the adjusted historical amount, the difference is:

\frac{7}{55}=0.12727\text{ approximately}=12.73\%

The adjusted historical reference is $$55$ million, and the modeled scenario is approximately 12.73% higher. The check $40(1.10)(1.25)=55$ confirms the adjustment arithmetic. This comparison does not show that a $$62$ million loss is a particular return-period event; it only compares two amounts under the specified adjustments. A common mistake is to adjust for exposure but omit severity, yielding $$50$ million rather than $$55$ million.

Alternative Calculations as Reasonableness Benchmarks

A separate calculation can reveal whether a complex model result is broadly consistent with a simpler representation. For example, a covariance aggregation can serve as a rough benchmark for two risk components when their capital amounts can reasonably be treated as standard-deviation-like quantities under a common scaling. The formula below is not an identity for arbitrary risk measures. Its usefulness depends on the assumptions behind the component figures and the dependence estimate.

Benchmark formula: For two components with capital-like amounts $K_1$ and $K_2$, a covariance-style aggregate is $\sqrt{K_1^2+K_2^2+2\rho K_1K_2}$, where $\rho$ is the relevant correlation. Use this as a comparison only when the amounts and correlation are defined consistently and the approximation is suitable for the risks being represented.

Worked Example: Comparing a Model Output with a Covariance Proxy

A hypothetical model reports $$118$ million of aggregate capital for two risks. A separate approximation uses component amounts $K_1=$$80 million and $K_2=$$60 million, with correlation $\rho=0.25$. Assume the component amounts can be used in the stated covariance-style proxy and that the correlation refers to the same basis. Calculate the proxy and the model’s percentage excess over it.

Method and assumptions. Substitute the component amounts and correlation into the covariance-style formula. This is a reasonableness comparison, not proof that the full model should equal the proxy.

Step 1: Calculate the squared terms.

K_1^2=80^2=6{,}400,\qquad K_2^2=60^2=3{,}600

Step 2: Calculate the dependence term.

2\rho K_1K_2=2(0.25)(80)(60)=2{,}400

Step 3: Aggregate and compare.

K_{\text{proxy}}=\sqrt{6{,}400+3{,}600+2{,}400} =\sqrt{12{,}400} \approx111.36\text{ million}

The model exceeds the proxy by $118-111.36=$$6.64 million, approximately. Relative to the proxy, the excess is $6.64/111.36\approx5.97\%$, or about 6.0%. The result is plausible as a comparison only to the extent that the proxy assumptions fit the model’s components and dependence basis. A common mistake is to omit the cross-term, which would produce $\sqrt{10{,}000}=$$100 million and incorrectly treat the risks as uncorrelated.

Interpreting a Benchmark Gap

A benchmark difference is evidence to investigate, not an automatic pass-or-fail test. A small gap may conceal offsetting differences; a large gap may be explained by legitimate differences in scope or method. The strength of the conclusion should reflect the reliability and comparability of the benchmark.

State the denominator. A percentage gap depends on whether it is measured relative to the model result, the peer result, or an adjusted historical amount.
Separate observed facts from assumptions. Identify which adjustments are based on documented information and which depend on simplifying judgments.
Do not over-interpret a single reference. Peer figures may be sparse or differently defined; a historical event is one observation; a simplified calculation omits features of a more detailed model.
Investigate material residuals. A persistent or unexplained gap may warrant examining scope, data lineage, assumptions, calibration, and implementation.
Keep the comparison on its intended basis. A benchmark of a capital amount is not, on its own, an analysis of realized outcomes or a substitute for evaluating model performance over time.

Key takeaway: Benchmark only after defining what is being compared. Align scope, loss basis, horizon, measure, exposure, and calibration where possible; make transparent adjustments where justified; then interpret the remaining gap in light of the benchmark’s limitations. This disciplined approach is aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.

Check Your Understanding

For each problem, identify any key comparability assumption before interpreting the arithmetic. All scenarios are hypothetical.

A peer reports $$54$ million of capital for $$600$ million of exposure. A comparable portfolio has $$750$ million of exposure and model capital of $$70$ million. Under proportional exposure scaling, calculate normalized peer capital and the model’s percentage excess relative to that benchmark. (Answer: normalized peer capital $$67.5$ million; model excess $$2.5$ million, or approximately 3.70%.)
A normal benchmark has mean loss $$2$ million and standard deviation $$3$ million. Using $z_{0.975}=1.9600$ and $z_{0.99}=2.3263$, calculate economic capital and VaR at each confidence level. (Answer: at 97.5%, capital $$5.8800$ million and VaR $$7.8800$ million; at 99%, capital $$6.9789$ million and VaR $$8.9789$ million.)
A historical loss of $$24$ million occurred at severity index 100 and exposure index 80. Current indices are 105 and 96. Assuming proportional multiplicative adjustments, calculate the adjusted historical loss. If a comparable model scenario is $$33$ million, calculate its percentage excess relative to the adjusted loss. (Answer: adjusted loss $$30.24$ million; model excess $$2.76$ million, or approximately 9.13%.)
A covariance proxy uses component amounts of $$30$ million and $$40$ million with correlation 0.50. A model reports $$64$ million. Calculate the proxy and the model’s percentage excess relative to it, rounding the proxy to cents and the percentage to two decimals. (Answer: proxy $\sqrt{3{,}700}\approx$$60.83 million; model excess approximately $$3.17$ million, or 5.21%.)
A model reports one-year 99.5% net capital. A benchmark reports one-year 99.5% gross capital, but no information is available about reinsurance recoveries or whether the same risks are included. Is a direct numerical comparison justified? State the principal issue. (Answer: no; gross-versus-net loss basis and potentially different risk scope have not been reconciled.)

What You'll Learn