Sampling Variability Of Empirical Tail Probabilities

Why an Empirical Tail Estimate Varies

An empirical tail probability summarizes how often observed losses pass a threshold. If a different sample of losses were collected from the same population, the proportion could change. The estimate is therefore not only a summary of the observed data; under a sampling model, it is also a random quantity with an expectation and a variance.

This tutorial considers the basic setting: a fixed threshold and a sample of independent, identically distributed, exact losses. The key device is an indicator variable, which records whether each loss does or does not exceed the threshold. The resulting expectation and variance apply only when the stated sampling assumptions hold.

Definition: Let $X_1,\ldots,X_n$ be independent, identically distributed exact losses with common distribution $X$. For a fixed threshold $t$, define the strict empirical exceedance estimate by $$ \widehat{S}_n(t)=\frac{1}{n}\sum_{i=1}^n I_i(t), \qquad I_i(t)=\mathbf{1}\{X_i>t\}. $$ Here $\mathbf{1}\{X_i>t\}$ equals 1 when $X_i>t$ and 0 otherwise. The population exceedance probability is $p=P(X>t)=S_X(t)$. The estimator $\widehat{S}_n(t)$ is the observed proportion of sampled losses strictly greater than $t$.

For an inclusive threshold, replace $X_i>t$ by $X_i\ge t$, and define $p_{\ge}=P(X\ge t)$. This distinction matters when the loss distribution has positive probability at exactly $t$. If there is no probability mass at $t$, the strict and inclusive probabilities agree.

The observed estimate $\widehat{S}_n(t)$ is a number once the sample is in hand. Before sampling, it is random because its indicator values are random. Its variance describes how much that estimate would vary across repeated samples of size $n$; it is not the variance of individual losses.

Expectation and Variance From Indicators

For a strict threshold, each $I_i(t)$ is a Bernoulli random variable: it is 1 with probability $p$ and 0 with probability $1-p$. Therefore $E[I_i(t)]=p$ and $\operatorname{Var}(I_i(t))=p(1-p)$. The estimate is the average of $n$ such indicators.

Write the sample proportion as an average.
$\widehat{S}_n(t)=n^{-1}\sum_{i=1}^n I_i(t)$, where each indicator records one strict exceedance.

Take the expectation.
Linearity of expectation gives $E[\widehat{S}_n(t)]=n^{-1}\sum_{i=1}^n E[I_i(t)]=n^{-1}(np)=p$.

Calculate the variance.
Independence makes the covariances between distinct indicators zero, so $\operatorname{Var}(\widehat{S}_n(t))=n^{-2}\sum_{i=1}^n\operatorname{Var}(I_i(t))=p(1-p)/n$.

Theorem: For independent, identically distributed exact losses and a fixed threshold $t$, the strict empirical exceedance estimate is unbiased and has variance $$ E[\widehat{S}_n(t)]=p, \qquad \operatorname{Var}(\widehat{S}_n(t))=\frac{p(1-p)}{n}, \qquad p=P(X>t). $$ Its standard deviation, often called the standard error under this model, is $\sqrt{p(1-p)/n}$. The same formulas hold for the inclusive estimate when $p$ is replaced by $P(X\ge t)$.

The expectation result follows because every sampled loss has the same probability $p$ of exceeding $t$, so the average of the indicators has mean $p$. To see why the variance has a factor $1/n$, expand the variance of the sum. Independence implies $\operatorname{Cov}(I_i(t),I_j(t))=0$ for $i\ne j$, leaving $n$ individual variances, each equal to $p(1-p)$. Dividing the sum by $n$ divides its variance by $n^2$, giving $p(1-p)/n$.

If $p$ is unknown, the formula still describes the estimator’s variance in terms of the unknown population probability. In practice, one may estimate its standard error by substituting the observed proportion $\widehat{S}_n(t)$ for $p$. That plug-in value is an estimated standard error, not the exact variance unless the substituted value happens to equal $p$.

Worked Applications

Worked Example: A Strict Threshold With an Observed Sample

A hypothetical portfolio has independent losses $X_1,\ldots,X_8$ from a common distribution. For each loss, the probability of exceeding $250 is 0.50. The eight exact observed losses are $100, $300, $200, $400, $300, $100, $400, and $200. Find the empirical strict exceedance estimate and the theoretical variance and standard error of that estimate.

Method. Represent each observation by $I_i=\mathbf{1}\{X_i>250\}$. The losses greater than $250 are $300, $400, $300, and $400, so their indicators are 1; the other four indicators are 0.

Observed estimate. There are four exceedances among eight losses: $$ \widehat{S}_8(250)=\frac{4}{8}=0.5000. $$

Theoretical variance. The population probability is $p=0.50$, and $n=8$. Thus $$ \operatorname{Var}(\widehat{S}_8(250)) =\frac{0.50(1-0.50)}{8} =\frac{0.25}{8} =0.03125. $$

Standard error. Taking the square root gives $\sqrt{0.03125}=0.176776\ldots$, or 0.1768 to four decimal places. The observed estimate happens to equal the stated population probability, but the variance calculation comes from the assumed sampling model, not from how close the observed proportion looks to $p$.

Check and common mistake avoided. The variance is less than the Bernoulli variance $0.25$, as expected for an average of eight independent indicators. Omitting the division by $n$ would incorrectly give 0.25, the variance of one indicator rather than the variance of the sample proportion.

Worked Example: Ties at the Threshold Change the Probability

Suppose exact losses have the following hypothetical distribution: $P(X=100)=0.35$, $P(X=250)=0.25$, and $P(X=600)=0.40$. For a sample of $n=25$ independent losses, calculate the expectation and variance of both the strict estimate for losses greater than $250 and the inclusive estimate for losses at least $250.

Strict estimate. A strict exceedance excludes losses equal to $250. Only the $600 outcome exceeds the threshold, so $p=P(X>250)=0.40$. Therefore $$ E[\widehat{S}_{25}(250)]=0.40, \qquad \operatorname{Var}(\widehat{S}_{25}(250)) =\frac{0.40(0.60)}{25} =\frac{0.24}{25} =0.0096. $$

Inclusive estimate. An inclusive exceedance includes both $250 and $600. Hence $p_{\ge}=P(X\ge250)=0.25+0.40=0.65$. Its expectation and variance are $$ E[\widehat{S}_{25,\ge}(250)]=0.65, \qquad \operatorname{Var}(\widehat{S}_{25,\ge}(250)) =\frac{0.65(0.35)}{25} =\frac{0.2275}{25} =0.0091. $$

Check and common mistake avoided. The inclusive probability is $0.65$, not $0.75$: the only included masses are $0.25$ and $0.40$, and they sum to $0.65$. The inclusive probability must be at least the strict probability, which it is. Applying the variance formula to the correct probabilities gives 0.0096 and 0.0091, respectively.

Worked Example: Choosing a Sample Size for a Target Standard Error

A hypothetical loss population has a known probability $p=0.20$ of exceeding a specified threshold. Determine the smallest sample size for which the standard error of the strict empirical exceedance estimate is no more than 0.04. Then, if a sample of that size contains 18 exceedances, report the observed estimate.

Method. Use the standard error $\sqrt{p(1-p)/n}$ and require it to be at most 0.04. Squaring both nonnegative sides gives $$ \frac{0.20(0.80)}{n}\le (0.04)^2, \qquad \frac{0.16}{n}\le0.0016. $$

Rearranging yields $n\ge 0.16/0.0016=100$. Thus the smallest allowable integer sample size is 100. At $n=100$, the variance is $0.16/100=0.0016$, and the standard error is $\sqrt{0.0016}=0.04$. A sample size of 99 would give variance $0.16/99=0.00161616\ldots$ and standard error approximately 0.0402, which exceeds the target.

With 18 observed exceedances, the empirical estimate is $18/100=0.18$. This observed proportion differs from the population probability 0.20, as sampling variation permits. The sample-size calculation used the stated population probability; it did not use the realized estimate.

Check and common mistake avoided. The required inequality is on the standard error, so it must be squared before solving for $n$. Using $0.04$, rather than $0.04^2$, as the variance target would produce an incorrect and much smaller sample size.

Scope, Interpretation, and Common Traps

The factor $p(1-p)$ is largest when $p=0.50$, where it equals 0.25. Consequently, without further information about $p$, the variance of the estimator is at most $0.25/n$, and its standard error is at most $0.5/\sqrt{n}$. This bound is useful for conservative planning, but it does not replace the exact formula when $p$ is specified.

The formula relies on two essential assumptions: the losses share one distribution, and they are independent. If losses are dependent, the variance of the average includes covariance terms. If the observations have different exceedance probabilities, their indicator variances need not be identical. In either case, $p(1-p)/n$ cannot be applied without additional justification. The result also assumes exact loss observations; censored observations do not directly reveal whether the threshold was exceeded in every case.

Do not confuse loss variance with estimator variance. The formula $p(1-p)/n$ is for the sample proportion, not for the dollar amount $X$.
Do not omit the sample-size divisor. A single exceedance indicator has variance $p(1-p)$; the average of $n$ independent indicators has variance $p(1-p)/n$.
State the threshold convention. $X>t$ excludes ties at $t$, whereas $X\ge t$ includes them.
Separate the observed estimate from its expected value. The observed proportion need not equal $p$, even though its expectation under the model is $p$.

Key takeaway: For independent, identically distributed exact losses, an empirical tail estimate is an average of Bernoulli indicators. Its expectation is the relevant population tail probability, and its variance is that probability times its complement, divided by the sample size. This result is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Check Your Understanding

For each problem, identify the relevant exceedance probability before applying the variance formula.

Independent, identically distributed exact losses have probability 0.30 of being strictly greater than a fixed threshold. For a sample of 40, find the expectation and variance of the empirical strict exceedance estimate. (Answer: expectation $0.3000$; variance $0.00525$.)
A loss distribution has probability 0.35 below $250, probability 0.25 exactly at $250, and probability 0.40 above $250. For $n=25$, find the expectation and variance for both strict and inclusive estimates. (Answer: strict: expectation $0.4000$, variance $0.0096$; inclusive: expectation $0.6500$, variance $0.0091$.)
For independent losses, the probability of exceeding a threshold is 0.12. Find the variance and standard error of the estimate from $n=100$ observations. (Answer: variance $0.001056$; standard error approximately $0.0325$.)
A known exceedance probability is 0.24. What is the smallest sample size that makes the standard error no greater than 0.03? (Answer: $n=203$.)
In a sample of 50 independent, identically distributed exact losses, 9 strictly exceed the threshold. Calculate the observed estimate and its plug-in estimated standard error. (Answer: estimate $0.1800$; estimated standard error $\sqrt{0.18(0.82)/50}\approx0.0543$.)

What You'll Learn