Bootstrap Limitations For Rare Events And Tail Quantiles

Why Rare Events Challenge the Bootstrap

A nonparametric bootstrap estimates sampling variability by repeatedly drawing records, with replacement, from the observed sample. This works by treating the empirical distribution as a stand-in for the population distribution. For ordinary features of a data set, that can be useful. For a rare event or an extreme loss, however, the stand-in has a strict limitation: it contains only outcomes that were observed.

If an unusually severe claim did not occur in the sample, an empirical resample cannot produce one. If the sample contains one unusually severe claim, a resample may omit it, or include it multiple times, but cannot produce a claim larger than it. A bootstrap distribution can therefore describe the consequences of rearranging observed tail information, but it cannot reveal what lies beyond the observed tail.

Definition: In a nonparametric bootstrap of $n$ observations, each bootstrap sample consists of $n$ independent draws with replacement from the empirical distribution that assigns probability $1/n$ to each observed record. Its support is limited to the observed values. A bootstrap estimate of a tail quantile is therefore conditional on the tail values present in the original sample.

This distinction matters most when the target is a high quantile, such as the 99th percentile of claim severity, or the probability of a very rare event. Such quantities depend on a small portion of the distribution. A sample may contain few relevant observations—or none—so resampling those records does not supply the missing information.

How Many Tail Observations Inform a Quantile?

For a continuous loss distribution with cumulative distribution function $F_X$, the population $p$-quantile is commonly defined as $q_p=F_X^{-1}(p)$. For an ordered sample $X_{(1)}\le \cdots \le X_{(n)}$, one simple empirical convention is to estimate it by $X_{(\lceil np\rceil)}$. Quantile conventions can differ slightly, particularly in small samples, so the convention should be stated when exact calculations matter.

With this order-statistic convention, a 99th-percentile estimate from 100 observations is $X_{(99)}$, near the upper end of the sample. There is little data beyond that estimate to indicate how the population tail continues. A 99.9th-percentile estimate from the same sample is $X_{(100)}$, the sample maximum. The bootstrap cannot turn that maximum into a larger observed value.

Key takeaway: The approximate number of observations above the population $p$-quantile in a sample of size $n$ is $n(1-p)$. When this number is small, tail estimates rely on very few observations. When it is below one, the data provide especially little direct information about the region beyond the target quantile.

The number $n(1-p)$ is a diagnostic, not a guarantee that exactly that many observations will fall above the true quantile. Actual counts vary. But it quickly shows why a high quantile from a modest sample deserves scrutiny: a 99th-percentile target in a sample of 50 has only about half an observation expected above it.

What Resampling Does to a Rare-Event Count

Suppose $m$ of the $n$ observed records meet a specified rare-event condition. In a bootstrap sample, each draw meets that condition with probability $m/n$. Let $K^*$ be the number of event records in the resample. Because the $n$ draws are independent conditional on the original data, $K^*$ has a binomial distribution.

K^* \mid \text{observed data} \sim \operatorname{Binomial}\left(n,\frac{m}{n}\right), \qquad E[K^*\mid\text{data}]=m, \qquad \operatorname{Var}(K^*\mid\text{data})=m\left(1-\frac{m}{n}\right).

In particular, the probability that a bootstrap sample contains no event records is

P(K^*=0\mid\text{data})=\left(1-\frac{m}{n}\right)^n.

To justify this result, each individual draw avoids the $m$ event records with probability $1-m/n$. The resample contains no event records only if all $n$ independent draws avoid them. Multiplying those probabilities gives $(1-m/n)^n$. Thus, even when an event appears in the original sample, many resamples may contain none of the observed examples of it.

Worked Example: No Observed Claims Above a Rare-Event Threshold

A hypothetical insurer reviews 100 independent policy records and finds no claim above $500,000. For illustration, suppose the actual probability of exceeding that threshold is $0.01$, with independent Bernoulli event indicators. There is no interest discounting or mortality model. What does the empirical bootstrap say about the event probability, and how likely is this sample to have missed the event entirely?

Method. First identify the empirical distribution of the event indicator; then compare what it can resample with the stipulated data-generating probability. The common mistake is to read a bootstrap result based on zero observed events as evidence that the true probability is zero.

Empirical bootstrap result. All 100 observed event indicators are zero. The empirical event probability is $0/100=0$. Every bootstrap draw therefore selects a zero, and every bootstrap sample has event rate $0/100=0$. The bootstrap distribution of the event-rate estimate is concentrated entirely at zero.

Probability of observing no events under the stipulated model. With true event probability $0.01$, each record has probability $0.99$ of not exceeding $500,000. Independence gives

P(\text{zero exceedances})=(1-0.01)^{100}=0.99^{100}\approx 0.3660.

Thus, under the stated model, a sample with no exceedances occurs about 36.60% of the time. The empirical bootstrap remains at zero because it conditions on the observed sample; it does not reproduce the possibility of an event that the sample did not contain. The calculation checks both conclusions: every resample has zero events, while the stipulated population model assigns probability 0.3660 to seeing zero events in 100 observations.

Tail-Quantile Estimates Cannot Exceed the Observed Maximum

Let the observed losses be $X_1,\ldots,X_n$, and let $X_{(n)}$ be their maximum. Every individual draw in a nonparametric bootstrap is one of those observed losses. Consequently, every bootstrap order statistic is at most $X_{(n)}$. A bootstrap percentile interval for a loss quantile cannot extend above the observed maximum, even if the population has a substantial probability of losses beyond it.

This is not a defect in the mechanics of resampling. It is a limitation of the information supplied to the method. Bootstrap variation reflects which observed records are repeated or omitted; it does not represent uncertainty about unobserved values unless an additional model introduces that information.

Worked Example: A Bootstrap 95th Percentile From Ten Losses

A hypothetical portfolio has ten observed individual losses, in thousands of dollars: $5, 8, 12, 15, 20, 25, 30, 40, 60, 100$. Use the empirical order-statistic convention $X_{(\lceil np\rceil)}$ to estimate the 95th percentile and assess what the bootstrap can say about it. Treat the empirical distribution as assigning probability $1/10$ to each listed loss. No interest or mortality assumption is involved.

Method. Calculate the empirical quantile index, then determine the probability that the bootstrap maximum equals or falls below the largest observed loss. The common mistake is to interpret bootstrap replications as if they could generate a loss above $100,000.

Original-sample estimate. Here $n=10$ and $p=0.95$, so the index is

\lceil np\rceil=\lceil 10(0.95)\rceil=\lceil 9.5\rceil=10.

The empirical 95th percentile is therefore $X_{(10)}=100$ thousand dollars, or $100,000.

Bootstrap behavior. In each bootstrap sample of ten losses, the estimated 95th percentile is again the tenth order statistic: the bootstrap maximum. Each draw selects the $100,000 observation with probability $1/10$. The probability that at least one draw selects it is

P(\text{bootstrap maximum}=100) =1-\left(\frac{9}{10}\right)^{10} =1-0.3486784401 \approx 0.6513.

The probability that no draw selects the $100,000 observation is $0.9^{10}=0.3486784401$, or approximately 0.3487. In that case, the bootstrap maximum is at most $60,000, the second-largest observed loss. In every case, however, it is at most $100,000. The arithmetic confirms that the bootstrap can show considerable variation in which observed extreme is represented, but it cannot explore losses above the observed maximum.

A Single Extreme Record Can Be Unstable

When a sample contains only one exceedance, the empirical bootstrap has two competing behaviors: it can omit that record, or repeat it. It cannot create an independent new extreme. The resampled event count may look variable, but all of that variation is generated from one observed example.

Worked Example: One Large Claim in Forty Records

In a hypothetical set of 40 independent policy records, one claim exceeds $300,000 and the other 39 do not. Consider the bootstrap distribution of the number of exceedances in a resample of 40 records. Assume the empirical distribution assigns probability $1/40$ to the one exceedance record and $39/40$ to the others. There is no discounting or mortality adjustment.

Method. Use the binomial count result with $n=40$ and $m=1$. The common mistake is to treat the resamples as 40 new independent observations from the underlying population; conditional on the data, they instead draw only from the 40 observed records.

Expected count and variability. The bootstrap count $K^*$ follows

K^* \sim \operatorname{Binomial}\left(40,\frac{1}{40}\right), \qquad E[K^*]=40\left(\frac{1}{40}\right)=1,

\operatorname{Var}(K^*)=40\left(\frac{1}{40}\right)\left(\frac{39}{40}\right) =\frac{39}{40}=0.975.

The probability that a resample contains no exceedance is

P(K^*=0)=\left(\frac{39}{40}\right)^{40} =0.975^{40}\approx 0.3632.

So about 36.32% of resamples omit the only observed claim above the threshold. The resampled count has mean one, but this does not mean the bootstrap has observed many distinct severe claims: it still has only one source record for the exceedance. The results are consistent: the count varies across resamples, while no resample can contain a claim larger than the original largest claim.

Interpreting Bootstrap Results for Tail Risk

A narrow bootstrap interval for a high quantile is not, by itself, evidence that the population tail is well known. If all resamples are assembled from a small set of similar observed tail values, the interval may look stable because those values are repeatedly recombined. The method has no way to account for tail behavior that is absent from the data.

When reviewing a bootstrap tail estimate, ask:

How many observations are in the part of the sample relevant to the target quantile?
Does the estimated quantile equal, or nearly equal, the sample maximum?
How often do resamples omit the observed rare-event records?
Is the conclusion about the population tail being drawn from resampling variation alone?

If the data contain little tail information, possible responses include collecting more relevant observations, using external evidence with its assumptions made explicit, or fitting a justified tail model and examining sensitivity to its assumptions. A model-based method can extrapolate beyond observed losses, but that additional reach comes from the model, not from empirical resampling. The choice of model and its validation require separate care.

Common Exam Trap

Wrong calculation: “There were no claims above the threshold, so every bootstrap estimate is zero; therefore the population probability is zero.” This conclusion confuses a conditional resampling distribution with the unknown population distribution.

Correction: State what the bootstrap conditions on: the observed records. If no threshold exceedance was observed, the empirical bootstrap event rate is always zero. That fact describes the resampling procedure, not certainty about the true event probability. For a tail quantile, likewise report whether the estimate is driven by one or a few order statistics and recognize that no empirical resample can exceed the sample maximum.

Key takeaway: The nonparametric bootstrap can quantify variation arising from resampling observed records. It cannot manufacture missing rare events or unobserved extreme losses. For tail conclusions, report the amount of observed tail information and do not mistake conditional resampling variability for complete uncertainty about the population tail.

Check Your Understanding

For each problem, focus on what the empirical bootstrap can and cannot represent.

A sample contains 25 records, with one record meeting a rare-event condition. What is the probability that a bootstrap sample of 25 records contains no such record? (Answer: $(24/25)^{25}\approx 0.3604$.)
A data set has 12 distinct observed losses. What is the probability that a bootstrap sample of size 12 includes the original sample maximum at least once? (Answer: $1-(11/12)^{12}\approx 0.6480$.)
Among 50 observed records, two exceed a specified loss threshold. What is the probability that a bootstrap sample of size 50 contains no exceedances? (Answer: $(48/50)^{50}=0.96^{50}\approx 0.1299$.)
Using the convention $X_{(\lceil np\rceil)}$, what order statistic estimates the 99th percentile from a sample of 80 losses, and can a nonparametric bootstrap estimate exceed the sample maximum? (Answer: $\lceil 80(0.99)\rceil=\lceil79.2\rceil=80$, so the estimate is the sample maximum; no bootstrap estimate can exceed it.)
In 100 observed records, five exceed a threshold. What are the conditional bootstrap mean and standard deviation of the exceedance count in resamples of size 100? (Answer: Mean $=5$; variance $=100(0.05)(0.95)=4.75$; standard deviation $=\sqrt{4.75}\approx 2.179$.)

What You'll Learn