When IID Resampling Fails: Dependence And Clustering

Why the Resampling Unit Matters

A bootstrap draws observations from the data to approximate how an estimator would vary across repeated samples. The key question is not simply how many rows are in the data set. It is which units could reasonably have been sampled independently. If several records share a household, policyholder, employer, or catastrophe, their outcomes may be related. Resampling those records independently can create combinations that could not have occurred in the original sampling process and can misstate uncertainty.

For example, two policies in the same household may share risk characteristics, while multiple claims from one storm may reflect a common event. Treating each policy or claim as an independent draw ignores that shared source of variation. The resulting bootstrap standard error is often too small when the dependence is positive, although the direction and size of the error depend on the dependence structure and statistic.

The methods in this tutorial are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus. The aim is to connect the data structure to the resampling design before calculating an uncertainty measure.

Definition: A cluster is a group of observations that may be dependent within the group, while the groups themselves are treated as independent sampling units. In a cluster bootstrap, one samples whole clusters with replacement and retains every observation in each selected cluster. The sampling unit is the independently sampled unit, not necessarily one row of data.

How Dependence Changes the Variance

Consider $K$ independent clusters, each containing $m$ observations $Y_{ij}$, where $i=1,\ldots,K$ indexes clusters and $j=1,\ldots,m$ indexes observations within a cluster. Suppose every observation has mean $\mu$ and variance $\sigma^2$, and any two distinct observations in the same cluster have correlation $\rho$. The pooled sample mean is $\bar{Y}=(Km)^{-1}\sum_i\sum_jY_{ij}$.

The following variance expression assumes equal cluster sizes, equal marginal variances, a common within-cluster correlation, and independence between clusters. It is not a general formula for unequal cluster sizes or arbitrary dependence.

\operatorname{Var}(\bar{Y}) = \frac{\sigma^2}{Km}\left[1+(m-1)\rho\right].

To see why, the variance of the sum within one cluster contains $m$ individual variances and $m(m-1)$ ordered covariance terms. Each covariance is $\rho\sigma^2$. Independence across clusters means the cluster variances add. Dividing the variance of the total by $(Km)^2$ gives the variance of the pooled mean.

\begin{aligned} \operatorname{Var}\left(\sum_{i=1}^{K}\sum_{j=1}^{m}Y_{ij}\right) &=K\left[m\sigma^2+m(m-1)\rho\sigma^2\right],\\ \operatorname{Var}(\bar{Y}) &=\frac{K m\sigma^2[1+(m-1)\rho]}{(Km)^2} =\frac{\sigma^2}{Km}[1+(m-1)\rho]. \end{aligned}

If the $Km$ observations were incorrectly treated as independent, the variance formula would omit the bracketed factor. That factor, $1+(m-1)\rho$, is called the design effect under these equal-size, common-correlation assumptions. With positive within-cluster correlation, the effective information is less than the row count suggests. When $\rho=0$, the design effect is 1; when $\rho>0$, it exceeds 1.

Worked Example: Quantifying the Effect of Household Clustering

Method: Use the equal-cluster design effect to compare the correct variance with the independence calculation. A hypothetical insurer observes claim indicators for 18 policies in 6 independently sampled households, with 3 policies per household. Each indicator has a hypothetical claim probability $p=0.20$, and the within-household correlation is assumed to be $0.50$. Find the standard error of the pooled claim proportion under the stated model, then compare it with the incorrectly calculated independent-observation standard error. There is no interest or mortality assumption; the outcome is a nonlife claim indicator.

For a Bernoulli indicator with $p=0.20$, the variance is $\sigma^2=p(1-p)=0.20(0.80)=0.16$. Here $K=6$, $m=3$, and $\rho=0.50$, so the design effect is $1+(m-1)\rho=1+2(0.50)=2$.

The variance of the pooled proportion is therefore $0.16/(6\cdot3)\times2=0.017777\ldots$, and its standard error is $\sqrt{0.017777\ldots}=0.1333$. Probabilities and standard errors are rounded to four decimal places.

If the 18 policies were incorrectly treated as independent, the variance would be $0.16/18=0.008888\ldots$, giving a standard error of $0.0943$. The ratio of the clustered standard error to the independent-observation standard error is $\sqrt{2}=1.4142$.

Check: The clustered variance is exactly twice the independence variance, as the design effect requires; its standard error is therefore $\sqrt{2}$ times as large. The common mistake is to count all 18 policies as independent units. The 18 rows do not represent 18 independent draws under the stated model.

Choosing and Resampling the Unit

The appropriate resampling unit follows from the sampling design and the target of the analysis. If independently sampled households each contribute several policies, resample households and retain all the policies in each selected household. If policies are independently sampled but each policy contributes a claim count and several claim severities, resample the complete policy-level record so its frequency and severity information stays together. If independent catastrophe events generate many claims, resample event-level groups when the target concerns variation across events.

After selecting clusters, calculate the original estimator on the resampled data. If cluster sizes differ, the correct calculation depends on the estimand. A mean loss per policy generally pools the policies in the selected clusters and divides the total loss by the total number of selected policies. It is not automatically the unweighted mean of the selected household averages: those estimate different quantities when household sizes differ.

Identify the dependence structure.
Ask which records share a household, policy, event, or other common source of variation. Use the data collection process, not only the table layout.

State the target and independent unit.
Specify what the estimator measures, such as average loss per policy, and which units were independently sampled.

Resample complete units.
Draw clusters with replacement and keep each selected cluster’s observations together. A cluster drawn twice appears twice in the replicate.

Recalculate the estimator.
Apply the same estimand to each resampled data set, including the appropriate weighting when cluster sizes differ.

Worked Example: Recomputing a Policy-Weighted Mean

Method: Resample households as whole clusters, then calculate total claim amount divided by total policy count. A hypothetical sample contains three independent households. Household A has two policies with losses of $100 and $300; household B has one policy with loss $0; household C has two policies with losses of $200 and $400. Losses within a household may be dependent. Find the original mean loss per policy and the estimate in a bootstrap replicate that selects A, B, and B. There is no interest or mortality assumption; these are hypothetical nonlife losses.

The original total loss is $100+300+0+200+400=\$1{,}000$, and the original policy count is $2+1+2=5$. Thus the original policy-weighted mean is $\$1{,}000/5=\$200$ per policy.

In the specified bootstrap replicate, A is selected once and B is selected twice. The replicate contains A’s two policies and two copies of B’s one-policy cluster: four policies in total. Its total loss is $100+300+0+0=\$400$. The replicate estimate is therefore $\$400/4=\$100$ per policy.

A common mistake is to average the three selected household means without regard to the number of policies. Those means are $200, $0, and $0, giving an unweighted average of $\$200/3=\$66.67$. That is not the policy-weighted mean requested. The replicate has four policy records, and its pooled loss divided by four is $100. Repeating B correctly contributes another copy of its entire cluster.

Clustered Data and Serial Dependence

Not all dependence is naturally divided into a fixed collection of groups. Monthly losses, claim counts, or development observations may be serially dependent: nearby periods can be more alike than distant periods. Resampling individual months independently breaks the ordering and dependence. When the data are a time series, a common approach is to resample contiguous blocks of observations rather than isolated time points. The block length must be chosen to retain meaningful local dependence; blocks that are too short can break it, while very long blocks leave few distinct pieces to resample.

This differs from a cluster bootstrap in its structure. A household or catastrophe cluster is a natural group that is retained whole. A time-series block is usually a segment of an ordered sequence, and its purpose is to preserve local dependence. Neither approach makes dependent groups independent by itself. The variance interpretation still relies on a sufficient number of effectively independent clusters or blocks.

Worked Example: Comparing Row and Event Resampling

Method: Compare the exact conditional variance of the bootstrap mean under row resampling and event-cluster resampling. A hypothetical portfolio has four independent storm events, each affecting two policies. The two policies in each event have the same loss: respectively, $10,000, $20,000, $30,000, and $40,000. Treat policies within an event as perfectly dependent for this illustration. Estimate the mean loss per policy. There is no interest or mortality assumption.

The eight policy losses sum to $2(10{,}000+20{,}000+30{,}000+40{,}000)=\$200{,}000$, so the observed mean is $\$200{,}000/8=\$25{,}000$.

For an ordinary row bootstrap, the empirical distribution assigns probability $1/4$ to each of the four loss values. Its variance is $[(-15{,}000)^2+(-5{,}000)^2+(5{,}000)^2+(15{,}000)^2]/4=125{,}000{,}000$ dollars squared. A bootstrap replicate draws eight rows independently from this empirical distribution, so the conditional variance of its mean is $125{,}000{,}000/8=15{,}625{,}000$. The resulting standard error is approximately $3,953.

For an event-cluster bootstrap, each draw is one of the four event losses and the selected event’s two policies remain together. The empirical variance of the four event-level losses is also $125{,}000{,}000$ dollars squared. A replicate draws four events, so its mean has conditional variance $125{,}000{,}000/4=31{,}250{,}000$, giving a standard error of approximately $5,590.

The event-level standard error is $\sqrt{2}$ times the row-level value because each event contributes two perfectly dependent policy observations. The row bootstrap can combine one policy from an event with policies from other events as though the two records were independent draws. That breaks the stated structure and understates the bootstrap variation here. The example’s small number of events also means the cluster bootstrap distribution has limited support; preserving dependence does not remove the limitations of having only four independent events.

Limits and Exam Traps

A cluster bootstrap is appropriate only when the clusters can reasonably be treated as independent sampling units for the analysis. If clusters share a common exposure or are themselves dependent, merely resampling them separately does not solve the problem. Likewise, a very small number of clusters can make bootstrap uncertainty unstable: resampling cannot create new independent information that was not observed.

Before interpreting a bootstrap standard error, check what one draw represents. A large row count can be misleading when many rows come from a small number of households or events. Conversely, grouping observations that were in fact independently sampled can discard information and produce an unnecessarily coarse resampling scheme.

Trap: equating rows with independent units. A file can have thousands of claim rows but far fewer independent events or policyholders.
Trap: breaking a cluster apart. Resampling individual policies when household membership creates dependence can generate unrealistic bootstrap samples.
Trap: changing the estimand by averaging cluster summaries. An unweighted mean of household means is not generally the same as a pooled policy-level mean.
Trap: assuming clustering always increases uncertainty. Positive within-cluster correlation increases variance under the stated design-effect assumptions; other dependence patterns and estimators need not follow that simple comparison.

Key takeaway: Resample the units that were independently sampled, retaining dependent observations together. Then recompute the estimator for the stated target. Correctly identifying the sampling unit is essential: an observation-level bootstrap can give a precise-looking answer while representing the wrong dependence structure.

Check Your Understanding

For each problem, identify the relevant unit and keep the target statistic distinct from the resampling design.

A hypothetical portfolio has four independent repair-shop clusters, each with five policies. The within-shop claim-indicator correlation is assumed to be $0.25$. If an independence calculation gives a standard error of $0.0300$ for the pooled claim proportion, calculate the design effect and the cluster-adjusted standard error. (Answer: Design effect $=2.00$; adjusted standard error $=0.0300\sqrt{2}=0.0424$, to four decimals.)
A sampled set of households contains several auto policies per household, and policies in the same household may have dependent claim costs. The target is mean claim cost per policy. Describe the bootstrap draw and the statistic to calculate when household sizes differ. (Answer: Resample whole households with replacement, retain all policies in each selected household, and divide the total selected claim cost by the total selected policy count.)
Three independent event clusters have respective mean losses of $4,000, $8,000, and $12,000 per policy; each event contributes the same number of policies, and losses within an event may be dependent. Calculate the mean of the cluster means and the conventional standard error based on the three independent cluster means. (Answer: Mean $=\$8{,}000$; cluster-mean sample variance $=16$ million dollars squared, so standard error $=\sqrt{16/3}$ thousand dollars $=\$2{,}309$, to the nearest dollar.)
Three independent households have policy losses H1: $0 and $200; H2: $100; H3: $300 and $500. A cluster-bootstrap replicate selects H1, H3, and H3. Calculate its policy-weighted mean loss. (Answer: Total loss $=0+200+300+500+300+500=\$1{,}800$, total policies $=6$, mean $=\$300$ per policy.)
A hypothetical insurer records monthly claim totals over several years, and adjacent months are expected to be dependent. Explain why resampling individual months is problematic and identify a suitable basic resampling unit. (Answer: Independent month draws break the serial ordering and local dependence; resample contiguous time blocks so nearby months remain together.)

What You'll Learn