Why Resampling Structure Matters
A bootstrap replicate is only informative about uncertainty if its resampling units reflect how the data were generated and how the target population is defined. In a population PK analysis, observations may be grouped by treatment arm, site, trial, or another design feature. Measurements from subjects at the same site can share collection procedures or clinical practice; multiple records from one subject are also not independent observations. Resampling rows as if they were unrelated can therefore produce an uncertainty distribution that is too narrow or otherwise mismatched to the target.
The earlier material on the subject-level nonparametric bootstrap establishes the basic idea of resampling subjects with replacement and recomputing a statistic. Here the central decisions are more specific: what should be resampled together, and which groups should be kept distinct during resampling? The answer depends on the independent units and the estimand, not on which data layout is easiest to code.
Choosing Strata and Resampling Units
Stratification and clustering solve different problems. Stratifying by treatment group, for example, keeps the number of sampled subjects in each arm fixed at its observed value. This is useful when the analysis is conditional on the observed allocation and the target is a treatment contrast for those arm sizes. It does not resample treatment assignment, and it does not represent uncertainty in the population proportions of the arms.
Clustering addresses dependence among observations. If a trial has several sites and subjects within a site may share relevant features, a site-level bootstrap draws sites with replacement and brings along all eligible subject records from each selected site. If a subject contributes repeated PK observations, those observations must remain together when the subject is the independent unit. The choice between subject and site as the unit follows the source of sampling uncertainty and the population to which inference is intended to apply.
A combined design may use both ideas. For example, sites could be sampled within region strata, retaining all their subjects, or subjects could be sampled within treatment strata when independent subjects are the appropriate units. Before computing replicates, specify what stays fixed: stratum sample sizes, the number of clusters, cluster membership, treatment allocation, and any weights used in the estimator. The bootstrap then reproduces the chosen design rather than an abstract notion of “random sampling.”
Why Clusters Change the Uncertainty
Suppose there are \(G\) independent sites and \(m\) subjects per site, and the target is the average subject-level outcome. Write \(Y_{gj}\) for the outcome of subject \(j\) at site \(g\). The pooled sample mean is \(\bar{Y}=(Gm)^{-1}\sum_g\sum_jY_{gj}\). If two subjects at the same site have positive covariance, the variance of this mean includes their covariance; treating all \(Gm\) records as independent drops those terms.
The second sum contains within-site dependence. Resampling individual records separately breaks the observed groupings and generally cannot reproduce that contribution. Resampling sites carries each site’s internal pattern together, so variation among selected sites contributes to replicate-to-replicate variation. This does not make cluster resampling automatically correct: it is appropriate only when sites are the relevant independent sampled units and the estimator and target handle cluster sizes consistently.
Worked Example: Stratified Bootstrap for a Treatment Contrast
Suppose a hypothetical trial has four subjects in each of two treatment arms. The outcome is subject-level AUC, in mg·h/L. The observed treatment values are 10, 11, 15, and 17; the control values are 8, 8, 12, and 14. To estimate uncertainty for the difference in arm means conditional on the observed arm sizes, resample four subjects with replacement separately within each arm.
In one hypothetical bootstrap replicate, the treatment draw is 11, 15, 15, and 17, while the control draw is 8, 8, 12, and 14. Repeated values occur because sampling is with replacement. The replicate contrast is:
For comparison, the observed arm means are \(53/4=13.25\) mg·h/L and \(42/4=10.5\) mg·h/L, giving an observed contrast of 2.75 mg·h/L. The bootstrap replicate’s 4.0 mg·h/L is one possible resampled estimate, not a new estimate that replaces the observed contrast. Repeating the resampling gives a distribution used to describe sampling variability under this within-arm scheme.
This design preserves four subjects per arm in every replicate. It is suitable for the stated conditional target if subjects are the independent units within arms and the observed subjects represent the relevant arm populations. It would not represent uncertainty from randomizing subjects to arms, nor uncertainty in a target population’s treatment-arm proportions.
Worked Example: Resampling Sites as Clusters
Consider a hypothetical PK study with three sites, each contributing two subjects. Their subject-level clearance values, in L/h, are site 1: 4 and 6; site 2: 6 and 8; and site 3: 8 and 10. Suppose the target is the average subject clearance, and equal site sizes make the pooled mean equal to the mean of the site means. The site means are 5, 7, and 9 L/h.
A site-level bootstrap draw selects sites 2, 2, and 3. Each selected site contributes both of its subjects, so the replicate contains values 6, 8, 6, 8, 8, and 10. Its pooled mean is:
The duplicate site represents selecting the same observed cluster more than once. Its subjects are duplicated together, preserving the within-site grouping. A subject-level bootstrap could instead draw six individual records and scatter them across hypothetical sites; that would not reflect site-to-site variation in the same way.
There are only three sites in this illustration. Although the resampling mechanics are clear, three independent clusters provide little information about a broad site population. A large number of bootstrap replicates cannot compensate for having only three distinct sites in the original data.
Stratified Cluster Resampling
When clusters belong to design strata, sample clusters separately within each stratum. This retains the observed number of clusters in each stratum while allowing cluster composition to vary within it. The statistic may then combine stratum-specific results using prespecified weights. The weighting must match the estimand: an equal-region average, a subject-weighted population average, and an average site effect are different targets when cluster sizes or population proportions differ.
Worked Example: Sites Sampled Within Regions
Imagine a hypothetical study with two regions, each containing two sites. Each site contributes the same number of subjects, and the site-level mean clearance values are 8 and 10 L/h in the North, and 12 and 14 L/h in the South. The target is an equally weighted average of the two regional means. The observed regional means are 9 and 13 L/h, so the target estimate is \((9+13)/2=11\) L/h.
For a stratified cluster bootstrap, draw two sites with replacement within each region. Suppose the North draw is 10 and 10 L/h, and the South draw is 12 and 14 L/h. The replicate regional means are 10 and 13 L/h, and the replicate target is:
The replicate keeps two sites in each region and retains the equal regional weights in the target. It reflects variation among the observed sites within regions, conditional on the region structure. If the target instead weighted regions according to population sizes, those weights would need to be used in every replicate; equal weighting would answer a different question.
Implementation, Diagnostics, and Alternatives
A practical workflow makes the resampling design explicit before running the bootstrap:
Name the model parameter or summary and the population it describes, including any treatment, region, or site weighting.
Use the study design and data provenance to decide what must remain together and which groups should be sampled separately.
Sample the specified units with replacement within the chosen strata, carrying all required records and identifiers with each selected unit.
Apply the same estimator and target weights to each replicate, recording the result and any fitting or data-handling failure.
Confirm that sample counts, strata, clusters, and repeated records follow the intended scheme, and investigate unusual concentration of results or influential units.
Useful diagnostics include tabulating how often each cluster is selected, checking that all required strata appear, and confirming that repeated records remain attached to their original subject or site. Examine whether one or two clusters dominate the estimate, whether cluster sizes are highly unequal, and whether a stratum has only one or two independent units. A bootstrap that frequently produces an empty or unusable stratum signals a design problem, not merely a computational nuisance.
The main alternatives answer different questions. A subject-level bootstrap is appropriate when subjects are independent sampling units and within-subject records are kept together. Stratifying that bootstrap by treatment or another fixed group preserves observed group counts. A cluster bootstrap is preferable when the independent sampling units are sites, trials, or other groups and dependence within groups matters. A parametric bootstrap draws from a specified fitted model rather than resampling empirical units; it relies on that model’s assumptions and is a distinct approach. None of these choices is a universal correction for a poorly represented sample.
For unequal cluster sizes, decide whether the target weights subjects, sites, or another unit. Simply averaging cluster means gives each cluster equal weight; pooling all records gives larger clusters more influence. These can both be legitimate, but they estimate different quantities. Likewise, resampling clusters while holding all observed subjects within each selected cluster fixed treats within-cluster sampling differently from a procedure that resamples subjects within selected clusters. Use multistage resampling only when that matches the sampling design and target.
Common Mistakes and Limitations
- Resampling rows from clustered data. This breaks dependence and can understate uncertainty when within-cluster outcomes are positively correlated. Keep together the unit whose members may be dependent.
- Assuming stratification captures every source of uncertainty. Sampling subjects within fixed treatment groups conditions on observed group sizes. It does not account for uncertainty in those sizes or group proportions.
- Resampling too few clusters and trusting a large replicate count. More replicates reduce simulation noise in the bootstrap summary; they do not create new independent sites or trials.
- Ignoring the estimand when cluster sizes differ. Equal weighting of site means and pooling all subjects generally produce different estimates. Choose weights based on the target, not convenience.
- Treating every observed grouping as a cluster. A label alone does not establish dependence or independent sampling at that level. Use the design and data-generating context to justify the resampling unit.
The central limitation is that nonparametric resampling can only reuse patterns present in the observed independent units. It cannot reliably reveal variation across sites when only a few sites were sampled, or variation across trials when the data come from one trial. It also does not repair biased enrollment, informative missingness, or an estimand that was not defined clearly.
Check Your Understanding
Answer each question, then compare with the solution immediately below it.
- A study has 40 subjects at each of four sites, with possible site-level differences in clinical practice. The target is uncertainty in a population-average clearance. What is a defensible resampling unit, and what is a key limitation if only four sites were sampled?
Solution: Resample sites with replacement and retain each selected site’s subject records, provided sites are the independent sampled units for the target. Four clusters give weak information about site-to-site variation; many bootstrap replicates do not remove that limitation. - A two-arm trial estimates the difference in mean AUC and wants to condition on the observed arm sizes. How should subjects be resampled, and what uncertainty is not represented?
Solution: Resample subjects with replacement separately within each arm, preserving each observed arm size. This does not represent uncertainty in treatment allocation or in population arm proportions. - Two regions each have two site means. North: 5 and 7 mg·h/L; South: 9 and 11 mg·h/L. Resample two sites within each region. A replicate selects North values 7 and 7, and South values 9 and 11. What is the equally weighted regional estimate?
Solution: The replicate regional means are 7 and \((9+11)/2=10\) mg·h/L. The equally weighted estimate is \((7+10)/2=8.5\) mg·h/L. - A cluster bootstrap draws sites, but the analysis averages the site means equally even though the target is the mean clearance for all patients in a population with unequal site sizes. What should be checked?
Solution: Check whether equal site weighting matches the target. If the target is patient-weighted, the estimator must reflect patient weights or population site-size weights; the unweighted mean of site means generally estimates a different quantity.