Why Stratify a Subject-Level Bootstrap?
In a nonlinear mixed-effects model, the sampling unit for a subject-level bootstrap is the subject: each selected subject contributes their complete observed record, including all visits, concentrations, doses, and covariates. When a study has prespecified groups with unequal sizes, an additional decision is whether each bootstrap replicate should retain the observed number of subjects in every group. Stratified resampling makes that choice explicit.
For example, a study might enroll different numbers of participants in each treatment arm, or include sites with substantially different enrollment. An ordinary subject-level bootstrap draws from the pooled subjects. Its replicate group counts can therefore differ from the observed counts, and a small group can be represented by very few subjects or none. A stratified subject-level bootstrap instead resamples separately within each group. It preserves each observed group count while allowing the identities and records of subjects within that group to vary.
The model is then refit to the replicate using the same model specification and the same estimation procedure as for the original data. The bootstrap distribution of the refitted estimates describes how those estimates vary under this particular resampling scheme. It is not a distribution that automatically includes every possible source of study uncertainty: its meaning depends on what was held fixed.
What the Resampling Conditions On
Stratified resampling conditions on the observed stratum counts \(n_1,\ldots,n_G\). Thus it does not represent uncertainty about how many subjects would be enrolled in each group under a different sample. Within each stratum, however, it resamples from the empirical distribution of the observed subjects. The identities selected, and therefore the covariate patterns and outcomes carried by those subjects, can vary across replicates.
This distinction matters especially when group sizes differ by design. If treatment allocation was fixed or deliberately planned, retaining the arm counts can match the study design and target uncertainty conditional on that allocation. If the group proportions themselves are random features of the population or sampling process and the analysis target includes their variation, conditioning on the observed counts omits that component. The appropriate scheme follows the target of inference, not just which code is easiest to run.
The estimand also matters. Suppose an overall summary is formed by weighting stratum-specific quantities in proportion to the observed group sizes. Preserving those sizes retains that weighting in every replicate. If the scientific target instead gives equal weight to each center, or uses population proportions different from the study proportions, the analysis must define and apply that target weighting explicitly. Stratifying the resampling does not change an estimator’s weighting rule or turn a sample-weighted summary into a population-weighted one.
Worked Example: Resampling Unequal Treatment Arms
Consider a hypothetical study with 8 subjects in Arm A and 20 in Arm B. A stratified subject-level bootstrap replicate draws 8 subjects with replacement from the 8 observed Arm A subjects, and 20 with replacement from the 20 observed Arm B subjects. It therefore contains 8 Arm A draws and 20 Arm B draws, even if some original subjects are drawn more than once.
For any particular Arm A subject, the chance of not being selected in one draw is \(7/8\). The chance of being absent from all 8 draws is \( (7/8)^8 \approx 0.3436 \). Thus the chance that this subject appears at least once is \(1-(7/8)^8\approx0.6564\). By summing these inclusion probabilities across the 8 subjects, the expected number of distinct Arm A subjects represented is \(8[1-(7/8)^8]\approx5.251\).
The replicate still has 8 Arm A subject records as draws, but on average only about 5.251 distinct Arm A subjects contribute them. This is not a contradiction: repeated draws are part of sampling with replacement. The corresponding Arm B draw count remains 20, regardless of which subjects are repeated.
If each subject has several concentration records, each draw carries all of that subject’s records together. The procedure does not independently resample concentrations or visits. The resulting fit therefore retains the within-subject record structure while varying which subjects represent each arm.
Construction and Justification
A valid implementation begins by defining strata before resampling, using the study groups relevant to the target analysis. For every replicate, sample subject identifiers independently within each stratum, with replacement, and copy all records for each selected subject. Assign repeated copies distinct replicate-level subject identifiers so that the NLME software treats them as separate sampled subjects rather than merging their records into one subject. Keep group labels and record-level information attached to the selected subject’s copied records.
Record the prespecified group membership and \(n_g\) for every stratum. Do not redefine groups separately in each replicate based on outcomes or fitted values.
For each group \(g\), make \(n_g\) draws with replacement from its observed subjects. Keep the draws from different strata separate.
Copy every record for each draw, give duplicate draws distinct replicate subject identifiers, and retain the group membership and relevant design variables.
Apply the same model and estimation procedure to the replicate, recording the parameter estimates and any fitting problems for later assessment.
The collection of refitted estimates describes variability conditional on the chosen stratum counts and the within-stratum empirical distributions.
The conditional interpretation can be expressed with the law of total variance. Let \(T\) be an estimator and let \(C\) denote the vector of group counts. Then
A stratified bootstrap fixes \(C=(n_1,\ldots,n_G)\), so its resampling is aimed at the first component: variation in \(T\) given those counts. A scheme that allows group counts to vary can also reflect count variation, corresponding to the second component when the sampling design and estimator make that component relevant. This identity explains why the two schemes need not produce the same uncertainty, even when both resample whole subjects.
Worked Example: The Weighting Implied by Unequal Centers
Suppose a hypothetical study has three centers with \(n_1=4\), \(n_2=10\), and \(n_3=6\) subjects, and observed center-specific means of a parameter-related subject summary are 10, 12, and 16 units, respectively. The sample-size-weighted overall mean is
A stratified replicate keeps 4, 10, and 6 draws in those centers. If the overall statistic is calculated by pooling subject-level values, its center weighting remains proportional to those counts. By contrast, an equal-center average of the three means is \( (10+12+16)/3=12.67 \) units (to four significant figures). These are different summaries and can represent different targets. Stratified resampling preserves the counts; it does not decide which summary is scientifically appropriate.
If the intended target is an equal-center average, that statistic must be calculated accordingly in each replicate. If the intended target is a population average with known center proportions different from the observed sample proportions, the analysis must use those target weights. Merely resampling within centers does not supply them.
Choosing Between Stratified and Pooled Resampling
An unstratified subject-level bootstrap draws \(N\) subjects from the pooled sample. Conditional on the observed data, its stratum counts vary from replicate to replicate. For example, when the observed proportion in group \(g\) is \(n_g/N\), the number drawn from that group has a binomial marginal distribution with \(N\) draws and success probability \(n_g/N\). Stratification replaces this variation with fixed counts. Neither approach is universally preferable: the choice should represent the sampling design and the uncertainty relevant to the estimand.
| Feature | Stratified subject-level bootstrap | Pooled subject-level bootstrap |
|---|---|---|
| Group counts in each replicate | Fixed at observed \(n_g\) | Vary across replicates |
| Within-group subject composition | Resampled within each group | Can vary as a consequence of pooled draws |
| Useful when | Inference is conditional on planned or observed group counts | Group counts are part of the sampling variation of interest |
| Potential concern | Omits uncertainty in group proportions | Small groups may be underrepresented or absent in a replicate |
In a mixed-effects analysis, group membership might be a treatment factor in the fixed-effects model, a center effect, or a stratification variable chosen to protect important design groups. Whatever its role, the bootstrap must resample at the subject level and preserve the complete record for each selected subject. A stratified bootstrap does not correct model misspecification, establish exchangeability within groups, or justify a causal interpretation of treatment effects.
Worked Example: Why a Small Group Can Disappear Under Pooled Resampling
Suppose a hypothetical pooled sample contains 2 subjects in a small group and 18 in a larger group, for \(N=20\). In an unstratified bootstrap of 20 subject draws, the probability of drawing no subject from the small group is
So roughly 12.16% of such replicates omit the small group entirely. If the fitted model requires that group to estimate a group-specific parameter, those replicates may make the parameter unidentified or the fit unstable. Under stratified resampling, each replicate instead draws exactly 2 subjects from the small group and 18 from the larger group, so the group is always represented.
This protection does not create new information about the small group. With only two observed subjects, the resampled data can contain at most those two distinct subjects, and often repeats one or both. The resulting uncertainty assessment is constrained by the sparse original data; preserving a group count is not a substitute for adequate sample size.
Assumptions, Diagnostics, and Limitations
The central sampling assumption is that subjects are exchangeable within each stratum for the purpose of the target inference. In practical terms, the observed subjects in a group must be reasonable empirical representatives of the subjects to which inference is intended to apply, conditional on the features used to define the strata. The strata should be prespecified or justified by the design and estimand. If important subject-level sampling differences remain within a stratum, simple resampling within that stratum may not represent the intended sampling process.
Inspect the replicate composition and fitting results. Every replicate should have the intended number of subject draws in each stratum; every selected subject’s records should remain together; and duplicate draws should have distinct subject identifiers. Check whether parameter estimates are systematically missing or extreme, whether fits repeatedly encounter boundary estimates or convergence problems, and whether sparse groups yield unstable estimates. Such findings can indicate limited information, incompatibility between the model and some resampled data sets, or an implementation error. Do not interpret successful fits alone as proof that the resampling assumptions are sound.
Stratification can be unhelpful when there are many tiny groups: it guarantees their observed counts but cannot supply within-group diversity. It may also condition away meaningful group-count variability when the group proportions are part of the target sampling process. Conversely, pooled resampling may produce replicates with severely altered group composition, which is undesirable if group allocation was fixed by design or if a model requires all groups to be represented. The choice must be reported with the resulting conditional interpretation.
A common mistake is to say, “The groups were resampled separately, so the bootstrap accounts for uncertainty in the group proportions.” The reasoning is incorrect: each group contributes exactly \(n_g\) draws in every replicate, so the proportions \(n_g/N\) are fixed. The correction is to state that the bootstrap captures within-group subject resampling variation conditional on the observed group counts. If uncertainty in group proportions is relevant, the resampling design and target need to allow for it.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A study has 12 subjects in one arm and 28 in another. How many subject draws from each arm appear in every stratified replicate?
- Does stratified resampling with fixed center counts include uncertainty from changing the proportion of subjects recruited at each center? Explain.
- Why must all records for a selected subject be copied together in a subject-level NLME bootstrap?
- An unstratified bootstrap draws 30 subjects, of whom 3 of the observed 30 are in a small group. What is the probability that a replicate contains no subject from that group?
- If the target gives equal weight to each of four centers, does preserving their unequal observed counts automatically produce that target? What must the analyst do?
Solutions. (1) Each replicate has 12 draws from the first arm and 28 from the second, with replacement. (2) No. The center counts are fixed in every replicate, so uncertainty due to changing center proportions is conditioned out. (3) The subject is the sampling unit, and copying the complete record preserves the observed within-subject structure; resampling visits independently would break that structure. (4) The probability is \( (27/30)^{30}=0.9^{30}\approx0.04239 \), or about 4.239%. (5) No. The unequal counts retain unequal representation if the statistic pools subjects. The analyst must calculate an equal-center-weighted statistic in each replicate if that is the estimand.