Why Control Subgroup Representation?
A covariate simulation may need to represent groups that occur at very different rates in the target population. If a clinically important subgroup is uncommon, a sample drawn in proportion to population prevalence can contain too few members of that subgroup to estimate its predictions reliably. Stratified sampling addresses this by setting the number of simulated records in each subgroup deliberately. The resulting sample can be useful for estimating subgroup-specific predictions, but its subgroup proportions generally do not describe the target population.
The central distinction is between how many records to simulate in each subgroup and how much each subgroup contributes to a target-population prediction. The first is a sampling allocation; the second is determined by the target population’s subgroup prevalences. Treating these as the same quantity can produce a biased population summary even when every subgroup prediction is calculated correctly.
Specify the Target Before Choosing the Allocation
A subgroup sampling plan begins by defining the estimand: the prediction or summary the simulation is intended to represent. Suppose \(X\) denotes an individual’s covariates and \(m(X)\) is a model prediction for that individual under specified model inputs. If the target is the population-average prediction, it is the expectation of \(m(X)\) over the target covariate distribution. Partitioning that expectation by subgroup gives a useful identity:
This identity explains why allocation and weighting are separate. Stratification can improve the precision with which the conditional means \(E\{m(X)\mid G=g\}\) are estimated. The target prevalences \(\pi_g\), not the chosen counts \(n_g\), determine how those means contribute to the population prediction.
The strata must be defined so that every target-population record belongs to exactly one stratum. The subgroup definitions, target prevalences, and conditional covariate distributions should all refer to the same target population and time frame. Within each subgroup, the simulated records must represent the intended conditional distribution of covariates \(X\mid G=g\). If, for example, the subgroup is defined by renal function category, sampling the category correctly is not sufficient if age or body size is then sampled from an inappropriate distribution within that category.
The target prevalence may come from a specified target dataset, a stated population assumption, or a design requirement. If it is unknown, a population-weighted prediction cannot be identified from the stratified sample alone: the sample allocation does not reveal target prevalence. In that case, report subgroup predictions separately or state and justify an assumed set of prevalences.
Allocate Records, Then Weight Predictions
Choose each \(n_g\) for the task at hand. Equal allocation can be useful when subgroup-specific results are important, especially when some groups are uncommon. A larger allocation may be assigned to groups with greater within-group variability or to groups for which greater precision is required. The allocation does not have to mimic population prevalence.
For simulated records \(i=1,\ldots,n_g\) in subgroup \(g\), let \(m_{ig}\) be the model prediction for record \(i\). The estimated subgroup mean prediction is \(\bar{m}_g=n_g^{-1}\sum_{i=1}^{n_g}m_{ig}\). The target-population estimate is then the weighted sum of subgroup means:
Equivalently, each simulated record in subgroup \(g\) can receive weight \(\pi_g/n_g\). The record weights sum to one because \(\sum_g\sum_{i=1}^{n_g}\pi_g/n_g=\sum_g\pi_g=1\). If subgroup-specific predictions are nonlinear functions of covariates, calculate predictions for the sampled records and average within subgroup; in general, predicting at an average covariate value is not equivalent to averaging individual predictions.
Worked Example: Equal Quotas, Unequal Target Prevalences
Suppose a hypothetical target population is 80% in subgroup A and 20% in subgroup B. A simulation deliberately draws 50 covariate records from each subgroup. The mean predicted response is 70 percentage points in A and 50 percentage points in B. Each mean is assumed to represent its subgroup’s conditional covariate distribution.
The simulation allocation is 50% A and 50% B, but the target prevalences are \(\pi_A=0.80\) and \(\pi_B=0.20\). The target-population prediction is:
The unweighted average of all 100 simulated predictions is \((50\times70+50\times50)/100=60\%\). That 60% estimates the average under the simulation’s 50:50 allocation, not under the target population’s 80:20 composition. For record-level weighting, each A record receives \(0.80/50=0.016\) and each B record receives \(0.20/50=0.004\). The weights sum to \(50(0.016)+50(0.004)=1\), and the weighted prediction is 66%.
Justification and Monte Carlo Precision
The weighted estimator follows directly from the subgroup decomposition. If the records within subgroup \(g\) are independent draws from the target conditional covariate distribution \(X\mid G=g\), then \(\bar{m}_g\) estimates the conditional mean prediction for that subgroup. Multiplying by \(\pi_g\) and summing estimates the population mean because the target population is a mixture of those conditional distributions with mixture proportions \(\pi_g\).
A useful Monte Carlo standard error can also be derived. Assume that records are sampled independently within each subgroup, the subgroup samples are independent of one another, and the subgroup prediction has finite variance \(\sigma_g^2\). Since the variance of a sample mean of \(n_g\) independent draws is \(\sigma_g^2/n_g\), independence gives:
Replacing each \(\sigma_g\) with the within-subgroup sample standard deviation \(s_g\) gives the estimated Monte Carlo standard error \(SE(\widehat{M}_{\mathrm{target}})=\sqrt{\sum_g\pi_g^2s_g^2/n_g}\). This quantifies sampling variability from drawing covariate records under the stated design. It does not, by itself, include uncertainty in model parameters, target prevalences, or the model structure.
Worked Example: Weighted Prediction and Its Monte Carlo Standard Error
In a hypothetical simulation, the target prevalences of three mutually exclusive subgroups are 0.50, 0.30, and 0.20. The simulation draws 100, 60, and 40 records, respectively. The subgroup mean predictions are 0.42, 0.55, and 0.68, and their within-subgroup standard deviations are 0.10, 0.12, and 0.08.
The weighted target prediction is:
Under independent sampling within and across subgroups, the estimated variance is:
Thus \(SE=\sqrt{0.0000530}=0.00728\). The weighted prediction is 0.511, with estimated Monte Carlo standard error 0.00728 prediction units. This standard error describes the precision of this simulation estimate under the specified sampling assumptions; it is not a measure of model uncertainty.
Allocation Choices and Alternatives
Stratified allocation is especially useful when the purpose is to estimate predictions in every subgroup or when a low-prevalence subgroup would otherwise be represented by very few records. It also makes subgroup sample sizes explicit and can support repeatable, auditable simulation plans. However, no single allocation is best for every objective.
| Approach | How records are allocated | Main implication |
|---|---|---|
| Prevalence-proportional sampling | Choose \(n_g/n\) approximately equal to \(\pi_g\). | The unweighted sample average reflects target composition, subject to sampling variation; rare groups may have small samples. |
| Controlled stratified sampling | Choose \(n_g\) to meet subgroup representation or precision needs. | Use target-prevalence weights for a target-population summary if allocation proportions differ from \(\pi_g\). |
| Unstratified sampling from a target covariate distribution | Draw complete covariate records directly from the target distribution. | Can represent the target population directly, but subgroup counts are not controlled and may be unstable for rare groups. |
For a target mean, an allocation with more records in a subgroup can reduce its contribution to Monte Carlo error, but the benefit depends on both \(\pi_g\) and \(\sigma_g\). From the variance expression, the contribution of subgroup \(g\) is \(\pi_g^2\sigma_g^2/n_g\). Consequently, allocating the same number to every group is a convenient design choice, not a universal precision optimum. More elaborate allocation choices are only useful if the target and computational objective are clearly stated.
The comparison also clarifies a common error: a prevalence-proportional sample and a controlled stratified sample may both estimate the same target quantity, but their raw averages have the same interpretation only when their sample proportions match the target prevalences. The weights must correspond to the population being summarized, not to a preference for a visually balanced sample.
Diagnostics and Failure Modes
Before interpreting a weighted result, check the sample construction and the weights. Useful checks include:
- Confirm stratum definitions. Verify that the subgroups are mutually exclusive and cover the target population. Check that every simulated record is assigned to exactly one intended subgroup.
- Compare allocation with target prevalence. Report both \(n_g\) and \(\pi_g\). A large difference is not inherently wrong, but it makes unweighted pooling inappropriate for a target-population estimate.
- Check conditional covariates. Within each stratum, verify that the sampled covariate records represent the intended conditional distribution. Oversampling a subgroup does not correct a distorted distribution within that subgroup.
- Check every weight and its sum. With record weights \(\pi_g/n_g\), weights should be nonnegative and sum to one when all target strata have positive sample sizes and the target prevalences sum to one.
- Inspect subgroup precision. A stratum with a small \(n_g\), high prediction variability, or a small but important target prevalence may contribute substantial Monte Carlo error. Report subgroup results and their sample sizes, not just the pooled estimate.
An empty stratum with \(\pi_g>0\) is a serious failure: its subgroup mean cannot be estimated from the sample, and the target-population prediction cannot be computed by the stated estimator without additional information or modeling assumptions. Likewise, assigning zero weight to a subgroup just because it was inconvenient to sample silently changes the target. If records are drawn without replacement from a finite list of observed records, account for the actual sampling design when quantifying uncertainty; the independent-draw standard error above does not automatically apply unchanged.
Worked Example: A Rare Subgroup in a Balanced Simulation
Consider a hypothetical target population with subgroup prevalences 0.70, 0.25, and 0.05. For subgroup analysis, a simulation draws 40 records from each group. Their mean predicted concentrations are 2.1, 3.0, and 5.0 mg/L, respectively. Assume the draws represent each group’s target conditional covariate distribution.
The unweighted average is \((2.1+3.0+5.0)/3=3.367\) mg/L, which summarizes an equally weighted mixture of the three groups. The target-weighted prediction is:
The rare subgroup is intentionally represented by 40 of 120 records, more than its 5% target prevalence would imply. That is useful for estimating its subgroup prediction, but it does not make the subgroup one-third of the target population. Each simulated record receives weight 0.0175, 0.00625, or 0.00125 according to its subgroup; the respective total weights are 0.70, 0.25, and 0.05.
Common Mistake and Practical Limits
Wrong reasoning: “The simulation has equal numbers in each subgroup, so the overall average should give each subgroup equal weight.” This conclusion confuses the design used to obtain enough subgroup records with the composition of the population the result is meant to represent. Equal weights are appropriate only if the target population itself assigns equal prevalence to those groups, or if the intended estimand is explicitly an equally weighted average of subgroup predictions.
Correction: State the target estimand and target prevalences first. Use the chosen allocation to estimate subgroup-specific predictions, then combine those predictions with the target prevalences. If the target prevalences are uncertain, present that uncertainty or report results under clearly stated alternative prevalence assumptions; the simulation quotas cannot supply the missing information.
Stratified sampling also has limits. It cannot compensate for an incorrect model, an unsupported conditional covariate distribution, or a subgroup definition that does not match the intended clinical question. A well-weighted estimate can still be misleading if the covariates within a stratum are sampled incorrectly. Moreover, the weighted average is specific to the target prevalences used; changing the target composition can change the summary even when all subgroup predictions remain unchanged.
Check Your Understanding
Work each problem, then compare with the solutions below.
- A target population is 90% subgroup A and 10% subgroup B. A simulation draws 30 records from each group, with mean predictions of 4 and 7 units. What is the target-weighted prediction, and what does the unweighted average represent?
- For two subgroups, \(\pi_1=0.6\), \(\pi_2=0.4\), \(n_1=80\), and \(n_2=20\). What record weight should be assigned to each record in each subgroup? Do the weights sum to one?
- Why can’t target prevalences be inferred from a deliberately balanced stratified sample when the allocation was chosen independently of population composition?
- Suppose one target subgroup has prevalence 0.15 but no records were simulated from it. Can the weighted target prediction be calculated from subgroup means alone? Explain.
Solutions. (1) The target-weighted prediction is \((0.90)(4)+(0.10)(7)=3.6+0.7=4.3\) units. The unweighted average is \((4+7)/2=5.5\) units, an equally weighted subgroup mixture, not the 90:10 target mixture. (2) Each subgroup 1 record receives \(0.6/80=0.0075\); each subgroup 2 record receives \(0.4/20=0.0200\). The total is \(80(0.0075)+20(0.0200)=0.6+0.4=1\). (3) The sample counts are set by the allocation plan, so they describe that plan rather than the population; external target information or an explicit prevalence assumption is needed. (4) No. Its conditional mean prediction is missing, and its positive target prevalence means it contributes to the target summary. Simulate that subgroup or provide a justified alternative estimate before calculating the full weighted prediction.