Sensitivity of Population-Level Decisions to Subgroup Assumptions

Why Subgroup Assumptions Can Change a Population Decision

A treatment can have different predicted effects in different patient subgroups. Even when the subgroup-specific predictions stay fixed, the population-level prediction changes if the proportions of patients in those subgroups change. Conversely, the population composition can stay fixed while alternative assumptions about subgroup response shift the aggregate prediction. A population decision therefore depends on both who is expected to receive treatment and what treatment is predicted to do in each group.

This tutorial focuses on that dependence. It does not establish that subgroup differences are causal or that one prevalence assumption is correct. Instead, it shows how to make the consequences of specified assumptions visible, and how to identify when those assumptions could alter a decision.

Definition: Suppose a target population is partitioned into \(K\) mutually exclusive and exhaustive subgroups. Let \(w_k\) be the proportion of the target population in subgroup \(k\), and let \(\theta_k\) be the subgroup-specific prediction on a common outcome scale. The population-level prediction is the prevalence-weighted average \(\theta_{\mathrm{pop}}=\sum_{k=1}^{K}w_k\theta_k\), where \(w_k\geq 0\) and \(\sum_{k=1}^{K}w_k=1\).

For two subgroups, call subgroup H the subgroup of interest, let \(q\) be its prevalence, and let \(1-q\) be the prevalence of subgroup L. If their predictions are \(\theta_H\) and \(\theta_L\), then

\theta_{\mathrm{pop}}(q)=q\theta_H+(1-q)\theta_L

The formula applies to quantities that can be averaged with population weights, such as a mean outcome or an absolute risk. The predictions must refer to the same outcome definition, time horizon, and treatment contrast. For example, subgroup-specific absolute risk reductions can be combined to obtain the population absolute risk reduction, provided they concern the same event and follow-up period.

What Must Be Assumed Before Aggregating

A calculation can be arithmetically correct but scientifically misleading if its inputs do not describe the same target population and outcome. Before interpreting an aggregate, check the following conditions:

The subgroups cover the target population. They must be defined so each target patient belongs to one and only one subgroup. If categories overlap or leave patients unclassified, the weights do not form a valid partition.
The prevalence weights match the decision population. A trial’s subgroup proportions are not automatically the proportions expected in a future clinical population. If a different target population is relevant, its prevalence assumptions must be used explicitly.
The subgroup predictions are comparable. The endpoint, time horizon, treatment contrast, and scale must align. Combining a 12-week response rate in one subgroup with a 24-week outcome in another is not a meaningful weighted average.
The predictions are appropriate inputs. Weighting does not correct biased subgroup estimates, an unsuitable model, or unsupported assumptions about treatment response. The calculation itself does not establish that treatment causes the predicted outcome.

When weights or subgroup responses are uncertain, it is useful to calculate the aggregate under multiple stated assumptions. The earlier material on probabilistic sensitivity analysis provides a framework for sampling uncertain inputs; here the emphasis is on understanding how subgroup prevalence and response assumptions shape a population conclusion. A set of named scenarios is a comparison of assumptions, not automatically a probability distribution over those assumptions.

Deriving the Effect of Subgroup Prevalence

For two subgroups with fixed predictions, the population prediction is a linear function of the prevalence \(q\). Rewriting it shows which subgroup drives the change:

\theta_{\mathrm{pop}}(q) =q\theta_H+(1-q)\theta_L =\theta_L+q(\theta_H-\theta_L)

Therefore, increasing the prevalence of H by \(\Delta q\) changes the population prediction by \(\Delta q(\theta_H-\theta_L)\). The direction depends on the difference between subgroup predictions; the magnitude depends on both that difference and the size of the prevalence shift.

\theta_{\mathrm{pop}}(q+\Delta q)-\theta_{\mathrm{pop}}(q) =\Delta q(\theta_H-\theta_L)

If \(\theta_H=\theta_L\), changing the prevalence has no effect on the aggregate. If the subgroup predictions differ, composition matters. This is a sensitivity identity, not evidence that changing the population composition would itself change an individual patient’s response.

A decision often compares an aggregate prediction with a threshold \(c\). When \(\theta_H\neq\theta_L\), the prevalence at which the prediction equals the threshold is

q^*=\frac{c-\theta_L}{\theta_H-\theta_L}

This is a relevant prevalence tipping point only if \(q^*\) lies between 0 and 1. If it lies outside that range, the aggregate does not cross the threshold for any possible two-subgroup composition under the fixed subgroup predictions. If \(\theta_H=\theta_L\), prevalence cannot move the aggregate across a threshold: the prediction is either always on one side or exactly at the threshold.

Worked Example: A Prevalence Shift Changes the Population Benefit

Hypothetical data. Consider a treatment’s predicted absolute reduction in a clinical event over one year. In subgroup H, the reduction is 12 percentage points; in subgroup L, it is 4 percentage points. A development decision requires a population reduction of at least 6 percentage points. Assume the two subgroups are exhaustive and that the subgroup predictions share the same event definition and one-year horizon.

If H makes up 30% of the target population, the aggregate absolute risk reduction is

$$ 0.30(0.12)+0.70(0.04) =0.036+0.028 =0.064 $$

The prediction is a 6.4 percentage-point reduction, which exceeds the 6-point decision threshold. If the prevalence of H is instead assumed to be 20%, the calculation becomes

$$ 0.20(0.12)+0.80(0.04) =0.024+0.032 =0.056 $$

That is a 5.6 percentage-point reduction, below the threshold. The subgroup predictions did not change; the conclusion changed because the assumed composition changed.

The threshold prevalence is

q^*=\frac{0.06-0.04}{0.12-0.04} =\frac{0.02}{0.08} =0.25

Under these fixed predictions, the population benefit meets or exceeds the threshold when H prevalence is at least 25%. This is conditional on the stated assumptions; it does not show that the target population’s true H prevalence is 25% or greater.

Changing Subgroup Response Assumptions

Composition sensitivity is only one part of the problem. If subgroup-specific treatment responses are themselves uncertain, alternative response assumptions can also move the aggregate. It is helpful to keep the two sources of change separate: first hold prevalence fixed and vary subgroup predictions; then hold predictions fixed and vary prevalence. Combined scenarios can be considered afterward, with each changed input stated.

Worked Example: Alternative Response Assumptions Cross a Threshold

Hypothetical data. Assume subgroup H comprises 35% of the target population, and the decision threshold is a population absolute benefit of 6 percentage points over the same follow-up period. Two plausible response assumptions are being compared. Under assumption A, the predicted benefit is 14 percentage points in H and 4 points in L. Under assumption B, it is 8 points in H and 4 points in L. The subgroup definitions and prevalence are held constant.

Under assumption A, the population prediction is

$$ 0.35(0.14)+0.65(0.04) =0.049+0.026 =0.075 $$

This is a 7.5 percentage-point population benefit, above the 6-point threshold. Under assumption B, it is

$$ 0.35(0.08)+0.65(0.04) =0.028+0.026 =0.054 $$

This is a 5.4 percentage-point benefit, below the threshold. Thus, with prevalence fixed at 35%, the decision differs under the two specified assumptions about response in H. Reporting only one aggregate would conceal the assumption that drives the conclusion. The comparison does not establish which response assumption is better supported; that requires evaluating the evidence and model behind each prediction.

Subgroup response assumptions may be correlated with prevalence assumptions or with one another. For example, a scenario that raises the expected prevalence of a biomarker-defined subgroup may also use a different estimate of treatment effect in that subgroup. Such combinations should be specified as coherent scenarios rather than assembled by mixing incompatible marginal extremes. When uncertainty is represented probabilistically, dependence among inputs should be preserved, as in the earlier material on preserving dependence among uncertain inputs.

When the Decision Uses Net Benefit

Some decisions combine benefits and harms or value several outcomes on a common utility scale. In that case, the same aggregation principle applies to subgroup-specific incremental net benefit, provided that each subgroup value uses the same comparator and units. Net benefit is not necessarily a risk difference; its units should be stated, such as utility units per patient. A zero threshold, for example, may represent indifference between treatment and comparator on that specified scale.

Worked Example: Composition Changes an Incremental Net-Benefit Decision

Hypothetical data. Suppose treatment has incremental net benefit of \(0.08\) utility units per patient in H and \(-0.03\) utility units per patient in L. Assume H and L exhaust the target population, and choose treatment if population incremental net benefit is positive. If H prevalence is 20%, then

$$ 0.20(0.08)+0.80(-0.03) =0.016-0.024 =-0.008 $$

The population incremental net benefit is \(-0.008\) utility units per patient, so the rule favors the comparator. If H prevalence is 40%, then

$$ 0.40(0.08)+0.60(-0.03) =0.032-0.018 =0.014 $$

The population incremental net benefit is now \(0.014\) utility units per patient, so the rule favors treatment. Solving for the prevalence at which net benefit is zero gives

q^*=\frac{0-(-0.03)}{0.08-(-0.03)} =\frac{0.03}{0.11} \approx 0.273

With these fixed subgroup values, treatment is favored when H prevalence exceeds about 27.3%. The conclusion is conditional on the utility scale, subgroup predictions, and decision rule; it is not a universal clinical recommendation.

Diagnostics, Comparisons, and Limitations

A useful sensitivity analysis should make it possible to see not only the aggregate result, but also why it changes. For a small number of subgroups, a table of scenarios often makes the assumptions and decision status especially clear.

Approach	What it calculates	When it is useful
Pooled or common-response prediction	Uses one prediction for everyone, or otherwise does not represent subgroup variation	A simple benchmark when a common response is defensible; it can obscure meaningful subgroup differences.
Subgroup-weighted aggregate	Combines subgroup predictions using specified population prevalences	Directly answers what the aggregate predicts under a stated composition and response assumption.
Separate subgroup decisions	Applies a decision rule within each subgroup	Can inform subgroup-specific action, but does not by itself answer the decision for a mixed population or account for implementation constraints.

Several checks help identify fragile or invalid results:

Verify the weights. Confirm that subgroup prevalences sum to 1 and correspond to the intended population. If the weights do not sum to 1, normalize them only when that operation is scientifically justified; otherwise identify the missing or overlapping categories.
Check the scale and direction. A larger number may mean more benefit for one endpoint and more harm for another. State whether the outcome is a risk, risk reduction, response probability, or utility, and use a consistent direction before interpreting a threshold crossing.
Inspect subgroup contrasts. If the aggregate moves sharply with prevalence, the subgroup predictions are far apart. If it barely moves, composition may matter little over the examined range, even if subgroup estimates differ.
Separate scenario ranges from uncertainty intervals. The minimum and maximum across chosen assumptions are not automatically a confidence interval, prediction interval, or probability statement. They summarize only the scenarios evaluated.
Check subgroup support and definition. Extremely small or poorly characterized groups may have unstable response estimates. A definition based on a threshold can also make prevalence sensitive to measurement and classification choices.

A common mistake is to say, “The treatment works in H, so the population benefits,” without considering how common H is or what happens in L. The correction is to calculate the weighted aggregate for the target population and compare that result with the decision criterion. A second mistake is to describe a scenario comparison as the probability that a decision is correct. A set of alternative assumptions has no probability interpretation unless probabilities have been assigned and justified. Finally, a weighted average is not a substitute for examining uncertainty in the subgroup predictions; precise-looking arithmetic cannot make uncertain inputs precise.

Key takeaway: A population-level conclusion is conditional on both subgroup response predictions and subgroup prevalence. State the target population, weights, subgroup outcomes, and decision threshold; then show whether plausible alternatives change the aggregate or the decision.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Two exhaustive subgroups have predicted benefits of 10 and 2 percentage points. The first subgroup has prevalence 40%. What is the aggregate predicted benefit?
Using the subgroup predictions in Question 1, at what prevalence of the first subgroup does the aggregate benefit reach 6 percentage points?
If the subgroup predictions are equal, can changing subgroup prevalence change the aggregate prediction? Explain.
Why is the range of results from three selected prevalence scenarios not automatically a 95% uncertainty interval?

Solutions. 1. The aggregate is \(0.40(0.10)+0.60(0.02)=0.040+0.012=0.052\), or a 5.2 percentage-point benefit. 2. Let \(q\) be the first subgroup’s prevalence. Solve \(0.10q+0.02(1-q)=0.06\), giving \(0.02+0.08q=0.06\), so \(q=0.50\), or 50%. 3. No. If both subgroup predictions equal \(\theta\), then \(q\theta+(1-q)\theta=\theta\) for every \(q\). 4. The three scenarios are selected assumptions, not necessarily sampled from a probability distribution. Without a defined uncertainty procedure and justified probability model, their range has no 95% coverage interpretation.