Sensitivity Analysis for Uncertain Sample-Size Inputs

Why Sample-Size Sensitivity Matters

A sample-size calculation is conditional on its inputs. A planned treatment effect, endpoint variance, within-participant correlation, dropout rate, and analysis model may each be uncertain when a study is designed. A target calculated from one set of assumptions can therefore give a misleading impression of precision. Sensitivity analysis makes the dependence visible: it recalculates power or sample size under plausible alternative assumptions and identifies which assumptions have the greatest practical influence.

The aim is not to find a single “correct” sample size by varying numbers mechanically. Each scenario should correspond to a scientifically credible assumption, and the endpoint contrast and analysis model should remain aligned with the estimand. The earlier material on Fisher information provides a general route from model information to precision; here the emphasis is on how uncertainty in planning inputs changes that precision and the resulting enrollment target.

A Reference Calculation and Its Assumptions

Consider a balanced, two-arm comparison of a continuous endpoint. Let \(m\) be the number of evaluable participants per arm, let \(\delta\) be the absolute treatment contrast in endpoint units, and let \(v\) be the endpoint variance in each arm. Assume independent participants, a common variance in the two arms, equal allocation, and a test of a nonzero contrast. For this reference calculation, use a two-sided significance level \(\alpha=0.05\) and target power \(1-\beta=0.90\).

With a normal approximation, the standard error of the estimated treatment contrast is \(\sqrt{2v/m}\). The approximate sample-size formula follows by requiring the standardized contrast to equal the sum of the critical normal quantile and the target-power quantile. Here \(z_{1-\alpha/2}=1.960\) and \(z_{1-\beta}=1.282\), so their sum is \(c=3.242\).

This is an approximation for a normal test with the stated variance and contrast assumptions. It is not an exact finite-sample calculation for a \(t\)-test with variance estimated from the data; when sample sizes are small or an exact analysis is required, power should be computed for the planned test or model. The formula also presumes the chosen \(\delta\) is the effect relevant to the estimand, not simply the largest effect considered plausible.

How Input Changes Propagate to Sample Size

The formula gives useful directional checks before any arithmetic. Required evaluable sample size is proportional to variance and to \(c^2\), and inversely proportional to the squared treatment effect. Consequently, a modest reduction in the assumed effect can have a substantial impact, while a proportional increase in variance produces the same proportional increase in the unrounded sample size. These relationships help identify which assumptions deserve careful justification.

Correlation and Endpoint Variance

A correlation assumption can affect power through the variance of the endpoint contrast. For example, suppose the planned endpoint is the average of two measurements for each participant, each with marginal variance \(\sigma^2\), and the within-participant correlation between the measurements is \(\rho\). The variance of that average is

This expression follows from the variance of a sum: \(\operatorname{Var}(Y_1+Y_2)=\operatorname{Var}(Y_1)+\operatorname{Var}(Y_2)+2\operatorname{Cov}(Y_1,Y_2)\), with \(\operatorname{Cov}(Y_1,Y_2)=\rho\sigma^2\). It applies to this particular average-of-two endpoint. A different endpoint definition, such as a change score, has a different variance formula; the relevant covariance structure must match the planned endpoint and model.

Model Structure Is a Planning Input

Sensitivity analysis should include plausible analysis structures when they imply different precision for the same estimand. For instance, an endpoint analyzed as an unadjusted change score may have a different variance from the treatment contrast estimated with baseline adjustment. The comparison is meaningful only if both analyses target the same contrast and their variance assumptions are justified. A more elaborate model is not automatically more precise: precision depends on the covariance structure, design, and assumptions used to estimate the contrast.

The following hypothetical comparison holds the contrast and test fixed and treats the variance values as planning inputs. It illustrates the impact of a 20% reduction in contrast variance under a baseline-adjusted model; it does not establish that this reduction will occur in an actual study.

A Practical Sensitivity Workflow

A useful analysis starts from a reference case and varies one input at a time to diagnose its effect. It then combines important uncertainties into coherent scenarios, such as a conservative effect paired with a higher variance and a higher dropout rate. One-at-a-time changes help explain what drives the calculation; combined scenarios show whether a plausible joint set of assumptions would make the plan unacceptable. The scenario values, rounding rule, and whether sample size refers to evaluable participants or enrollment should be stated explicitly.

Diagnostics, Alternatives, and Limitations

Several checks help reveal a fragile or internally inconsistent calculation. First, verify directions: under the reference model, larger variance or smaller absolute treatment effect must not reduce the required sample size. Second, check units: \(v\) has squared endpoint units and \(\delta^2\) has the same units, leaving \(m\) dimensionless. Third, verify the relationship between evaluable and enrollment targets; enrollment must be at least as large as the evaluable target. Finally, compare outputs with a validated power implementation for the actual planned model when the normal approximation may be inadequate.

A common mistake is to treat an assumed dropout percentage as if it only changed enrollment, even when dropout may be related to outcome, treatment, or time. The simple inflation \(m/(1-q)\) is justified only when the assumed retention fraction is a suitable approximation and losses do not otherwise alter the analysis information or estimand. A second mistake is to select the sample size from the most favorable scenario. A plan should be evaluated against the range of plausible assumptions and the consequences of missing its target.

Scenario analysis is transparent and easy to communicate, but it does not express the probability of each input value. A probabilistic sensitivity analysis can assign distributions to uncertain inputs and summarize the resulting distribution of power or required sample size; those distributions require defensible justification and do not eliminate structural uncertainty. A conservative single scenario is simpler, but may be unnecessarily large or fail to reflect which combinations of assumptions are plausible. For complex longitudinal models, simulation can estimate operating characteristics under the planned analysis, including features that a closed-form approximation cannot represent. Each approach has limitations: scenario grids depend on chosen values, probabilistic analyses depend on input distributions, and simulations depend on the data-generating and analysis models implemented.