Setting Planning Objectives and Constraints

From Planning Objectives to Acceptance Criteria

A planning objective such as “achieve high power without exceeding the available budget” is not yet a usable rule for choosing a group sequential design. “High” and “available” need numerical definitions, and the design’s performance must be assessed under specified assumptions. Otherwise, different candidate designs can appear preferable simply because they were judged against different targets.

The practical task is to translate the trial’s objectives into criteria that can be checked consistently. Some criteria are hard constraints: a design that fails one is not acceptable, even if it performs well on other measures. Other features are preferences used to rank designs that meet all hard constraints. For example, a smaller expected sample size may be desirable, but it does not compensate for failing a required power threshold.

Definition: A planning acceptance criterion is a pre-specified, measurable rule used to decide whether a candidate design satisfies an objective or constraint. A candidate is feasible only if it meets every criterion designated as a hard constraint, evaluated under the stated design assumptions.

Specify What Is Being Evaluated

An acceptance rule is meaningful only if each candidate is assessed on a comparable basis. Before applying thresholds, specify the trial objective and the assumptions that define the operating characteristics. At a minimum, state the null and alternative hypotheses, the direction of the test, the trial-level significance level $\alpha$, the target effect at which power is required, and the assumptions needed to calculate power. State how sample size is counted: for example, whether the maximum is the number randomized or the number expected to provide an evaluable primary outcome.

For a group sequential design, the type I error criterion concerns the probability of rejecting $H_0$ at any planned analysis when $H_0$ is true. It is a trial-level probability, not a separate allowance of $\alpha$ at every look. Power is the probability of rejecting $H_0$ under a specified alternative, accounting for the design’s stopping rule. The target effect must be stated in the units of the outcome or estimand; “90% power” without an effect scenario is incomplete.

The candidate designs should also share the same endpoint, estimand, analysis method, sidedness, and relevant nuisance assumptions unless the purpose of the comparison is explicitly to examine differences in those assumptions. If one candidate’s power is calculated for a larger treatment effect, its result cannot fairly be compared with another candidate’s power at a smaller effect. Likewise, a maximum sample size of 500 evaluable participants is not directly comparable with a maximum of 500 randomized participants if follow-up loss is expected.

Definition: For a specified design and assumptions, the design’s operating characteristics include its trial-level type I error, its power at a stated alternative, and its sample-size behavior. Acceptance criteria compare these quantities with pre-specified thresholds; they do not replace the assumptions under which those quantities were obtained.

Write Criteria as Explicit Rules

A concise acceptance specification gives each criterion a quantity, threshold, direction of comparison, and status as either a hard constraint or a preference. For example, “power should be about 90%” leaves room for interpretation; “power must be at least 0.90 at a treatment effect of 5 mmHg” does not. A threshold should be written with an inequality so that equality is handled unambiguously.

\begin{aligned} \text{Type I error:} \quad & \alpha_{\mathrm{actual}} \leq \alpha_{\mathrm{planned}}, \\ \text{Power at target effect } \delta^*: \quad & \operatorname{Power}(\delta^*) \geq \pi_{\min}, \\ \text{Maximum sample size:} \quad & N_{\max} \leq N_{\mathrm{limit}}. \end{aligned}

Here $\alpha_{\mathrm{actual}}$ is the probability of rejecting $H_0$ under the null for the whole group sequential design, $\delta^*$ is the specified target effect, $\pi_{\min}$ is the minimum acceptable power, and $N_{\mathrm{limit}}$ is the sample-size cap. The symbols are placeholders for thresholds chosen in context; the inequality directions are part of the criteria, not a matter of presentation.

Resource limits can be expressed in the same way. A monetary budget might be a maximum total cost calculated using a stated cost model. A recruitment limit might be a maximum number of participants that can be enrolled within the available period. Where several resource constraints apply, list each one separately rather than hiding them in a vague overall feasibility judgment. If a limit is uncertain, state a defensible planning value or a range and explain how it will be used.

Worked Example: Checking Type I Error, Power, and Maximum Sample Size

Hypothetical planning scenario. A two-sided group sequential trial must control trial-level type I error at 0.05 or less, achieve at least 0.90 power to detect a mean difference of 5 mmHg, and require no more than 500 randomized participants. Assume both candidate designs target the same estimand and use the same outcome-variance assumption. Their operating characteristics have already been calculated using the same analysis assumptions.

Candidate	Actual type I error	Power at 5 mmHg	Maximum randomized sample size
A	0.049	0.902	496
B	0.050	0.895	480

For Candidate A, the checks are $0.049 \leq 0.05$, $0.902 \geq 0.90$, and $496 \leq 500$. It meets all three hard constraints and is feasible.

For Candidate B, the type I error and sample-size checks pass: $0.050 \leq 0.05$ and $480 \leq 500$. Its power check fails because $0.895 < 0.90$. It is therefore not feasible under this specification, even though it has a smaller maximum sample size. The conclusion is about the stated acceptance rules, not a general judgment that one design is better in every respect.

Why Feasibility Is a Conjunction

The logic of hard constraints can be written as an intersection of requirements. Let $C_1,\ldots,C_m$ denote the conditions that a candidate must satisfy. The feasible set consists of candidates for which every condition is true:

\mathcal{F} = \{d : C_1(d) \text{ and } C_2(d) \text{ and } \cdots \text{ and } C_m(d)\}.

This is a conjunction, not an average score. If the type I error limit, minimum power, and maximum sample size are all mandatory, a candidate failing any one is outside $\mathcal{F}$. Ranking or optimization should occur only among feasible candidates, unless the planning team explicitly revises a requirement.

This rule prevents an attractive feature from concealing a material failure. For example, reducing the maximum sample size does not repair power below the required minimum. Nor does exceeding the type I error limit become acceptable because power is high. A weighted score can be useful for ranking preferences, but using one to combine hard constraints permits trade-offs that the stated requirements may not allow.

Worked Example: Add a Resource Limit

Hypothetical planning scenario. A sponsor’s recruitment capacity is 420 randomized participants. The design team also estimates a total cost of $1,200 per randomized participant plus $80,000 in fixed start-up and oversight costs. The total budget is $600,000. A candidate design has a maximum sample size of 410 randomized participants, actual type I error 0.024 under a one-sided 0.025 criterion, and power 0.91 at the specified target effect, exceeding a required minimum of 0.90.

First check the recruitment cap: $410 \leq 420$, so the candidate meets that constraint. Its estimated total cost is

\$80{,}000 + \$1{,}200(410) = \$80{,}000 + \$492{,}000 = \$572{,}000.

Since $\$572{,}000 \leq \$600{,}000$, the budget criterion also passes. The type I error and power criteria pass because $0.024 \leq 0.025$ and $0.91 \geq 0.90$. Subject to the stated cost model and common analysis assumptions, the candidate satisfies all four hard constraints.

This conclusion depends on counting the maximum as randomized participants in both the capacity limit and the cost calculation. If the cost model instead applied per evaluable participant, or if the capacity were stated in evaluable participants, the calculation would need to be aligned to that definition before judging feasibility.

Separate Requirements From Preferences

Once the feasible candidates are identified, preferences can guide the choice among them. Common preferences include lower expected sample size, shorter expected duration, simpler implementation, or more room below a resource cap. These are not automatically hard constraints. Marking a preference as mandatory changes the feasible set and should be an intentional decision.

Worked Example: Rank Only After Applying the Constraints

Hypothetical planning scenario. Two candidates both meet a maximum type I error of 0.05, a minimum power of 0.90 at the target effect, and a maximum sample size of 500. Candidate C has expected sample size 345 and maximum sample size 490. Candidate D has expected sample size 320 and maximum sample size 500. The planning team has designated expected sample size as a preference and all three thresholds as hard constraints.

Both designs are feasible: each passes the type I error and power criteria, and both satisfy $N_{\max} \leq 500$. If the chosen preference is to minimize expected sample size, D ranks ahead of C because $320 < 345$. Its maximum sample size sits exactly at the cap, but equality passes because the stated rule is $N_{\max} \leq 500$.

If the team instead required a safety margin of at least 10 participants below the cap, that would be a new hard constraint, $N_{\max} \leq 490$. Under that revised rule, C would pass and D would fail. The ranking therefore depends on the criteria declared in advance, not on an unstated interpretation of “room below the cap.”

Assumptions, Diagnostics, and Limitations

A pass-or-fail result is only as credible as the inputs behind it. Before accepting a comparison, check that the target effect is clinically meaningful and expressed in the correct units; the significance level matches the planned sidedness; the reported error rate is trial-level; and the power values refer to the same target effect and assumptions. Confirm that every sample-size limit uses the same definition and that each resource calculation uses a stated cost or capacity model.

Warning signs include criteria with no numeric threshold, a power requirement with no target effect, inconsistent assumptions across candidates, and a “maximum sample size” that leaves unclear whether it refers to randomized or evaluable participants. Also investigate results that sit very close to a cutoff: rounding a reported power of 0.8996 to 0.900 can make a failing value look like a passing one. Compare using adequate precision, and state how rounding is handled for the decision.

Acceptance criteria do not establish that the assumptions are correct, guarantee recruitment will meet the plan, or show that the selected effect is clinically worthwhile. They provide a transparent rule for comparing designs conditional on their assumptions. A design may meet formal numerical criteria while remaining operationally difficult or dependent on uncertain assumptions; those issues need to be made visible rather than treated as resolved by the pass-or-fail label.

A common mistake is to say, “This candidate is best because it has the smallest maximum sample size,” without first checking type I error and power. The correction is to apply every hard constraint first, exclude any candidate that fails one, and then compare feasible candidates using the preferences that were stated. Another mistake is to treat a value exactly at a limit as a failure despite having written an inclusive inequality. Use the specified rule consistently: $N_{\max} \leq N_{\mathrm{limit}}$ includes equality, whereas $N_{\max} < N_{\mathrm{limit}}$ does not.

Key takeaway: Make each objective measurable, define its assumptions and units, state whether it is a hard constraint or a preference, and write the decision rule explicitly. Feasibility requires passing every hard constraint; preferences rank the candidates that remain.

Check Your Understanding

Answer each question using the stated criteria. Solutions follow the questions.

A design must have actual type I error at most 0.025, power at least 0.80 at a target effect of 3 units, and maximum sample size at most 300. A candidate has values 0.024, 0.80, and 301, respectively. Is it feasible? Explain.
Why is “at least 90% power” incomplete unless the target effect and assumptions are specified?
Two candidates meet all hard constraints. Their expected sample sizes are 280 and 260. If expected sample size is a preference to minimize, which candidate ranks higher, and does that preference make the other candidate infeasible?
A budget is $500,000, with fixed costs of $50,000 and variable costs of $1,000 per randomized participant. What is the maximum number of randomized participants affordable under this model?

Solutions. (1) No. The type I error passes because $0.024 \leq 0.025$, and power passes because $0.80 \geq 0.80$, but the sample-size criterion fails because $301 > 300$. (2) Power depends on the alternative effect and the assumptions used to calculate the probability of rejecting $H_0$; without them, the 90% claim does not specify a comparable performance target. (3) The candidate with expected sample size 260 ranks higher. The other remains feasible because expected sample size is a preference, not a hard constraint. (4) The variable-cost allowance is $\$500{,}000-\$50{,}000=\$450{,}000$, so the affordable maximum is $\$450{,}000/\$1{,}000=450$ randomized participants.