What Reverse Planning Answers
A power calculation usually starts with a proposed effect and asks how likely a test is to detect it. Reverse planning starts instead with a sample size, significance level, and desired power, then asks how large an effect must be for the test to reach that power. This smallest effect is useful when planning a study or judging what a fixed sample can reasonably detect.
A minimum detectable effect is not a prediction of the effect that will occur, and it is not a boundary below which an effect is impossible to detect. It is tied to a specific test, model, significance level, target power, and sample size. Changing any of those inputs can change the result.
The examples below use Z tests. Their assumptions are that observations are independent, the relevant population standard deviation or standard error is known, and the sampling distribution used for the test is normal under the null and alternative. This is exact for normally distributed observations with known variance, and can be an approximation when a normal approximation is justified. We use \(\Phi\) for the standard normal cumulative distribution function and \(z_q\) for its \(q\)-quantile, so \(\Phi(z_q)=q\).
Reverse Planning for a One-Sided Z Test
Suppose a one-sample upper-tailed test examines \(H_0:\mu=\mu_0\) against \(H_1:\mu>\mu_0\). Let \(\sigma\) be known and let \(n\) independent observations have sample mean \(\bar X\). At significance level \(\alpha\), the test rejects when \(\bar X\) exceeds \(\mu_0+z_{1-\alpha}\sigma/\sqrt n\). Write the positive effect as \(\Delta=\mu-\mu_0\).
To obtain power at least \(1-\beta\), the argument of \(\Phi\) must be at least \(z_{1-\beta}\), because \(\Phi\) is increasing. Solving that inequality for \(\Delta\) gives the MDE.
This result applies to an upper-tailed test when the effect of interest is positive. For a lower-tailed test, the same calculation gives the minimum detectable magnitude of a negative effect, with the direction reversed. The formula does not directly apply if the variance is unknown and the test uses a \(t\) statistic; that setting requires the corresponding \(t\)-based power calculation.
Worked Example: Detectable Increase in a Processing Time
A hypothetical system is evaluated using \(n=81\) independent processing times. Assume the times are normally distributed with known standard deviation \(\sigma=9\) seconds. A one-sample upper-tailed Z test compares the mean with \(\mu_0=40\) seconds at \(\alpha=0.05\). Find the smallest increase giving target power \(0.90\), and report the corresponding mean. There is no interest rate or mortality assumption; this is a sampling-distribution model with parameters \(\mu\) and known \(\sigma=9\).
Method. Apply the one-sided MDE formula. The standard error is \(\sigma/\sqrt n=9/\sqrt{81}=9/9=1\) second. The required normal quantiles are \(z_{0.95}=1.6449\) and \(z_{0.90}=1.2816\).
Substitute the values: \[ \Delta_{\min}=(1.6449+1.2816)(1)=2.9265\text{ seconds}. \] Thus the mean must be at least \(40+2.9265=42.9265\) seconds to attain the target power under the model. The MDE is about \(2.93\) seconds. The calculation avoids the common mistake of using \(z_{0.90}\) alone: the critical value set by \(\alpha\) must also be overcome. The units check is seconds, since the quantiles are dimensionless and the standard error is in seconds.
Two-Sided Tests
For a two-sided test of \(H_0:\mu=\mu_0\) against \(H_1:\mu\ne\mu_0\), the rejection region is outside the two critical values \(\mu_0\pm z_{1-\alpha/2}\sigma/\sqrt n\). If the true effect is positive, define the standardized effect \(\lambda=\Delta\sqrt n/\sigma\). The power is the probability of falling above the upper critical value plus the probability of falling below the lower critical value:
Unlike the one-sided case, this equation generally cannot be inverted into a simple exact expression for \(\Delta\), because the effect contributes to both tail probabilities. A common planning approximation is \(\Delta_{\min}\approx(z_{1-\alpha/2}+z_{1-\beta})\sigma/\sqrt n\). It treats the probability in the tail opposite the effect as negligible. For a specified two-sided test, an exact MDE can instead be found by solving the full power equation numerically. The approximation should be identified as such, particularly when the target power or significance level makes the opposite-tail probability non-negligible.
Worked Example: Detectable Shift in a Two-Sided Calibration Check
A hypothetical calibration study takes \(n=49\) independent readings. Assume the readings are normally distributed with known standard deviation \(\sigma=7\) units. A two-sided Z test checks \(H_0:\mu=\mu_0\), using \(\alpha=0.05\) and target power \(0.80\). Find the positive MDE and explain the approximation. No interest rate or mortality assumption is relevant; the sampling model has known \(\sigma=7\) and a mean that may differ from \(\mu_0\).
Method. The standard error is \(7/\sqrt{49}=1\) unit, and the two-sided critical quantile is \(z_{0.975}=1.9600\). The target-power quantile is \(z_{0.80}=0.8416\). The usual approximation gives \[ \Delta_{\min}\approx(1.9600+0.8416)(1)=2.8016\text{ units}. \]
To check the approximation, set \(\lambda=2.8016\) in the exact power formula. The upper-tail contribution is approximately \(\Phi(2.8016-1.9600)=\Phi(0.8416)\), or about \(0.8000\). The opposite-tail contribution is \(\Phi(-2.8016-1.9600)=\Phi(-4.7616)\), about \(0.000001\). Thus the exact power at this rounded value is just over \(0.8000\); solving the full equation gives \(\lambda\) approximately \(2.80158\), which also rounds to an MDE of \(2.8016\) units. The approximation is appropriate here because the omitted tail is tiny. A common error is to use \(z_{1-\alpha}\) rather than \(z_{1-\alpha/2}\), which would use the wrong critical value for a two-sided test.
Two Independent Samples
Reverse planning also works for a difference in means. Suppose two independent samples have sizes \(n_1\) and \(n_2\), and both populations have known standard deviation \(\sigma\). Under normal sampling, or a suitable normal approximation, the difference in sample means has standard error \(\sigma\sqrt{1/n_1+1/n_2}\). For an upper-tailed test of a positive difference \(\mu_1-\mu_2\), replace the one-sample standard error in the MDE formula with this two-sample standard error. The result relies on independent samples and known variances; it is not the pooled \(t\)-test formula for unknown variance.
Worked Example: Detectable Difference Between Two Service Designs
A hypothetical study compares mean completion times under two independent service designs. Each design's completion times are normally distributed with the same known standard deviation, \(\sigma=8\) minutes. The planned sample sizes are \(n_1=64\) and \(n_2=100\). An upper-tailed Z test targets a positive difference \(\mu_1-\mu_2\), with \(\alpha=0.025\) and power \(0.90\). Find the MDE in minutes. No interest rate or mortality assumption applies; the normal model has common known standard deviation 8 minutes and independent observations between and within groups.
Method. First calculate the standard error: \[ \sqrt{\frac{1}{64}+\frac{1}{100}} =\sqrt{0.015625+0.010000} =\sqrt{0.025625} \approx0.1600781. \] Multiplying by \(\sigma=8\) gives \[ SE=8(0.1600781)\approx1.2806248\text{ minutes}. \]
For an upper-tailed test, \(z_{1-\alpha}=z_{0.975}=1.9600\); for target power \(0.90\), \(z_{1-\beta}=z_{0.90}=1.2816\). Therefore, \[ \Delta_{\min}=(1.9600+1.2816)(1.2806248) =(3.2416)(1.2806248) \approx4.1513\text{ minutes}. \] A difference of about \(4.15\) minutes is the smallest positive mean difference targeted for 90% power under these assumptions. The common mistake is to use \(1/\sqrt{n_1+n_2}\) for the standard error; independent sample means instead contribute the two separate variance terms \(1/n_1\) and \(1/n_2\).
Interpreting and Using an MDE
An MDE is most informative when stated in the original measurement units and alongside its planning assumptions. For example, “about 4.15 minutes” is interpretable; a standardized value alone is not. When the standard error is expressed in the outcome's units, multiplying it by dimensionless normal quantiles returns an effect in those same units.
The formulas also clarify the role of the design choices. A more stringent significance level increases the critical quantile and generally increases the MDE. A higher target power increases \(z_{1-\beta}\), also increasing the MDE. A larger standard deviation increases the MDE, while increasing sample size lowers the standard error and reduces the MDE. These statements compare calculations with other inputs held fixed.
Decide whether the test is one- or two-sided and state the effect in the original units.
Record the known standard deviation, sample size or sample sizes, and any independence or normality assumptions.
Use the critical quantile that matches the test and the quantile for the desired power.
Compute the standard error, solve for the MDE, and report its direction and original measurement units.
A frequent interpretation trap is to treat “minimum detectable” as “minimum important.” Statistical detectability is defined by the test's power and assumptions; practical importance must be judged from the context. An effect smaller than the MDE may still be real or important, but this design has less than the target power to detect it. Conversely, reaching the target power does not guarantee rejection in any particular study.
Check Your Understanding
For each problem, use the stated test and assumptions. Report an effect in the original units where requested.
- A one-sample upper-tailed Z test has known \(\sigma=10\), \(n=64\), \(\alpha=0.05\), and target power \(0.80\). Find the positive MDE. (Answer: approximately \(3.1081\) units.)
- A two-sided one-sample Z test has known \(\sigma=15\), \(n=100\), \(\alpha=0.01\), and target power \(0.90\). Use the usual two-sided approximation to find the positive MDE. (Answer: approximately \(5.7861\) units.)
- Two independent samples have \(n_1=36\) and \(n_2=64\), with common known \(\sigma=6\). For an upper-tailed test at \(\alpha=0.05\), find the MDE for target power \(0.80\). (Answer: approximately \(3.1081\) units.)
- A lower-tailed one-sample Z test has known \(\sigma=4\), \(n=100\), \(\alpha=0.05\), and target power \(0.95\). Find the minimum detectable magnitude and state the direction of the effect. (Answer: approximately \(1.3159\) units below the null mean.)
- A one-sample upper-tailed Z test has known \(\sigma=12\), \(n=100\), and \(\alpha=0.05\). If the true mean exceeds the null mean by 3 units, calculate the test's power. (Answer: \(\Phi(0.8551)\approx0.8038\).)