Minimum Detectable Effect and Study Design

What Does a Study Have a Good Chance of Detecting?

Before collecting data, a researcher may want to know not only how many observations a study needs, but also what size of effect that study can reliably detect. A design can have a high probability of detecting a large effect and a much lower probability of detecting a subtle one. The minimum detectable effect (MDE) puts this design property on the scale of the scientific question: it is the smallest effect, in a specified direction or measured by a specified magnitude, for which the planned test reaches a chosen power.

An MDE is not a promise that every effect at or above the threshold will produce a statistically significant result. It describes a probability under a model: if the true effect equals the MDE and the assumptions hold, the test has the chosen probability of rejecting the null hypothesis. The threshold depends on the test, significance level, target power, sample size, and variability. It should therefore be reported with those design choices, not as a universal property of the study.

Definition: For a specified hypothesis test, significance level \(\alpha\), and target power \(1-\beta\), the minimum detectable effect is the smallest effect size, in the stated direction or magnitude, for which the probability of rejecting \(H_0\) is at least \(1-\beta\). The MDE is a planning threshold; it is not an estimate of the effect that will occur.

A Model for Calculating an MDE

We will use a one-sample \(Z\) test for a population mean to make the calculation precise. Suppose independent observations come from a population with mean \(\mu\) and known standard deviation \(\sigma\). The null hypothesis is \(H_0:\mu=\mu_0\), and an effect is the difference \(\delta=\mu-\mu_0\), measured in the original units of the outcome. Under the normal model, or an adequate normal approximation for the sample mean, the sample mean has standard error \(\sigma/\sqrt{n}\). The test uses this standard error and a rejection rule chosen to have significance level \(\alpha\).

For a two-sided test, the null is rejected when the test statistic falls below \(-z_{1-\alpha/2}\) or above \(z_{1-\alpha/2}\), where \(z_q\) is the \(q\)-quantile of the standard normal distribution. If the true mean differs from \(\mu_0\) by \(\delta\), the standardized statistic has a normal distribution with standard deviation 1 and mean \(\delta/(\sigma/\sqrt{n})\). Thus, the power at that effect is the probability that this shifted distribution falls in either rejection tail.

\operatorname{Power}(\delta) = \Phi\left(-z_{1-\alpha/2}-\frac{\delta}{\sigma/\sqrt{n}}\right) + 1-\Phi\left(z_{1-\alpha/2}-\frac{\delta}{\sigma/\sqrt{n}}\right)

Here \(\Phi\) is the standard normal cumulative distribution function. For an effect in the positive direction, the lower-tail term is often extremely small near the MDE. Setting the upper-tail power to \(1-\beta\) gives the familiar planning approximation below. Because the two-sided test also has a lower rejection tail, this formula is an approximation rather than an exact inversion of the two-tail power equation.

\operatorname{MDE} \approx \left(z_{1-\alpha/2}+z_{1-\beta}\right)\frac{\sigma}{\sqrt{n}}

The formula shows that the MDE is expressed in the outcome's units. Dividing it by \(\sigma\) gives the corresponding standardized effect. For a one-sided test aimed at detecting an increase, the rejection region is above \(z_{1-\alpha}\). In that case the power equation gives the following MDE for a positive effect:

\operatorname{MDE} = \left(z_{1-\alpha}+z_{1-\beta}\right)\frac{\sigma}{\sqrt{n}}

For this one-sided normal test, the expression follows directly from its single rejection tail. In either formula, \(n\) is the number of usable independent observations assumed in the model, and \(\sigma\) is treated as known for the \(Z\) test. Other tests and designs can have different power equations, so these formulas should not be transferred to them without checking their assumptions.

Worked Example: MDE for a Two-Sided Mean Test

Suppose a hypothetical study will compare the mean of an outcome with a reference value using a one-sample, two-sided \(Z\) test. The design has \(n=64\) independent observations, known \(\sigma=12\) outcome units, \(\alpha=0.05\), and target power \(0.90\). Find the approximate MDE.

Check the model and test. Assume the outcome is normally distributed with the stated known standard deviation, or that the sample mean is adequately approximated by a normal distribution. The observations are independent, and the two-sided test and its significance level are specified in advance. The standard error is:

SE=\frac{\sigma}{\sqrt{n}}=\frac{12}{\sqrt{64}}=\frac{12}{8}=1.5\text{ units}

For a two-sided test with \(\alpha=0.05\), \(z_{1-\alpha/2}=z_{0.975}\approx1.960\). Target power \(0.90\) means \(\beta=0.10\), so \(z_{1-\beta}=z_{0.90}\approx1.282\). Substituting:

\operatorname{MDE} \approx (1.960+1.282)(1.5) = 3.242(1.5) = 4.863\text{ units}

Using the more precise quantiles, the approximation is \( (1.959964+1.281552)(1.5)\approx4.8623\) units. At that effect, the standardized shift is about \(3.2415\); the exact two-sided power is approximately \(\Phi(-5.2015)+\Phi(1.2816)\approx0.9000\) (about \(0.9000001\)). The planned test therefore has approximately 90% power for a true mean difference of about 4.862 units. This is a planning statement, not a claim that the study will necessarily detect that effect.

Direction and the Choice of Test Matter

A one-sided MDE is tied to a direction selected before the data are observed. It can be smaller than the two-sided MDE at the same \(\alpha\), power, sample size, and variability because the one-sided test places its rejection probability in one tail. This does not make it a general shortcut: a one-sided test does not count an effect in the opposite direction as evidence for the stated alternative. The direction must be justified by the research question and chosen in advance.

Worked Example: MDE for a One-Sided Mean Test

A hypothetical laboratory study is designed to test whether a new procedure increases the mean measurement relative to a reference value. The researchers plan a one-sided \(Z\) test with \(\alpha=0.05\), target power \(0.80\), \(n=100\) independent observations, and known \(\sigma=10\) measurement units. Find the MDE for an increase.

Assume a normal population or an adequate normal approximation for the sample mean, and assume independent observations with the stated known standard deviation. The standard error is \(10/\sqrt{100}=1\) measurement unit. The one-sided critical value is \(z_{0.95}\approx1.645\), and \(z_{0.80}\approx0.8416\). Therefore:

\operatorname{MDE} = (1.645+0.8416)(1) = 2.4866\text{ measurement units}

At a true increase of about 2.487 units, the standardized statistic has mean about 2.4866. The test rejects when the statistic exceeds 1.645, so its power is \(1-\Phi(1.645-2.4866)=1-\Phi(-0.8416)=0.80\), up to rounding. This threshold concerns increases; it does not describe the chance of detecting a decrease with this one-sided test.

Compare the MDE With a Meaningful Effect

A detectable effect is not automatically an important effect. Researchers should identify what difference would matter scientifically, clinically, or operationally, then compare that value with the MDE. If the MDE is larger than effects that would matter, the study may have little chance of detecting an important but modest effect. If the MDE is smaller than the meaningful-effect threshold, the design has a better chance of detecting effects of practical interest, subject to the assumptions used for planning.

This comparison makes design discussion concrete. If an MDE seems too large, possible responses include increasing \(n\), reducing measurement variability through better measurement or a more controlled design, or reconsidering whether the chosen \(\alpha\) and target power are appropriate. Any change must be scientifically and ethically justified. A smaller MDE is not automatically better if achieving it requires an infeasible study or compromises elsewhere.

Worked Example: Is the MDE Small Enough to Matter?

Suppose a hypothetical study plans a two-sided one-sample \(Z\) test of a mean, with \(n=25\), known \(\sigma=8\) outcome units, \(\alpha=0.05\), and target power \(0.80\). Before the study, the research team decides that a difference of 3 units would be scientifically meaningful. Assess the design using the MDE approximation.

Assume independent observations and a normal population, or an adequate normal approximation for the mean. With \(z_{0.975}\approx1.960\), \(z_{0.80}\approx0.8416\), and standard error \(8/\sqrt{25}=1.6\) units:

\operatorname{MDE} \approx (1.960+0.8416)(1.6) = 2.8016(1.6) = 4.483\text{ units}

The approximate MDE, 4.483 units, is larger than the 3-unit meaningful difference. This design therefore does not reach 80% power at the effect the team considers meaningful. To estimate the sample size needed for a 3-unit effect using the same approximation, rearrange the two-sided formula:

n \approx \left(\frac{(z_{1-\alpha/2}+z_{1-\beta})\sigma}{\delta}\right)^2 = \left(\frac{(1.960+0.8416)(8)}{3}\right)^2 \approx55.8

Rounding up gives \(n=56\) usable observations. As a check, the standardized shift at \(n=56\) is \(3\sqrt{56}/8\approx2.806\); the two-sided power is approximately \(\Phi(-1.960-2.806)+1-\Phi(1.960-2.806)\approx0.801\). Thus 56 observations just meet the approximate target for a 3-unit effect under this model. This calculation does not establish that the effect is meaningful; that judgment comes from the scientific context.

Why the MDE Changes

The formula also helps explain how design choices affect the threshold. Holding the test and other quantities fixed, a larger \(n\) reduces the standard error and lowers the MDE in proportion to \(1/\sqrt{n}\). Greater variability \(\sigma\) increases the MDE in direct proportion. Requiring higher power increases \(z_{1-\beta}\), and using a smaller significance level increases the relevant critical value; either change raises the MDE when other choices are held fixed.

These relationships do not mean that a researcher should simply choose whichever values produce a convenient threshold. The significance level, target power, and meaningful effect should reflect the consequences of errors and the scientific aim. An MDE is useful precisely because it makes the consequences of those choices visible on the outcome scale.

Specify the target.
State the outcome scale, null value, direction or two-sided alternative, significance level, and target power.

State the model.
Identify the test, sampling assumptions, variability value, and planned number of usable independent observations.

Calculate and interpret.
Find the MDE in context and compare it with an independently justified scientifically or practically meaningful effect.

Common Mistakes

Treating the MDE as a hard detection boundary. An effect just below the MDE can still be detected, and one at the MDE can fail to be detected. The MDE is defined by a target probability, not certainty.
Calling the MDE the smallest effect that exists or matters. It is a feature of the planned test and design. Scientific importance must be judged separately.
Reporting an MDE without its design assumptions. State the test, \(\alpha\), target power, \(n\), and variability assumptions so readers can interpret the threshold.
Choosing a one-sided test after seeing the data. Direction must be specified in advance; otherwise, the claimed significance level and power calculation do not describe the procedure actually used.
Confusing low power with evidence of no effect. A study designed to detect only large effects may not be informative about smaller effects that could still matter.

Key takeaway: The minimum detectable effect translates a planned test's power into the units of the research question. Use it to assess whether a design can detect effects worth detecting, while keeping statistical detectability distinct from practical importance.

Check Your Understanding

Answer each question using the assumptions and design choices stated.

In your own words, what does it mean for a study to have 80% power at its MDE?
For a two-sided mean test, what happens to the approximate MDE if \(n\) increases while all other design choices stay fixed?
A team chooses a one-sided test for an increase. What does its MDE say about the test's ability to detect a decrease?
A design's MDE is 5 units, while the team considers 2 units meaningful. What does that comparison suggest, and what does it not prove?
Why should the MDE be reported with the test, significance level, target power, sample size, and variability assumptions?