Superiority by a Margin Test for Two Means

Methodology

This calculator implements the two-independent-means superiority-by-a-margin t-test described in the this method procedure Superiority by a Margin Tests for Two Means using Differences. this method describes the design as a parallel-group comparison of two independent normally distributed means and gives both the equal-variance two-sample t-test and Welch's t-test for unequal variances.

Hypotheses

When higher values are better, the hypotheses are:

When higher values are worse, the hypotheses are:

Here, M is the positive superiority margin, Group 1 is the treatment group, and Group 2 is the reference group.

Test statistic

For higher values being better, the observed difference is compared with the margin:

For higher values being worse, the corresponding statistic is:

With equal variances, the standard error uses the pooled variance and the degrees of freedom are n₁ + n₂ − 2. With unequal variances, the calculator uses Welch's standard error and Welch-Satterthwaite degrees of freedom.

Decision rule

The test is one-sided. For higher values being better, a p-value below alpha rejects the null in favor of superiority; for higher values being worse, a p-value below alpha rejects the null in the lower tail. The displayed one-sided confidence bound is the corresponding 1 − alpha bound for the treatment-minus-reference mean difference.

Validation example

A fixed implementation check uses n₁ = n₂ = 20, means 2.00 and 0.00, standard deviations 2.00 and 2.00, superiority margin M = 0.50, higher values better, equal variances, and alpha = 0.05. The expected calculation is t = 2.371708, df = 38, one-sided p = 0.0114379, so the null hypothesis is rejected.

References

1. the software, LLC. Superiority by a Margin Tests for Two Means using Differences, this method Sample Size Software, Chapter 448. The documentation presents the hypotheses, pooled two-sample t-test, Welch's t-test, and superiority-margin framework.
2. Chow, S.-C., Shao, J., Wang, H., & Lokhnygina, Y. (2018). Sample Size Calculations in Clinical Research, 3rd ed. CRC Press. the relevant methodological literature cites pp. 50–51 for superiority-test sample-size formulas.
3. Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29(3/4), 350–362.
4. Statsols. Advanced 9.4.1 User Manual. The documentation describes “Inequality” as the menu category for superiority hypothesis tests and lists two-group mean procedures.