Bayesian Sample Size Methods
Fixed-N Bayesian design analysis for a two-group comparison of means. Monte Carlo simulation estimates how often a study will produce compelling evidence for the expected effect or for the null, using the Jeffreys-Zellner-Siow (JZS) Bayes factor and a Cauchy prior for the standardized effect size. Runs entirely in your browser.
Bayes Factor Design Analysis (BFDA) is a simulation-based approach to planning studies in which Bayes factors are used as the measure of evidence. For a fixed-N design, repeated datasets are generated under the expected alternative and under the null, the Bayes factor is calculated for every simulated dataset, and the resulting distribution of evidence is summarized. This allows the investigator to estimate the probability of obtaining a specified level of evidence before collecting the actual data.
This calculator implements the fixed-N two-independent-group mean comparison described by Schönbrodt and Wagenmakers. Under H0, the standardized population mean difference is zero. Under H1, the standardized difference is the user-specified expected effect δ. Each simulated group is normally distributed with common variance and equal sample size.
The analysis uses the Jeffreys-Zellner-Siow framework for the two-sample t test. The standardized effect under the alternative is assigned a symmetric Cauchy prior centered at zero with user-selected scale r. The default scale is r = √2/2, the conventional medium prior scale used for the two-sample JZS t test.
Because the BFDA design described in the literature commonly concerns directional research hypotheses, the calculator reports the directional JZS Bayes factor for the expected positive effect. For positive observed t statistics, the numerator integrates the noncentral-t likelihood over the positive half of the Cauchy prior. For negative observed statistics, the corresponding directional evidence is obtained from the symmetric JZS model.
For each simulated study, the calculator generates two independent samples, computes the pooled-variance t statistic, and evaluates the directional JZS Bayes factor. The process is repeated separately under H1 and H0.
The target probability is therefore analogous to a true-positive rate: it is the long-run probability that a study generated under the specified expected effect reaches the selected Bayes-factor threshold. Unlike ordinary frequentist power, the design criterion is based on the strength of Bayesian evidence rather than a p-value cutoff.
A design that does not reach the target probability at the selected sample size can be explored by increasing N per group and rerunning the analysis. Because BFDA is simulation-based, the displayed probabilities contain Monte Carlo error. Increasing the number of simulations reduces that error. The random seed is exposed so that a particular simulation run can be reproduced.
The published BFDA literature uses an independent two-group design with 20 observations per group, an expected standardized effect of δ = 0.5, a Cauchy analysis prior with scale √2/2, and a Bayes-factor decision boundary of 10 as an illustrative fixed-N design. With the default seed and 10,000 simulations in this implementation, the calculator produces a reproducible Monte Carlo result for that configuration. Because Monte Carlo BFDA results vary with the simulation seed and number of replications, published figures should not be treated as an exact deterministic output for every implementation.
Schönbrodt, F. D., & Wagenmakers, E.-J. (2018). Bayes factor design analysis: Planning for compelling evidence. Psychonomic Bulletin & Review, 25(1), 128–142. doi:10.3758/s13423-017-1230-y.
Stefan, A. M., Gronau, Q. F., Schönbrodt, F. D., & Wagenmakers, E.-J. (2019). A tutorial on Bayes Factor Design Analysis using an informed prior. Behavior Research Methods, 51, 1042–1058. doi:10.3758/s13428-018-01189-8.
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237.
Morey, R. D., & Rouder, J. N. BayesFactor: Computation of Bayes Factors for Common Designs. The package documentation describes the JZS t-test implementation and its Gaussian-quadrature computation.
, Sample Size for Bayesian Statistics. The Bayesian module includes Bayes-factor-based sample-size/design procedures and describes Bayes factors as a method for directly comparing statistical hypotheses.