Why Simultaneous Targets Need a Joint Probability
A design may be required to satisfy several operating-characteristic (OC) requirements: for example, adequate power, controlled type I error, and a tolerable expected sample size. Reporting a probability for each requirement separately does not answer the question, “How likely is it that the design satisfies all of them at once?” That question concerns a joint probability.
In simulation-based design assurance, uncertainty about inputs is represented by a distribution over plausible input configurations. Examples include different recruitment rates, nuisance-parameter values, or missing-data conditions. For any one configuration, the design has OC values. The joint assurance is the probability, under the specified distribution of input configurations, that every OC requirement is met. It is not the probability that one simulated trial happens to show all desirable outcomes.
For example, \(q_1(\theta)\) might be power and its target might be at least 0.80, while \(q_2(\theta)\) might be the type I error rate and its target might be at most 0.025. The event of interest is that both inequalities hold for the same \(\theta\). The distribution \(G\), including its support and weights, is part of the assurance question; a different distribution can yield a different answer. The earlier material on go-no-go assurance under uncertain inputs provides the setting for this simultaneous extension.
Estimating the Intersection
When a finite set of \(M\) input configurations is used, assign each configuration a weight \(w_m\), where \(w_m\ge 0\) and \(\sum_{m=1}^{M} w_m=1\). For each configuration, determine whether all \(K\) criteria are met. The weighted proportion of configurations that pass every criterion estimates joint assurance. If the configurations are independent draws from \(G\), each has equal weight \(1/M\).
With equal weights, this reduces to the number of configurations passing all requirements divided by \(M\). The calculation must use the same configuration to assess each criterion before forming \(I_m\). Multiplying separately estimated probabilities is not a general substitute.
Worked Example: Joint Assurance From Eight Configurations
Suppose eight equally weighted hypothetical input configurations are evaluated against three requirements: power at least 0.80, type I error at most 0.025, and expected sample size at most 500. The table records whether each requirement passes at each configuration. These are hypothetical simulation results.
| Configuration | Power passes | Type I error passes | Sample-size passes |
|---|---|---|---|
| 1 | Yes | Yes | Yes |
| 2 | Yes | Yes | Yes |
| 3 | Yes | Yes | No |
| 4 | Yes | No | Yes |
| 5 | Yes | No | No |
| 6 | No | Yes | Yes |
| 7 | No | Yes | No |
| 8 | No | No | No |
Only configurations 1 and 2 pass all three requirements. Thus,
The marginal pass proportions are \(5/8=0.625\) for power, \(5/8=0.625\) for type I error, and \(4/8=0.500\) for expected sample size. Their product is \(0.625\times0.625\times0.500=0.1953\), not 0.250. The product would represent the joint probability only if the three pass events were independent under the input distribution. Here they are not independent.
Why the Requirements Can Be Dependent
The criteria are often evaluated under shared uncertain inputs. A change in an input such as recruitment rate can affect expected sample size and power together; a change in a nuisance parameter can affect power and type I error together. Consequently, configurations that pass one requirement may be more or less likely to pass another. Dependence concerns the events “requirement \(k\) passes,” not merely whether the numerical OC values move in the same direction.
For two pass events \(E_1\) and \(E_2\), the exact identity
follows because \(E(I_{E_1}I_{E_2})=P(E_1\cap E_2)\) and \(E(I_{E_k})=P(E_k)\). Thus the product of marginal probabilities is correct only when the two indicators have zero covariance; independence is a sufficient condition, but should not be assumed just because separate marginal results are available. For several criteria, the joint probability likewise depends on their joint pattern of passing and failing.
This is why joint assurance should be computed from the scenario-level pass indicators or from weighted joint outcomes, rather than reconstructed from marginal summaries. Reporting marginal probabilities remains useful: it shows which individual requirements are frequently met. It does not replace the joint result.
Worked Example: Joint Assurance With Unequal Scenario Weights
Suppose four hypothetical configurations represent an input distribution with weights 0.10, 0.20, 0.30, and 0.40. A configuration passes the combined requirements only if it passes every target. Assume configurations 1 and 3 pass all targets, while configurations 2 and 4 do not.
The joint assurance estimate is the total weight of the configurations that pass:
The answer is 0.40, not \(2/4=0.50\), because the four configurations are not equally weighted. The interpretation is that 40% of the probability mass in the specified input distribution lies on configurations satisfying all the requirements. It is not a claim that 40% of future trials will meet all targets.
If the weights describe a deliberate scenario mixture rather than random draws, the result is conditional on that mixture. Changing the weights changes the assurance estimate even if the design and the scenario-level pass/fail results stay the same.
Simulation Uncertainty and Threshold Classification
Often an OC value at a particular configuration is not known exactly; it is estimated by simulating trial outcomes. This creates a second layer of uncertainty. The outer layer concerns variation across uncertain input configurations. The inner layer concerns Monte Carlo error in estimating an OC at a given configuration. Near a target boundary, inner simulation error can change whether a configuration is labelled as passing.
For a probability OC estimated from \(R\) independent simulated trials at a fixed configuration, with \(\hat{p}\) the simulated proportion of the specified outcome, the estimated standard error is approximately \(\sqrt{\hat{p}(1-\hat{p})/R}\). This approximation describes simulation error for that proportion; it does not describe variation in the uncertain inputs. If the estimated OC is close to its target, a hard pass/fail label can therefore be unstable.
Worked Example: A Power Estimate Near Its Target
At one hypothetical input configuration, suppose 332 of \(R=400\) simulated trials meet the prespecified success criterion. The estimated power is \(\hat{p}=332/400=0.830\), just above a target of 0.80. The estimated Monte Carlo standard error is
A rough normal 95% interval for the simulated power is \(0.830\pm1.96(0.0188)\), or approximately 0.793 to 0.867. Because this interval crosses 0.80, the simulation does not precisely distinguish a true power just below the target from one just above it. Calling the configuration a definite pass based only on the point estimate can overstate the certainty of its classification.
One practical response is to increase the number of simulated trials at configurations near decision boundaries, then reassess the OC estimate. For illustration, if 3,320 of 4,000 hypothetical trials meet the criterion, the estimated power remains 0.830 and \(SE\approx\sqrt{0.830(0.170)/4000}=0.00594\); the corresponding rough 95% interval is about 0.818 to 0.842. The additional simulation reduces inner Monte Carlo error. It does not remove uncertainty about which input configurations are plausible.
Justifying the Outer Simulation Estimate
If \(\theta_1,\ldots,\theta_M\) are independent draws from \(G\), and the true joint pass indicator for draw \(m\) is \(I_m\), then each \(I_m\) is a Bernoulli random variable with success probability \(A\). The estimator is their sample mean. Its expectation and variance follow directly:
The variance equality uses independence of the outer draws and \(\operatorname{Var}(I_m)=A(1-A)\). Replacing \(A\) by \(\widehat{A}\) gives the estimated standard error \(\sqrt{\widehat{A}(1-\widehat{A})/M}\). This quantifies outer Monte Carlo error when classifications are treated as known. With unequal fixed scenario weights, the weighted estimate remains the relevant average, but this simple equal-weight standard-error formula does not generally apply.
Worked Example: Reporting Outer Monte Carlo Error
Suppose \(M=100\) independent hypothetical input draws are sampled from the specified distribution, and 37 configurations pass every target. Then \(\widehat{A}=37/100=0.370\). The estimated standard error from outer sampling is
A rough normal 95% interval is \(0.370\pm1.96(0.0483)\), approximately 0.275 to 0.465. This is an approximate interval for joint assurance under repeated outer sampling, subject to the assumed distribution \(G\) and the accuracy of each pass/fail classification. It is not a 95% probability statement about a fixed but unknown assurance value. The approximation can be poor with very small samples or estimates near zero or one; the number of outer draws and the interval method should be reported alongside the estimate.
Interpretation, Diagnostics, and Alternatives
Joint assurance is useful when design acceptance genuinely requires simultaneous performance. It prevents a design from appearing adequate simply because each marginal probability looks favorable in isolation. It also makes explicit the dependence induced by shared uncertain inputs. Its limitations are equally important: the result depends on \(G\), the chosen targets, the model used to generate trial outcomes, and how uncertain OC estimates are classified. It does not establish that the assumed input distribution is scientifically appropriate.
A sound analysis should check that scenario weights are nonnegative and sum to one, that every target has a clear inequality and unit or scale, and that the same scenario-level information is used to determine joint passage. For simulation-based OCs, examine whether decisions are sensitive to Monte Carlo error near cutoffs. If many configurations sit close to a threshold, the estimated joint assurance may be particularly sensitive to simulation precision.
Joint assurance complements rather than replaces other summaries. Marginal pass probabilities reveal the frequency with which each criterion is met, while the joint probability answers whether they are met together. A worst-case assessment asks whether requirements hold under the most adverse specified inputs, which is a different question from the probability of passing under \(G\). Weighted scenario averages also depend on the stated weights. These summaries can disagree without contradiction because they answer different questions.
A common mistake is to multiply the marginal pass probabilities and call the result joint assurance. That calculation silently assumes independence of the pass events. When the same uncertain inputs affect several OCs, that assumption may fail. The correction is to form the all-target indicator for each input configuration and average those indicators using the specified weights. A second mistake is to interpret joint assurance as the chance that one future trial satisfies every target; the targets concern design-level OCs, and assurance is a probability over uncertain inputs.
Check Your Understanding
Answer each question, then compare with the short solutions below.
- Two target-pass events each have marginal probability 0.70. If they are independent, what is their joint probability? Why can it not be assumed without evidence?
- Five equally weighted input configurations have joint pass indicators \(1,1,0,1,0\). Find the joint assurance estimate.
- For four configurations with weights 0.15, 0.25, 0.20, and 0.40, only configurations 2 and 4 pass all targets. Find the weighted estimate.
- In 500 simulated trials, an OC proportion is estimated as 0.84. Calculate its approximate Monte Carlo standard error.
- Explain why a worst-case assessment and a joint assurance estimate can lead to different conclusions.
Solutions. (1) Under independence, \(0.70\times0.70=0.49\). Without independence, the joint probability depends on how the two pass events co-occur. (2) Three of five configurations pass, so \(\widehat{A}=3/5=0.600\). (3) Add the passing weights: \(0.25+0.40=0.650\). (4) \(SE=\sqrt{0.84(0.16)/500}=\sqrt{0.0002688}\approx0.0164\). (5) Worst-case assessment concerns performance at the most adverse specified input, while joint assurance averages the all-target indicator over the input distribution; the average can be favorable even if one adverse configuration fails.