Preventing Medication Dispensing Errors in Pharmacy Practice With Interpretable Machine Intelligence: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Preventing Medication Dispensing Errors in Pharmacy Practice With Interpretable Machine Intelligence (NCT06245044), a randomized na study. The primary outcome is Reaction Time.

2. Clinical Question

3. Trial Design

The registry describes a randomized, unmasked, crossover study of behavioral conditions. Participants completed medication-verification tasks under No MI Help and the two MI scenarios. Because the design is crossover, the conditions are repeated within a participant rather than assigned as separate parallel groups. This matters when interpreting comparisons: measurements from the same person can be related, and an analysis that accounts for that relationship may differ from a comparison that treats observations as independent.

The record lists enrollment of 68 and a completed status. For the primary reaction-time outcome, the posted summaries list 50 observations in each condition. The registry record does not explain the relationship between the enrollment figure and the condition-level observations, or how many participants contributed to every outcome. The sequence of conditions, any washout or familiarization procedures, and the handling of incomplete task data are not reported in the registry record.

The primary outcome was reaction time throughout the verification task. Secondary outcomes listed in the record include cognitive effort, workload and usability, although detailed results are posted for workload and usability. The timeframe for those measures is after completing 100 mock verification trials in each condition. The registry lists 8 outcome measures and no reported analyses. The statistical analysis plan, including whether the crossover structure was handled in the analyses, is not reported in the registry record.

The study is marked completed. Its recorded start date is 2024-04-11, and its primary completion date is 2024-12-04. The record lists the phase as NA. It identifies the University of Michigan as the sponsor. No masking was used, so participants and task administrators were not described as blinded in the registry design.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

No MI help

Reasons for not completing this period:

Scenario 1

Scenario 2

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Reaction Time, measured over Throughout the verification task.

The primary endpoint is reaction time, measured in milliseconds throughout the medication-verification task. The registry describes the underlying task time as the time from starting a task until accepting or rejecting a medication image. This captures how long a verification decision takes; it does not, by itself, indicate whether that decision was accurate.

Task time is relevant to the study’s stated question because MI advice may arrive at different moments in the decision process. A pop-up triggered by disagreement with the participant’s decision could affect time differently from assistance displayed alongside the images. Comparing reaction time across these conditions can therefore describe whether the decision process took longer or shorter in the recorded tasks.

The endpoint is reported as a mean with a standard deviation. A mean describes the average task time, while the standard deviation describes how spread out task times were around that average. The posted means are 3668.0 milliseconds for No MI Help, 4727.0 milliseconds for Scenario #1 and 4510.0 milliseconds for Scenario #2. The corresponding standard deviations are 924.0, 1040.0 and 1339.0 milliseconds.

These timing results address speed, not the full range of outcomes named in the study rationale. Accuracy and trust results are not reported in the registry record. Thus, reaction time alone cannot establish whether faster decisions were more accurate, whether participants trusted the MI, or how task-time differences would translate to routine pharmacy practice.

7. Results Posted in the Registry

• Scenario #1: mean 4,727 versus 3,668 on control; difference 1,059 (95% CI 673.39 to 1,445), standardized effect d = 1.08; z-test p = <0.001.
• Scenario #2: mean 4,510 versus 3,668 on control; difference 842.00 (95% CI 391.06 to 1,293), standardized effect d = 0.73; z-test p = <0.001.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Workload

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Scenario #1: mean 7.50 versus 7.40 on control; difference 0.10 (95% CI -1.37 to 1.57), standardized effect d = 0.03; z-test p = 0.894.
  • Scenario #2: mean 6.90 versus 7.40 on control; difference -0.50 (95% CI -1.89 to 0.89), standardized effect d = -0.14; z-test p = 0.482.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Scenario #1 and the control. The interval gives the range of effects that remain compatible with the data.
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Scenario #2 and the control. The interval gives the range of effects that remain compatible with the data.

Usability

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Scenario #1: mean 70.40 versus 72.10 on control; difference -1.70 (95% CI -8.15 to 4.75), standardized effect d = -0.10; z-test p = 0.605.
  • Scenario #2: mean 73.00 versus 72.10 on control; difference 0.90 (95% CI -5.39 to 7.19), standardized effect d = 0.06; z-test p = 0.779.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Scenario #1 and the control. The interval gives the range of effects that remain compatible with the data.
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Scenario #2 and the control. The interval gives the range of effects that remain compatible with the data.

The posted secondary results cover workload and usability, with the registry also listing cognitive effort among its secondary outcomes. Across workload and usability, the unadjusted comparisons do not show a clear separation from No MI Help: the confidence intervals include no difference, and the p-values are not small. The overall pattern is therefore different from the primary reaction-time comparisons, where both MI conditions had longer mean times and the unadjusted confidence intervals were above no difference.

These outcome domains answer different questions. Workload and usability scores describe participants’ reported experience of the conditions; reaction time describes how long the verification task took. A lack of a clear difference in the posted secondary comparisons does not negate the observed timing pattern, and the timing pattern does not establish a difference in workload or usability. Nor can these outcomes fill the gaps in accuracy and trust results, which are not reported in the registry record.

Secondary findings generally carry less evidential weight when many outcomes or comparisons are examined without a reported multiplicity plan. Each additional comparison creates more opportunities for chance findings. The registry lists 8 outcome measures and no reported analyses, and it does not report whether secondary outcomes were prespecified as confirmatory or whether multiplicity was addressed. Accordingly, these results are best treated as descriptive and exploratory rather than as independent confirmation of the primary finding.

The record gives detailed results for workload and usability but not all listed secondary outcomes. It does not report the secondary analyses or their analysis populations. The pattern across the posted outcomes is informative for framing further study, but it does not provide a complete account of participants’ cognitive effort, trust or other unreported measures.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Cognitive Effort
• Cognitive Effort
• Workload
• Usability

9. Statistical Interpretation

10. Statistical Methods Explained

The registry identifies the study as randomized and crossover, with no masking. Randomization is intended to reduce systematic differences in the order or assignment of conditions, while a crossover design allows participants to encounter more than one condition. In a crossover study, observations from the same participant may be correlated. General statistical principle: an analysis should account for that dependence when estimating condition differences and their uncertainty. The registry record does not report the randomization sequence, period effects, carryover handling or a model accounting for repeated measurements.

What does the mean represent?

The primary outcome is a continuous measure summarized by its mean and standard deviation. The mean gives the average recorded task time within a condition. The standard deviation describes variation in task times among the observations. The reported mean times are 3668.0 milliseconds for No MI Help, 4727.0 milliseconds for Scenario #1 and 4510.0 milliseconds for Scenario #2. Their standard deviations are 924.0, 1040.0 and 1339.0 milliseconds, respectively. A standard deviation is not the same as a confidence interval: it describes variation in observations, whereas a confidence interval describes uncertainty around an estimated quantity.

How should a mean difference be read?

The posted comparison is an unadjusted mean difference between each scenario and No MI Help. A positive difference means the scenario’s reported mean time is higher than the comparator’s reported mean time. The estimates are 1059.0 milliseconds for Scenario #1 and 842.0 milliseconds for Scenario #2. The accompanying confidence intervals quantify uncertainty under the calculation used for those comparisons. The registry lists no reported analyses, and it does not identify a sponsor analysis for these outcomes. The statistical plan is not reported in the registry record.

These comparisons are based on condition-level summary statistics. General statistical principle: in a crossover study, a paired estimate can use the fact that each participant serves as their own comparison. Its uncertainty depends on how strongly measurements within the same person are related. That within-person covariance is not reported in the registry record, so it cannot be recovered from the posted means and standard deviations alone. The unadjusted summary comparisons therefore should not be assumed to reproduce a paired crossover analysis.

What do the p-values tell the reader?

The reported p-values for both primary comparisons are p < 0.001. A p-value measures how compatible the observed data are with a specified null model, given the assumptions of the test. It is not the probability that the null hypothesis is true, and it does not measure the size or practical importance of an effect. The comparison estimates and confidence intervals are more informative about the direction and range of the observed difference. The record does not report whether any multiplicity adjustment was applied.

Why can design details change the analysis?

In a crossover design, order may matter: task performance can change with practice, fatigue or exposure to an earlier condition. Carryover can also matter if an earlier condition affects behavior in a later one. These are general design considerations, not evidence that such effects occurred in this study. The registry record does not report an analysis of order, period or carryover effects, nor does it describe how the crossover nature of the data was incorporated into the reported comparisons.

The record lists 68 enrolled participants, while each primary condition summary reports 50 observations. Those figures describe different registry entries and should not be combined or treated as an explanation of participant flow. The number of participants contributing paired observations, the treatment of missing data, and the analysis population are not reported in the registry record.

Finally, the record lists 8 outcome measures but no reported analyses. The posted primary and secondary summaries allow descriptive and unadjusted comparisons, but they do not provide a complete account of prespecified hypotheses, model specification or analysis decisions. General statistical principle: interpretation is stronger when estimates, uncertainty and methods are considered together, particularly when the study has repeated measurements and multiple outcomes.

11. Confidence Intervals and Precision

Confidence intervals express uncertainty around an estimated mean difference under the assumptions used to calculate them. They show a range of values compatible with the data and calculation method; they are not a guarantee that the true value lies within the interval. Here, the primary comparison interval for Scenario #1 versus No MI Help runs from 673.4 to 1445.0 milliseconds. The interval for Scenario #2 versus No MI Help runs from 391.1 to 1293.0 milliseconds. Both intervals are entirely above no difference, consistent with longer mean task times in the two MI conditions in these unadjusted comparisons.

The interval widths also matter. Each allows a range of plausible mean differences rather than a single exact effect. The estimates should therefore not be treated as precise predictions for every pharmacist or task. The standard deviations of the condition-level task times—924.0 milliseconds, 1040.0 milliseconds and 1339.0 milliseconds—show that individual observations varied around their respective means. Those standard deviations describe observed spread, not uncertainty in the difference estimate.

Precision depends on the data and the analysis assumptions. In this crossover study, a paired analysis would use within-participant information. The registry record does not report the covariance needed to reconstruct such an estimate, nor does it provide a statistical analysis plan or detailed model. Therefore, the displayed unadjusted intervals should be read as intervals for the posted summary-statistic comparisons, not as proof that every relevant feature of the crossover design has been accounted for.

The intervals address task time only. They do not quantify uncertainty about accuracy, trust, clinical outcomes or effects in routine practice, for which results are not reported in the registry record.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size planning figures are participants per arm for 80% power and participants per arm for 90% power. They are based on two-sided significance level 0.05, equal allocation between the two arms, a normal approximation and an effect set to the stated fraction of the observed effect. The original trial’s own sample-size assumptions and justification are not reported in the registry record.

The 80% power column describes a planning scenario in which a study designed with the stated assumptions would have an 80% chance of detecting the chosen effect if that effect were the true one. The 90% column describes the corresponding planning target at 90% power. These are planning illustrations, not observed power for this completed study.

Rows planning for 100%, 75% or 50% of the observed effect show how the assumed effect size changes the participant requirement. Smaller effects are harder to distinguish from random variation, so planning for a smaller fraction of the observed effect requires more participants per arm. The very small observed workload difference in Scenario #1, for example, produces much larger planning figures than the primary reaction-time comparisons. Such figures are sensitive to the observed effect and assumptions, so they should not be treated as a recommended sample size without a design-specific justification.

The calculations use a two-arm framework for each comparison. The registry describes a crossover study, but the listed planning assumptions do not describe a paired crossover calculation. The figures therefore should not be interpreted as a reconstruction of the trial’s original design or analysis plan.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = 1059.000; pooled standard deviation σ = 983.711.
2. n = 2 × (1.96 + 0.842)² × 983.711² / 1059.000² = 13.5, rounded up to 14 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- 1059                      # observed difference in means
sigma <- 983.711339774021                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(6245044)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 14 (80% power), 19 (90% power)
 75% of observed effect: n per arm = 25 (80% power), 33 (90% power)
 50% of observed effect: n per arm = 55 (80% power), 73 (90% power)
power.t.test: 14.6 
Simulated power at n = 14 per arm: 0.784 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

Multiplicity arises when a study evaluates more than one outcome, contrast or analysis. Even when each test uses a conventional significance threshold, performing many tests increases the chance that at least one result will appear statistically notable by chance. This general issue is relevant because the registry lists 8 outcome measures and the study includes two MI scenarios compared with a no-help condition. The record does not report a multiplicity adjustment, a hierarchy of confirmatory outcomes, or a prespecified plan identifying which comparisons were primary and which were exploratory.

The registry identifies reaction time as the primary outcome, while workload and usability results are posted as secondary outcomes. The registry also lists cognitive effort among secondary outcomes, but detailed results for all listed measures are not available. The record lists no reported analyses. It therefore does not establish whether the displayed secondary comparisons were planned in advance, whether any additional analyses were performed, or how missing secondary results should be understood. Absence of a posted result is not evidence that an outcome showed no effect.

Missing-data methods and analysis populations are also not reported in the registry record. The record lists enrollment of 68, while primary condition summaries list 50 observations per condition. It does not explain participant flow, the number with complete data across conditions, reasons for missing observations, exclusions, or whether all randomized participants were analyzed. The posted counts should not be used to infer those details.

These omissions matter for interpretation. Incomplete data can affect estimates if the chance of missingness is related to a participant’s performance or experience. General statistical principle: conclusions may depend on whether analyses include all randomized participants, only those with complete measurements, or another defined population. Without a reported analysis plan and participant flow, readers cannot assess how those choices were handled. The primary contrasts remain descriptive evidence from the posted summaries, but the registry record does not permit a full appraisal of multiplicity, missingness or analysis-population decisions.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The serious adverse event figures list 0 events for No MI Help, 0 for Scenario #1 and 0 for Scenario #2. The record also lists 0 deaths in each condition. These figures indicate that no serious adverse events or deaths were recorded in the posted results for the listed groups.

Zero recorded events should not be interpreted as proof that an intervention is risk-free. The figures are bounded by the study setting and the events captured in the registry. This study involved mock verification tasks, and the record does not describe clinical exposure to medication dispensing or patient outcomes. It also does not provide detailed information about how adverse events were solicited or adjudicated.

The registry lists 50 at risk for serious adverse events and 50 at risk for other adverse events in each condition. No other adverse events were reported in these entries. The registry record does not provide further safety analyses or event definitions. These data therefore describe the posted event counts, not a comprehensive assessment of clinical safety.

16. What the Result Does, and Does Not, Mean

The main result is a difference in task time under the tested mock conditions. Mean reaction time was higher in each MI scenario than in No MI Help, and the posted unadjusted comparisons have confidence intervals above no difference. This establishes what the summaries show about timing in these tasks, not what the assistance would do in routine pharmacy practice.

Task time is only one part of verification performance. The study rationale refers to accuracy and trust, but results for those outcomes are not reported in the registry record. A longer time could reflect additional checking, distraction, uncertainty or other processes; the timing result alone does not identify which explanation applies. It cannot show whether the extra time improved error detection, whether participants accepted MI advice, or whether medication errors were prevented.

Nor do the posted results establish that MI support is ineffective. The study compares task times across conditions, and the observed direction is slower timing with MI. That does not answer whether the assistance may have benefits on other outcomes or in other settings. The secondary workload and usability comparisons do not resolve the absent accuracy and trust evidence.

The crossover design makes the analysis method important. The registry record does not report whether the comparisons accounted for within-participant correlation, condition order or carryover. The unadjusted estimates are therefore best read as comparisons of posted condition summaries, not as a complete model-based assessment of all design features.

Generalizability is also limited by the task context. The outcomes were recorded during mock verification, with secondary measures collected after 100 mock trials in each condition. The registry does not report results from actual dispensing, patient outcomes or sustained use. The results can inform questions for future evaluation, but they do not quantify clinical benefit, patient risk or effects in routine workflow.

17. Why This Trial Matters Statistically

This study raises a statistical question that is increasingly important for decision-support tools: how should performance be evaluated when assistance changes both the information available and the timing of a decision? Reaction time offers a measurable outcome, and the two MI formats allow the timing and presentation of advice to be considered separately from the no-help condition.

The posted results also illustrate why a statistically clear difference is not the same as a complete assessment of a technology. Both MI conditions had longer mean task times than No MI Help in the unadjusted primary comparisons. That pattern is relevant to workflow, but speed has to be interpreted alongside accuracy, trust and safety. Those results are not reported in the registry record. Without them, the central trade-off the study rationale names cannot be evaluated from the posted outcomes.

The crossover design gives the study an opportunity to compare conditions within participants, which can be statistically useful when the analysis properly accounts for repeated measurements. At the same time, such a design requires attention to order, practice and carryover. The registry record does not describe whether these issues were handled. This makes transparent reporting of the analysis especially important.

Finally, the record distinguishes a planned study question from the results available for interpretation. A completed study with posted outcome summaries can still leave key questions unanswered when analyses, missing-data handling and results for relevant outcomes are not reported. The case demonstrates why estimates, uncertainty intervals, design-aware methods and complete outcome reporting all matter when evaluating machine intelligence in clinical work.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to the posted record and task setting. The study used mock verification tasks, so the results do not directly describe routine pharmacy dispensing or patient outcomes. The registry record does not report whether the task conditions reproduce the interruptions, workload or consequences encountered in practice.

The primary outcome captures speed but not correctness. Although the study rationale names accuracy and trust, those results are not reported in the registry record. The posted results therefore cannot determine whether the longer reaction times were associated with more accurate decisions, greater trust, or a change in error detection.

The study is described as crossover, but the posted comparisons are unadjusted comparisons of condition summaries. The record does not report how within-participant dependence, condition order, learning or carryover were handled. It also does not report the analysis plan, missing-data procedures, participant flow or analysis populations. Enrollment is listed as 68, whereas the primary condition summaries each list 50 observations; the record does not explain this relationship.

There are multiple outcomes and comparisons, but no multiplicity strategy or reported analyses are listed. Detailed results are not available for all secondary outcomes named in the record. This limits assessment of the completeness and confirmatory status of the posted findings.

Finally, the adverse-event entries show no recorded serious events or deaths, but the record does not describe event ascertainment in detail. In a mock-task study, such counts do not constitute a broad evaluation of clinical safety. The registry record also does not report whether participants had prior experience with MI tools or whether results varied by experience.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: Preventing Medication Dispensing Errors in Pharmacy Practice With Interpretable Machine Intelligence — NCT06245044.
• Registry record last updated 2025-11-26; status last verified 2025-09.
• Results first posted 2025-11-26.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

The registry record describes a completed, randomized, unmasked crossover study sponsored by the University of Michigan. It evaluated No MI Help and two MI presentation scenarios during mock medication verification. Enrollment is listed as 68.

The primary outcome was reaction time throughout the verification task. Mean times were 3668.0 milliseconds for No MI Help, 4727.0 milliseconds for Scenario #1 and 4510.0 milliseconds for Scenario #2. The unadjusted mean differences versus No MI Help were 1059.0 milliseconds and 842.0 milliseconds, respectively; both had p < 0.001.

Workload and usability summaries are also posted. Serious adverse events and deaths were listed as 0 in each condition. Accuracy and trust results, the statistical analysis plan, missing-data methods and details of crossover analyses are not reported in the registry record.