Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance (NCT05816473), a na na study. The primary outcome is Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT.

2. Clinical Question

3. Trial Design

This was a parallel-group simulation study concerning technology acceptance for machine-learning support in upper gastrointestinal bleeding. The registry lists the study as completed, with Yale University as sponsor. The recorded start date is 2023-05-23 and the primary completion date is 2024-12-31. The recorded enrollment is 108.

The two described conditions were large language model-based interaction and the machine-learning dashboard alone. In the first, a chatbot accompanied the dashboard and could provide a rationale for its risk assessment using interpretability information. In the second, participants had access to the dashboard without the chatbot. Participants were exposed to simulated cases; this is not described as a clinical treatment trial involving care decisions for actual patients.

There is an important ambiguity in how the comparison is characterized. The study summary says participants would be randomized to one of the two conditions, but the allocation entry is listed as NA, and the registry record cautions that the arms are not a like-for-like control comparison. The registry therefore does not establish a randomized allocation method. Masking is listed as none. The record does not report further details about assignment procedures, simulation cases, or how the two groups were made comparable.

The study is listed as not applicable for phase. Its stated focus is usability, attitudes, and provider trust when using a machine-learning clinical decision support system, rather than a clinical efficacy endpoint. The registry record does not report the number of participants who completed each stage or contributed data to the primary outcome analysis.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Baseline demographics presented here are for completers.

6. Primary Endpoint

The registry lists the primary outcome as Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT, measured over Approximately 60 minutes.

The primary outcome is described as the median change in attitudes toward machine-learning algorithms in clinical care, measured using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey. The survey assesses perceived usefulness, perceived ease of use, attitudes toward using the system, behavioral intentions, and trust. Responses use a 5-point Likert scale.

The record describes measurement at recruitment before the scenarios and immediately after their completion, with approximately 60 minutes between the assessments. It frames the comparison as a contrast between the large language model-based interaction and the machine-learning dashboard condition. A higher change is described as indicating greater acceptance or intention to use GutGPT+Dashboard.

The endpoint description is not fully consistent about the quantity being measured: the title specifies median change, while the explanatory text refers to percent change in survey response. The registry record does not resolve whether the intended outcome is a raw change, a percent change, or another summary. It also does not report how the survey items are combined, how change is calculated, or how the median and its 95% confidence interval are estimated.

Attitudes and trust are relevant to evaluating whether users find decision-support interfaces acceptable, but they are not themselves evidence of diagnostic accuracy or improved patient outcomes. No observed primary outcome estimate is reported in the registry record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The registry record identifies the primary outcome as a median change, with a 95% confidence interval, in attitudes measured using an adapted UTAUT survey. It does not report the analysis method, the observed outcome values, or the confidence interval itself. The explanations below describe statistical concepts in general and distinguish them from the methods documented for this study.

What does a median summarize?

In general, the median is the middle value after observations are ordered. It can describe the center of a distribution without being as sensitive as the mean to unusually high or low values. The UTAUT survey uses a 5-point Likert scale, for which response categories have an order. A median can summarize ordered responses, but the exact interpretation depends on how the survey items and any composite score are defined. The registry record does not say how individual responses are combined, whether the outcome is calculated for each participant before summarizing, or how missing items are handled.

What is a change outcome?

In general, a pre-to-post change compares a participant’s measurement after an intervention or experience with their measurement before it. A between-condition analysis could then compare changes between groups, or could use another specified model to account for baseline scores. The registry describes baseline measurement before the scenarios and a second measurement immediately after, approximately 60 minutes later. It does not report the precise calculation or model used to compare conditions.

There is a particular definition issue in this record. The primary outcome title says median change, while its description calls the comparison a percent change in UTAUT survey response. These are not interchangeable descriptions. The registry record does not clarify which value was the prespecified target, how a percentage would be calculated, or whether the same definition was used across the survey domains. Consequently, readers cannot identify the exact estimand from the record alone.

How should a confidence interval be read?

In general, a confidence interval describes uncertainty around an estimate under the statistical procedure used to construct it. Its width reflects the estimate’s precision under that procedure; a narrower interval generally indicates greater precision than a wider interval. The interval does not by itself resolve bias, confounding, or problems with the outcome definition. Although the primary outcome entry specifies a 95% confidence interval, the registry record does not report the interval, the method used to calculate it, or whether the interval is for each condition or a contrast between conditions.

What does the design imply for comparison?

A parallel-group design assigns different participants or units to different conditions and compares their outcomes. In general, a well-documented random assignment process can help balance measured and unmeasured characteristics across groups, although it does not guarantee identical groups. Here, the summary says participants would be randomized, but allocation is recorded as NA and the registry explicitly cautions that the arms are not a like-for-like control comparison. The assignment process and basis for that caution are not reported in the registry record. The evidence available therefore does not support assuming that the contrast has the interpretation of a clearly documented randomized trial.

Masking is listed as none. In general, lack of masking can matter for subjective outcomes because participants know which interface they use, and expectations may influence survey responses. The record does not describe any procedure to reduce that possibility or an analysis that accounts for it. This is a design consideration, not evidence that the reported attitudes were in fact affected by expectations.

What analysis details are unavailable?

The registry record reports no analyses. It does not specify a statistical model, a test, a method for estimating the median or its interval, handling of baseline measurements, or adjustments for participant characteristics. It also does not report assumptions, diagnostic checks, or an approach for multiple outcomes. These omissions mean that the analysis cannot be independently assessed from the registry entry. The record’s enrollment and safety denominators should not be substituted for the unknown number analyzed for the primary outcome.

More generally, statistical estimates from survey outcomes describe the responses collected under a particular instrument and setting. They do not automatically quantify model accuracy, clinical benefit, or the effect of deploying a system in routine care. This study used simulation scenarios, and the registry record does not provide results that would connect survey changes to patient outcomes.

10. Multiplicity, Missing Data and Analysis Populations

The registry record lists two outcome measures and identifies clinician decision making about triage of gastrointestinal bleeding as a secondary outcome. It does not report a multiplicity plan, such as which outcome would be treated as primary for inference or how uncertainty would be handled across multiple outcomes or survey domains. The primary outcome itself includes several constructs: usefulness, ease of use, attitudes, behavioral intentions, and trust. The record does not say whether these are analyzed separately, combined, or treated as components of a single score.

No analyses are listed in the registry record, and no outcome estimates or hypothesis tests are reported. It is therefore not possible to assess how many comparisons were conducted, which analyses were selected, or whether the results were adjusted for multiplicity. The registry record also does not report whether analyses were prespecified in a statistical analysis plan.

Missing-data handling is not reported. The entry does not state how incomplete surveys, absent post-scenario responses, or missing item responses were treated. It also does not report analysis populations, exclusions, or participant flow from enrollment through outcome measurement. The enrollment entry of 108 does not establish the number included in the primary analysis.

These omissions limit interpretation of both the estimates and their precision. They do not establish that missing data or multiple testing affected the results; they mean the registry record does not provide the information needed to evaluate those issues.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The safety entries list serious adverse events, other adverse events, and deaths for each condition. For the large language model-based interaction, 52 participants are listed as at risk and 0 as affected for serious adverse events, other adverse events, and deaths. For the machine-learning dashboard condition, 56 participants are listed as at risk and 0 as affected for each of those categories.

These are the figures recorded in the adverse-event portion of the registry. They describe reported counts within the listed at-risk groups; they do not provide details about how events were solicited, defined, or assessed. The record does not list serious-event terms. A zero recorded count should therefore be read as no events entered in that category in the registry record, not as proof that no untoward experiences occurred.

The safety entries are not results for the primary attitude outcome and should not be used as its analysis denominators. The registry record does not report a separate safety analysis or a fuller account of how safety information was collected during the simulation study.

12. What the Result Does, and Does Not, Mean

The study addresses a practical statistical and implementation question: whether adding a natural-language large language model interaction to a machine-learning dashboard is associated with different user-reported attitudes after simulated upper gastrointestinal bleeding cases. Its planned primary measure covers acceptance-related views, including perceived usefulness, ease of use, attitudes, behavioral intentions, and trust. A secondary outcome concerns clinician decision making in triage.

The registry record does not report the observed primary outcome or the secondary outcome. It therefore does not show whether the interface conditions produced different survey responses or triage decisions. The completed status and enrollment entry do not substitute for outcome estimates. Without estimates and uncertainty intervals, neither the size nor the precision of any difference can be judged.

Even if a change in attitudes were reported, such a result would describe responses on the specified survey in the context of this simulation. It would not, by itself, establish that the machine-learning tool gives accurate risk assessments, improves clinical decisions, or changes patient outcomes. Likewise, survey trust or intention to use should not be treated as equivalent to safe or effective use in practice.

The registry record also leaves uncertainty about the comparison: its summary refers to randomization, but allocation is NA and the record cautions that the conditions are not a like-for-like control comparison. Without clearer information on assignment and analysis, a contrast between conditions should not automatically be interpreted as a causal effect of adding the chatbot.

The findings that can be stated from this record are therefore about the study’s purpose, design as described, and reported safety counts. The record does not provide the statistical results needed to answer whether the added interface changed acceptance or decision making.

13. Why This Trial Matters Statistically

This study raises a distinction that matters in clinical biostatistics: evaluating a decision-support tool involves more than measuring whether users like or trust an interface. Acceptance measures can help characterize how participants respond to a system, while decision-making measures address how it is used in a task. Model performance and patient outcomes are different questions. Keeping these endpoints separate prevents an attitude score from being interpreted as evidence of clinical effectiveness.

The comparison also illustrates why the design description matters. The summary describes random assignment, but the allocation entry is NA and the registry record warns that the arms are not a like-for-like control comparison. Statistical interpretation depends on knowing how groups were formed and what differences the comparison is intended to estimate. A named parallel design alone does not resolve those questions.

Outcome definition is another important issue. The primary endpoint is titled as a median change, while its description refers to percent change. When the target quantity is ambiguous, even a clearly reported numerical estimate can be difficult to interpret or reproduce. Clear definitions of the score, time contrast, group comparison, and uncertainty method are essential for a useful statistical result.

Finally, the registry entry demonstrates the difference between study completion and a reportable result. It lists completion and enrollment, yet gives no outcome analyses. Without estimates, uncertainty intervals, and analysis details, readers cannot judge magnitude, precision, or the role of missing data and multiple outcomes. These limits are especially relevant when survey outcomes are used to inform implementation decisions.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record contains no primary or secondary outcome estimates and lists no analyses. Thus, the results needed to assess the study’s central questions are not reported in the registry record. The record also does not give confidence interval values or explain how the interval specified for the primary outcome was constructed.

The primary outcome is ambiguously defined. Its title refers to median change, but its description refers to percent change. The record does not explain the scoring of the adapted UTAUT instrument, how its domains are combined, or which of the stated constructs constitute the endpoint. The exact contrast and statistical estimand are therefore not fully specified.

There is also an inconsistency in the description of allocation. The study summary says participants would be randomized, whereas allocation is listed as NA and the registry record cautions that the arms are not a like-for-like control comparison. Details of assignment, group comparability, and the reason for the caution are not reported.

The registry record does not report participant characteristics, participant flow, the number with complete surveys, analysis populations, or missing-data procedures. It does not report a statistical analysis plan, multiplicity handling, or model assumptions. The record describes simulated cases, so it does not establish how results would translate to routine clinical care. These limitations concern what can be concluded from the registry record; they do not establish that the study itself lacked additional documentation or analyses.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance — NCT05816473.
• Registry record last updated 2026-05-22; status last verified 2026-05.
• Results first posted 2026-05-22.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

The registry describes a completed, parallel-group simulation study sponsored by Yale University, with a recorded enrollment of 108. It compares a large language model-based interaction paired with a machine-learning dashboard against the dashboard alone for simulated upper gastrointestinal bleeding cases. Masking is listed as none, and allocation is listed as NA, despite the study summary’s reference to randomization.

The primary outcome concerns change in attitudes measured using an adapted UTAUT survey, with a stated 95% confidence interval and an approximately 60-minute interval between baseline and post-scenario measurement. The record is unclear whether the outcome is median change or percent change. Clinician decision making about triage is listed as a secondary outcome.

No outcome analyses or estimates are reported. The safety entries list 0 affected for serious adverse events, other adverse events, and deaths in each condition. The registry record does not provide the analysis populations, missing-data procedures, or further statistical methods.