Clinical Decision Support for Opioid Use Disorders in Medical Settings: Usability Testing in an EMR: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Clinical Decision Support for Opioid Use Disorders in Medical Settings: Usability Testing in an EMR (NCT03559179), a non-randomized na study. The primary outcome is Intervention PCP Confidence in Assessing and Treating OUD.

2. Clinical Question

The broader pilot objectives were to program the tool for an electronic medical record and assess provider usability and acceptability. The study also planned to examine whether opioid use disorder case-finding, medication-assisted therapy use, and referral patterns differed before and after deployment or between providers with and without access. Those objectives describe what the pilot set out to assess; they are not evidence that any particular change occurred.

The question is about provider confidence and the feasibility of a clinical support approach, rather than a patient-level treatment comparison. The posted record does not report a patient outcome result or a completed statistical comparison for the secondary objectives. Consequently, it cannot establish whether access changed diagnoses, medication use, referrals, or patient outcomes.

3. Trial Design

This was a completed pilot usability study of the Opioid Wizard, a clinical decision support tool for opioid use disorder in an electronic medical record. The registry lists the study as non-randomized, with no masking and a parallel model. Enrollment was 55. The study began on 2018-04-30 and reached primary completion on 2019-02-14; its status is completed. The phase is listed as NA.

The registry describes three provider groups: waivered providers who received the tool, non-waivered providers who received it, and non-waivered providers who did not receive it. The latter group was described as continuing usual care. The arm descriptions state that non-waivered providers would be randomized to receive or not receive the tool, while the overall allocation field says non-randomized. This inconsistency means the assignment process cannot be fully characterized from the registry record. Waivered providers were described as receiving the tool, so access was not allocated in the same way for all provider groups.

The parallel structure indicates that groups were considered during the same study period, rather than the registry describing a single group crossing over between conditions. It does not, by itself, establish that groups were comparable at baseline. Masking is listed as none, so the record does not describe blinded assignment or assessment. Whether providers knew all details of the study or how endpoint responses were collected is not reported in the registry record.

The stated primary objective combined programming the tool with assessing provider usability and acceptability. A secondary objective was to evaluate its usefulness through pre- and post-deployment comparisons of case-finding and medication-assisted therapy, as well as referral comparisons between intervention and control groups after deployment. The registry lists five outcome measures, but it reports zero analyses. A protocol-level statistical plan, recruitment details, and the rationale for the enrollment target are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: This outcome was to help show proof of the OUD-CDS concept using responses from intervention PCPs. One intervention provider stopped working for the organization during the intervention. Two additional intervention providers in the "Non-Waivered providers who received the Opioid Wizard" arm did not complete follow-up surveys.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Intervention PCP Confidence in Assessing and Treating OUD, measured over This outcome measure was calculated at approximately month 10 of the pilot study.

The primary endpoint is the count of intervention primary care providers who reported feeling moderately or very confident in assessing and treating opioid use disorder. The registry says it was calculated at approximately month 10 of the pilot study. It was intended to help demonstrate proof of concept for the decision support tool and relates to the stated objective of obtaining high provider usability and acceptability.

This is a provider-reported measure of confidence, not a direct measure of clinical skill, adherence to guidelines, patient diagnosis, medication use, or patient health. Combining the moderately and very confident response categories defines which responses qualify for the count, but the record does not report the number meeting that definition, the number answering the question, or the response instrument and its scoring details. The specific analysis population is also not reported in the registry record.

Confidence can be useful in an early usability pilot because a tool intended to support clinical work must be usable and acceptable to its intended users. As a general measurement principle, however, self-reported confidence reflects a participant’s own assessment and should not be treated as interchangeable with observed behavior or clinical outcomes. The endpoint may speak to provider perceptions among those whose responses were analyzed, but the absent result and analysis details prevent assessment of its magnitude or precision.

The registry description of the primary endpoint population is incomplete. It notes that one intervention provider stopped working for the organization during the intervention and begins to describe two additional providers in an intervention arm, but the description ends before explaining their status. The consequences for endpoint eligibility and missing responses are not reported in the registry record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The registry describes a binary-style primary outcome as a count of participants meeting a stated response definition: intervention primary care providers who reported feeling moderately or very confident. A count can be informative, but interpretation depends on knowing how many people could have contributed and how many actually responded. Those denominator and response details, together with the observed count, are not reported in the registry record.

In general, a confidence interval describes the range of effect estimates that are reasonably compatible with the observed data under a specified statistical model. Its width conveys precision: a wider interval indicates greater uncertainty about the estimate. No confidence interval or primary effect estimate is reported for this endpoint, so its precision cannot be judged. A p-value, if reported, would address compatibility with a specified null hypothesis under a model; it would not by itself describe effect size or practical importance. No p-value is reported in the registry record.

In general, a comparison between groups requires a clearly defined contrast, an outcome measured comparably in each group, and assumptions that support interpreting the difference. This study’s descriptions distinguish waivered providers from non-waivered providers and distinguish non-waivered providers with and without tool access. The overall allocation is listed as non-randomized, while the arm descriptions say non-waivered providers would be randomized. Because the record does not reconcile these statements, the assignment mechanism and resulting basis for a causal comparison are unclear.

Even when assignment is randomized, groups can differ by chance; when assignment is not randomized, pre-existing differences and other factors may also influence comparisons. In general, adjustment can address measured differences under appropriate assumptions, but cannot automatically remove unmeasured confounding. The registry does not report baseline characteristics, adjustment variables, or an adjusted analysis. It therefore does not support a claim that a difference between provider groups, if one were reported elsewhere, would be attributable solely to the tool.

The study describes pre- and post-deployment assessment of case-finding and medication-assisted therapy use, along with a post-deployment comparison of referral patterns. In general, a pre/post comparison uses observations from different time periods and may be affected by changes over time unrelated to an intervention. A comparison group can help contextualize such changes, but interpretation still depends on how groups were formed, whether measurement was consistent, and how time-related factors were handled. The corresponding numerical results and methods are not reported in the registry record.

In general, over-dispersion occurs when outcome variability is greater than a simple statistical model assumes. It can make standard errors too small if the model fails to account for the extra variability, and can affect confidence intervals and tests. The registry record does not report a model or dispersion assessment for the outcomes. No claim about over-dispersion or a model’s adequacy can therefore be made for this study.

The registry’s sample-size planning description specifies a two-sided significance level of 0.05, equal allocation between two arms, a normal approximation, and an effect defined as a stated fraction of the observed effect. It characterizes these figures as a planning illustration rather than the trial’s own design. In general, such a calculation asks how many participants per arm would be needed under chosen assumptions to detect a target effect at a chosen power. It is conditional on those assumptions and is not a report of enrollment, an observed result, or observed power. The record does not report the protocol’s sample-size justification. Since the primary result is not reported, the basis and applicability of any illustration tied to an observed effect cannot be evaluated from the registry record.

For count outcomes, general statistical approaches may include models suited to counts or direct comparisons of counts, depending on the design and data structure. For a binary response, approaches may instead focus on the probability of meeting the response definition. The selected method, any covariate adjustment, the handling of clustering by provider or practice, and the corresponding assumptions are not reported in the registry record. No method should be attributed to this study beyond the design and planning details explicitly documented.

The registry lists five outcome measures and zero analyses. This distinction matters: an outcome measure names something the study intended to assess, while an analysis would specify how observed data were used to estimate or compare results. The presence of an outcome description does not demonstrate that its result was posted, that a particular model was used, or that an inferential test was performed. For this record, the statistical interpretation is limited by the absence of reported results and analysis details.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists five outcome measures and zero analyses. It does not report a statistical plan identifying a primary hypothesis-testing procedure, a hierarchy among outcomes, or an adjustment for multiple comparisons. The record therefore does not establish whether multiplicity was considered in the study’s analysis. In general, testing several outcomes can increase the chance of observing at least one apparently unusual result by chance; methods such as prespecifying a primary comparison or adjusting inference can address particular multiplicity concerns. No such method should be assumed.

The stated secondary objectives include pre- and post-intervention comparisons of opioid use disorder diagnosis in high-risk patients and medication-assisted therapy use, as well as a post-intervention comparison of referral patterns between intervention and control groups. Results for these objectives are not reported in the registry record. It is not possible to determine whether all planned outcomes were measured, analyzed, or reported, or whether analyses differed from an original plan.

Missing-data handling is also not reported. The endpoint population description notes that one intervention provider stopped working for the organization during the intervention and begins to mention two additional intervention providers in an intervention arm, but the description is truncated before stating what happened. It does not identify how many providers completed the confidence measure, whether any responses were missing, or how missing responses were treated.

The record does not specify an intention-to-treat, per-protocol, or other analysis population for the primary endpoint or secondary outcomes. It also does not report exclusions, reasons for exclusion, or sensitivity analyses. In general, the way incomplete responses and eligibility are handled can affect both the estimate and its precision. Without those details and the endpoint results, the direction or extent of any missing-data impact cannot be assessed.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry’s adverse-event groups list zero serious adverse events among 8 waivered providers who received the Opioid Wizard, zero among 24 non-waivered providers who received it, and zero among 23 non-waivered providers who did not receive it. Each group’s listed number at risk matches its listed number for other adverse events; the number of other adverse events is zero in each group. Deaths are listed as zero in all three groups. No serious adverse-event terms are listed.

These entries describe what the registry reports for the listed groups. They do not establish that no unrecorded event occurred, and they do not show that the tool has no safety risks. In general, observing no events in a study does not prove that the underlying event risk is zero; the ability to detect uncommon events depends on the amount and duration of observation and on how events were collected and reported.

The record does not describe the adverse-event collection procedures, the observation period for these entries, or whether all event types were systematically assessed. The group counts should therefore be read as registry entries, not as a complete safety evaluation or a comparison establishing equivalence between groups.

12. What the Result Does, and Does Not, Mean

This trial record documents a pilot intended to assess the usability and acceptability of an opioid use disorder decision support tool in an electronic medical record. Its primary endpoint concerns provider-reported confidence, and its secondary objectives concern case-finding, medication-assisted therapy use, and referral patterns. The record also identifies provider groups with and without tool access and lists the study as completed.

What the registry does not provide is central to interpretation: no observed primary endpoint count, no response denominator, and no analysis results are reported. The secondary outcomes are also not accompanied by numerical results. It is therefore not possible to state how confident providers were, whether the outcomes changed, whether groups differed, or how precise any estimate was. The absence of a posted analysis is not evidence of a null result.

Even if an outcome estimate were available, the design information would require careful interpretation. Allocation is listed as non-randomized, although the descriptions of non-waivered provider arms refer to randomization. Waivered providers were described as receiving the tool, while non-waivered providers were separated into access and no-access groups. These features make the assignment process and comparability of groups important uncertainties. The registry does not report baseline comparisons or adjusted analyses that could clarify them.

The primary endpoint is self-reported confidence, which is relevant to usability but is not a direct measure of provider behavior or patient health. The study’s stated objectives should not be mistaken for results, and a planning illustration should not be read as evidence that a particular effect was observed or that the study had a particular power. The record does not report the statistical analysis plan or sample-size justification.

The most supportable conclusion is narrow: the study was a completed pilot with an intended provider confidence measure and planned clinical process outcomes, but the registry record does not report the outcome analyses needed to quantify findings. It cannot establish whether the tool changed clinical practice or patient outcomes, nor can it establish the absence of an effect.

13. Why This Trial Matters Statistically

This study raises statistical issues common to early clinical decision support pilots. Its central outcome is provider-reported confidence, while its broader objectives concern clinical processes such as diagnosis, medication-assisted therapy, and referral. Separating these constructs matters: a favorable perception of usability would not, by itself, establish a change in clinical behavior or patient outcomes. Conversely, process measures and confidence measures answer different questions and should not be treated as interchangeable.

The arm structure also makes allocation important. The record labels the overall design non-randomized but describes random assignment among non-waivered providers. Waivered providers were assigned tool access in the arm description. These distinctions affect what comparisons can mean and which differences might be attributable to the intervention rather than provider characteristics or other factors. Clear reporting of assignment, eligibility, and analysis populations is especially useful when groups are assembled differently.

Another important issue is the difference between a completed pilot and a reported evaluation. The registry lists enrollment and completion but no analyses. Without the endpoint count, denominators, estimates, and uncertainty measures, readers cannot assess the size or precision of the findings. Naming outcomes and study objectives is not a substitute for reporting results.

The record also illustrates why transparent reporting of missingness and multiplicity matters. One intervention provider is noted to have stopped working for the organization, and the endpoint population description is incomplete. The registry does not explain how those circumstances affected analysis. Multiple stated outcomes create questions about prioritization and inference, but no analysis strategy is documented. These gaps limit reproducibility and make it difficult to distinguish planned assessment from completed analysis.

For biostatistical interpretation, the main contribution of the record is therefore methodological context rather than an estimable effect. It identifies the intended population, intervention, comparisons, and endpoint while leaving key details and results unreported. That distinction helps prevent study aims and planning assumptions from being mistaken for evidence about outcome effects.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record does not report the primary endpoint result, the number of intervention providers who answered the confidence question, or an estimate with a measure of uncertainty. It lists five outcome measures but zero analyses, and provides no numerical secondary outcome comparisons. This prevents assessment of effect magnitude, precision, or statistical evidence for any stated objective.

The allocation information is internally unclear: the overall design is listed as non-randomized, while descriptions of the non-waivered arms say providers would be randomized. The record does not explain this discrepancy or report how assignment was implemented. Because waivered providers were described as receiving the intervention, the provider groups may differ in ways relevant to interpretation; baseline characteristics and adjustment methods are not reported.

The primary endpoint population description is incomplete. It notes one intervention provider stopped working for the organization and begins to describe two additional intervention providers, but does not complete the explanation. The registry does not report response rates, exclusions, missing-data methods, or analysis populations.

Further limitations include the absence of a reported protocol-level analysis plan, sample-size justification, multiplicity strategy, endpoint instrument details, and adverse-event collection procedures. The pilot is framed around provider usability and acceptability, and its confidence endpoint is self-reported. These features limit what the measure could establish even if its result were available. The registry record does not report patient-level outcomes demonstrating clinical benefit or harm.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Clinical Decision Support for Opioid Use Disorders in Medical Settings: Usability Testing in an EMR — NCT03559179.
• Registry record last updated 2020-03-17; status last verified 2019-02.
• Results first posted 2020-03-10.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT03559179 is a completed pilot usability study of the Opioid Wizard, an opioid use disorder clinical decision support tool for an electronic medical record. The registry lists non-randomized allocation, no masking, a parallel model, enrollment of 55, and phase NA. The study began on 2018-04-30 and reached primary completion on 2019-02-14.

The primary endpoint was the count of intervention primary care providers reporting moderate or very high confidence in assessing and treating opioid use disorder, measured at approximately month 10. Planned secondary assessments concerned diagnosis in high-risk patients, medication-assisted therapy use, and referral patterns. The registry lists five outcome measures and zero analyses.

No primary or secondary results are reported in the registry record. The assignment description is inconsistent about randomization, and the endpoint population description is incomplete. The record therefore documents the pilot’s aims and design but does not permit estimation of outcome effects or their precision.