Evaluation of a Computerized Opioid Overdose Prevention Program: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Evaluation of a Computerized Opioid Overdose Prevention Program (NCT02040077), a randomized na study. The primary outcome is Knowledge Gain.

2. Clinical Question

These descriptions define the comparison the study set out to make, not a conclusion about which format produced more learning. The record also lists whether participants would recommend the intervention to a family member or friend as a secondary outcome. Its results are not reported in the registry record.

3. Trial Design

The registry record identifies NCT02040077 as a completed, randomized, single-masked, parallel-group study sponsored by Johns Hopkins University. The study title is “Evaluation of a Computerized Opioid Overdose Prevention Program,” and its official title is “Development and Validation of a Computerized Opioid Overdose Intervention.” The phase is recorded as NA. Enrollment is listed as 75. The recorded study start is 2015-01, and primary completion is recorded as 2015-09.

Participants were assigned to one of three behavioral-intervention conditions: Computer + Fluency, Computer Only, or Treatment as Usual. The first condition included computer-based education with periodic mastery demonstrations required before proceeding through modules. The second also used the computerized intervention but did not require mastery demonstrations. The treatment-as-usual condition used publicly available pamphlets with the same information and did not include computer access or mastery practice.

Randomized allocation indicates that assignment was by chance, but the registry record does not describe the allocation sequence, allocation concealment, or the procedures used to implement assignment. It records single masking without stating which people involved in the study were masked or how masking was maintained. Further details about the study setting, recruitment, eligibility, and participant characteristics are not reported in the registry record.

A parallel-group design compares groups assigned to different conditions. The registry record’s three arms therefore represent distinct intervention experiences rather than a crossover in which each participant receives multiple conditions. The stated primary assessment was a knowledge test before and immediately after intervention. The timeframe describes the post-test as immediately after the intervention, averaging 10 minutes. No longer-term primary outcome assessment is described in the registry record.

The record lists two outcome measures and no reported statistical analyses. The primary outcome is knowledge gain; the secondary outcome concerns whether participants would recommend the intervention to a family member or friend. Although study status is listed as completed, completion alone does not supply the numerical outcome results or establish what the comparisons found.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Although the study expected to enroll and randomize 75 participants, an additional participant was randomized into the study - resulting in a final overall sample of 76 randomized participants.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Knowledge Gain, measured over Before the intervention (pre-test) and immediately after the intervention (post-test), an average of 10 minutes..

The primary endpoint is change from baseline on a knowledge test administered immediately before and after the intervention. It is intended to capture whether participants answered more test items correctly after receiving the educational material. The registry record says the test was developed for this study because validated scales for general overdose information in a true/false format were not available at the time.

The test included 51 items, each answered as “true,” “false,” or “I don't know.” Including “I don't know” was intended to discourage random guessing from accidentally producing correct answers. The test summed correct responses into one measure, with a range of 0-51 and no subscales. Higher scores indicate more correct responses. The test included information covered by the intervention, making the outcome directly relevant to the educational content being taught.

In general, a pre-to-post change score summarizes how a participant’s measured score differs after an intervention from their score at baseline. Comparing changes across randomized groups can address whether the groups’ measured learning differed over the study period. The registry record does not report the observed scores, change estimates, variability, or between-group comparisons. It is therefore not possible to describe the amount or precision of any knowledge change from the posted results.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The central statistical task in this study would be to compare knowledge-test performance between randomized groups. The outcome is continuous: each test score is a count of correct responses on a scale whose stated range is 0-51. The registry record identifies the outcome summary as a mean with standard deviation. A mean describes the average score or change in score, while a standard deviation describes how dispersed individual scores are around that average. They are complementary: an average alone does not show whether participant scores were tightly clustered or widely spread.

What does a change-from-baseline outcome represent?

A change score compares a participant’s post-intervention result with their own pre-intervention result. In general, comparing group-level changes can account for the fact that participants may begin with different baseline knowledge. Another possible analytical approach is to model the post-intervention score while adjusting for baseline score. These approaches have different assumptions and can produce different estimates. The registry record does not report which analysis was planned or performed, so neither approach should be attributed to this trial.

The stated timeframe is before the intervention and immediately afterward, with the post-test averaging 10 minutes. This close timing focuses the endpoint on immediate performance after exposure to the educational content. It does not establish whether knowledge is retained later. The registry record does not describe a later primary assessment.

How should a group comparison be understood?

With three randomized conditions, the study question could involve comparisons between the mastery-based computer intervention and the computer-only condition, between each computer condition and pamphlets, or an overall comparison among all conditions. In general, each choice answers a different question. A direct contrast between the two computer conditions focuses on the addition of mastery practice, while comparisons with pamphlets address the computerized formats relative to the stated treatment-as-usual materials. The registry record does not specify which contrasts were prespecified or report any estimates.

An estimate should be read with its uncertainty. In general, a confidence interval indicates a range of values compatible with the observed data under the analysis assumptions. A narrow interval indicates greater precision than a wide interval, while an interval spanning values that would support materially different interpretations leaves the comparison uncertain. A p-value is not a measure of effect size or practical importance. Neither confidence intervals nor p-values for the primary outcome are reported in the registry record.

What can the scale and its distribution tell us?

The test sums correct answers and has no subscales. Higher values mean more correct responses on these items. The “I don't know” option was included to discourage accidental correct answers from guessing. As a general measurement principle, a test score reflects performance on the items included; it does not automatically establish competence in every related situation. The registry record describes the content and scoring but does not report validation analyses or the distribution of participants’ scores.

Means and standard deviations are common summaries for continuous outcomes. In general, a mean can be sensitive to unusually high or low values, and a standard deviation does not reveal the shape of the score distribution. If a measure is bounded or scores cluster near an endpoint, the distribution may not be well represented by a simple symmetric model. The record does not provide observed score distributions or enough analysis detail to evaluate this issue in the trial.

What do the planning assumptions mean?

The registry record describes a planning illustration based on the observed effect, with a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. It also describes the target effect as a stated fraction of the observed effect. These are assumptions for a planning calculation, not a description of the trial’s actual design or an observed measure of power. The registry record does not report the numerical effect used, the fractions illustrated, or the resulting sample-size values. The illustration therefore cannot be independently interpreted from the recorded details alone.

In general, sample-size planning asks how many participants would be needed under particular assumptions about the effect to detect, outcome variability, significance level, and desired power. A planning calculation is conditional on those assumptions and is not evidence that an enrolled trial had a particular probability of detecting an effect. The record’s planning assumptions specifically describe equal allocation between two arms, whereas the registered study lists three intervention conditions. The registry record does not explain how the illustration relates to the study’s three-arm enrollment or design.

Over-dispersion is a general concern when count data vary more than a chosen statistical model expects. When present, a model that assumes less variability can understate uncertainty and make results appear more precise than warranted. The test score is a count of correct responses, but the registry record does not report an over-dispersion assessment or a model for the score. No particular model should therefore be inferred.

What analysis details are unavailable?

The registry record reports zero statistical analyses. It does not report the analysis population, treatment of missing scores, model specification, covariates, multiplicity procedure, or handling of protocol deviations. These details affect how an estimate should be interpreted and how closely it reflects the randomized comparison. The lack of those details does not establish that no analysis occurred outside the registry; it means the analysis is not reported in the registry record.

10. Multiplicity, Missing Data and Analysis Populations

The study has a primary knowledge outcome and a secondary recommendation outcome, and it includes three intervention conditions. The registry record does not describe which between-group comparisons were prespecified, how many comparisons were intended, or whether any adjustment for multiple comparisons was planned. When several outcomes or contrasts are examined, the chance of obtaining at least one apparently notable result can increase. In general, multiplicity procedures are used to control that issue, but the method depends on the questions and analysis plan. No such method is reported in the registry record.

Missing outcome data can affect both the number of participants contributing information and the validity of a comparison. In general, an analysis needs to state how missing measurements were handled and whether assumptions about why data are missing are plausible. The registry record does not report the number of missing primary outcome measurements, reasons for missingness, imputation methods, or sensitivity analyses. It also does not report an analysis population, such as whether participants were analyzed according to randomized assignment or according to intervention received.

The registry record’s enrollment is 75, and the safety record separately lists participants at risk in each group. Those entries do not establish how many participants contributed paired pre- and post-intervention scores or how the primary analysis handled incomplete records. Statistical analyses are not reported in the registry record, so the impact of missing data, multiplicity, and analysis-population choices cannot be assessed from it.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry record lists serious adverse events, other adverse events, and deaths by group. For Computer + Mastery, it records 27 participants at risk for serious and other adverse events, with 0 affected in each category and 0 deaths. For Computer Only, it records 24 at risk for each adverse-event category, with 0 affected and 0 deaths. For Pamphlet, it records 25 at risk, with 0 serious adverse events, 0 other adverse events, and 0 deaths.

These entries indicate that no events were recorded in the listed categories for the participants counted at risk in each group. They do not establish that an event was impossible, nor do zero recorded events quantify the risk with high precision. In general, safety counts should be interpreted alongside the number followed, the duration and completeness of follow-up, and how events were defined and collected. The registry record does not report further details about safety monitoring or the event-collection period. The recorded safety group labels are Computer + Mastery, Computer Only, and Pamphlet; the intervention descriptions use Computer + Fluency and Treatment as Usual for the corresponding study conditions.

12. What the Result Does, and Does Not, Mean

The trial addresses a statistical question about immediate knowledge performance after different forms of opioid overdose education. Its randomized, parallel-group design places computerized education with mastery practice, computerized education without mastery practice, and pamphlet-based treatment as usual in direct comparison. The primary outcome is a study-developed score based on correct answers to 51 items. These features define what the study intended to evaluate.

They do not reveal the comparative result. No primary outcome scores, change estimates, confidence intervals, or statistical tests are reported in the registry record. Consequently, the posted record cannot establish whether scores changed, whether changes differed between conditions, or how uncertain any difference would be. Completion status and enrollment do not substitute for outcome results.

In general, immediate performance on a knowledge test is distinct from retained knowledge, behavior, or health outcomes. A higher score, if observed, would indicate more correct answers on the test as scored; it would not by itself show that participants later applied the information or that overdose outcomes changed. The registry record describes an immediate post-test but does not report sustained knowledge or clinical outcomes.

The study-developed scale is relevant to the intervention because its answers were included in the educational content. Its results would therefore need interpretation in light of the test’s construction and the absence of reported validation analyses. The recommendation outcome measures stated willingness to recommend the intervention, not necessarily actual recommendations or effectiveness.

The available safety entries record zero serious adverse events, zero other adverse events, and zero deaths in each listed group. Those data are descriptive and do not replace the missing primary efficacy results. Overall, the registry record establishes the study design, educational conditions, outcome definition, and recorded safety counts. It does not provide enough statistical evidence to draw a comparative conclusion about knowledge gain or the broader consequences of the intervention.

13. Why This Trial Matters Statistically

This trial is statistically notable for separating two components of computerized education: access to the computer-based material and a requirement to demonstrate mastery before advancing. The pamphlet condition provides a further comparison using publicly available materials described as containing the same information. That structure makes the study question more specific than a simple comparison of education with no education: it can distinguish the delivery format and the added mastery requirement, provided those contrasts are defined and analyzed.

The primary outcome also illustrates a measurement choice common in educational research. The investigators developed a test because a validated general overdose knowledge scale in the intended true/false format was not available at the time. The test’s “I don't know” response was intended to discourage random guessing, and the intervention included the answers to all items. Such design decisions link the measure closely to the teaching content, while making the scale’s interpretation dependent on the quality and scope of its items.

At the same time, the registry record’s statistical value is constrained by the absence of reported outcome results and analysis details. Without estimates and uncertainty, readers cannot assess the magnitude or precision of group differences. Without a reported comparison plan, they cannot tell which contrasts were prioritized or how multiplicity was addressed. The record therefore illustrates why a registration entry is useful for understanding the intended design and outcome, but why posted results and methods are essential for evaluating evidence.

The safety record lists no adverse events or deaths in the recorded groups, but the primary knowledge findings remain unavailable. The study is therefore informative about how the question was framed and measured, not about the answer to that question.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The population description is blank, and the registry record does not report eligibility criteria, participant characteristics, or recruitment details. This limits assessment of who the findings would represent, even if outcome results were available. The total enrollment is recorded as 75, but the record does not explain how that enrollment was distributed for the primary analysis or how many participants contributed complete outcome measurements.

The endpoint uses a scale developed for the study. The description explains its items and scoring, but the registry record does not report psychometric evaluation, reliability, validation, or score distributions. The immediate post-intervention assessment is useful for measuring near-term test performance, but no sustained primary assessment is described. Test performance also does not directly establish later behavior or overdose outcomes.

Important design and analysis details are not reported in the registry record. These include which party was masked, how random assignment was generated or concealed, prespecified contrasts, the statistical model, the analysis population, handling of missing data, and any multiplicity strategy. The record reports no statistical analyses and no primary outcome estimates. As a result, neither the comparative findings nor their precision can be evaluated.

The study lists three conditions, while the planning illustration is described using equal allocation between two arms. The record does not explain how that illustration relates to the trial design, and it does not report the numerical planning inputs or outputs. Finally, the safety record provides event counts but not additional information about collection procedures or follow-up. These limitations describe what is unavailable in the registry record; they do not establish that the information was absent from all study documents.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Evaluation of a Computerized Opioid Overdose Prevention Program — NCT02040077.
• Registry record last updated 2017-11-17; status last verified 2017-10.
• Results first posted 2017-09-26.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT02040077 is a completed, randomized, single-masked, parallel-group study sponsored by Johns Hopkins University. The registry record lists enrollment of 75 and records a start of 2015-01 and primary completion of 2015-09. The three behavioral conditions were Computer + Fluency, Computer Only, and Treatment as Usual.

The primary outcome was change from baseline on a study-developed knowledge test administered immediately before and after intervention. The test included 51 true, false, or “I don't know” items, summed as correct responses on a 0-51 scale; higher values indicated more correct answers. A recommendation outcome was listed as secondary.

The registry record does not report primary outcome results, statistical comparisons, or analysis methods. Its safety entries record zero serious adverse events, zero other adverse events, and zero deaths in each listed group. The available record therefore describes the study design and measures but does not establish the comparative knowledge findings.