← Clinical Trials
Opioid Abuse, Unspecified NA Continuous Endpoint NCT04477304

Application of Economics & Social Psychology to Improve Opioid Prescribing Safety Trial 1: EHR Nudges: Complete Statistical Analysis

This analysis examines a randomized clinic-level trial of electronic health record nudges intended to discourage unnecessary opioid prescribing. It focuses on the reported change in clinicians’ weekly log total MME for opioid-naïve and at-risk patients, alongside the posted secondary outcome.

Trial start: 2020-12-01  ·  Enrollment: 555  ·  Sponsor: University of Southern California
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for Application of Economics & Social Psychology to Improve Opioid Prescribing Safety Trial 1: EHR Nudges (NCT04477304), a randomized na study. The primary outcome is Change in Average Per-clinician Log Total Weekly MME Between Baseline and Intervention for Opioid Naïve and At-Risk Patients (Below 50 MME Daily Dose Rxs).

555
Enrollment
NA
2
Arms
In the primary outcome
-0.09
Mean difference
Behavioral Intervention Arm
-0.09–-0.09
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT04477304
PhaseNA
ConditionOpioid Abuse, Unspecified
AllocationRandomized
Design modelParallel
MaskingNone (open label)
Enrollment555
Arms in the primary outcome2
Lead sponsorUniversity of Southern California
Results postedYes

2. Clinical Question

Population

Clinicians enrolled at AltaMed and Northwestern University who prescribed an opioid prescription below 50 MME during the study period were included in the primary endpoint description. The endpoint also excludes specified prescriptions and patient groups, including historical prescriptions, suppositories or injectionables, prescriptions to patients diagnosed with cancer, and prescriptions to patients under 18 years. The complete analysis-population details are not reported in the registry record.

Intervention

Clinicians in clinics randomized to the behavioral intervention received an EHR nudge when a patient’s prescribing history placed them in one of three categories: opioid naïve, at-risk for long-term use, or a long-term opioid recipient. The stated intervention description says the nudges include elements, but does not report those elements in full.

Comparator

Clinicians in clinics randomized to control received web-based guideline education, consisting of an online educational module at the start of the study period. The record describes the module as including clinical content related to CDC guidelines and the Oregon Pain Guidance document. Further details of the control content are not reported in the registry record.

Primary question

Over the 18-month intervention, did the average per-clinician log total weekly MME for opioid-naïve and at-risk patients change differently from the 6-month baseline in clinics assigned to EHR nudges compared with clinics assigned to guideline education? The posted primary result is a comparison of change between the randomized groups, not a direct estimate of an individual patient’s outcome.

3. Trial Design

This completed study, NCT04477304, is a randomized, parallel-group trial with no masking. The registry lists enrollment of 555 and identifies the sponsor as the University of Southern California. The study began on 2020-12-01, and primary completion was 2023-08-06. The listed phase is NA. Clinics, rather than individual clinicians, were randomized to the behavioral intervention or control condition.

The intervention applied behavioral insights through electronic health record prompts. Clinicians in intervention clinics were to receive a nudge when the patient’s prescribing history matched one of three categories: opioid naïve, at-risk for long-term use, or long-term opioid recipient. The control group received an online educational module at the beginning of the study period, with clinical content related to guidelines named in the registry description. The prompt elements and full educational-module content are not reported in the registry record.

The primary endpoint compares a 6-month baseline with an 18-month intervention period. Its population is described as clinicians enrolled at AltaMed and Northwestern University who prescribed an opioid prescription below 50 MME during the study period, subject to listed exclusions. The arm counts for the primary endpoint are 275 for the behavioral intervention and 207 for control. These are the endpoint analysis counts, not a complete account of how all enrolled participants were handled. The registry record does not report the original sample-size justification, the number of clinics, details of the allocation procedure, or an interim-analysis plan.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

StageBehavioral Intervention ArmControl
STARTED322233
COMPLETED275207
NOT COMPLETED4726

Reasons for not completing this period:

ReasonBehavioral Intervention ArmControl
No valid opioid prescriptions during study timeframe4726

Analysis population for the primary outcome, as defined in the registry: Clinicians enrolled at AltaMed and Northwestern University who prescribed an opioid Rx \< 50 MME during study period. Historical Rxs, suppositories/injectionables, control clinicians who erroneously received alerts, and Rxs prescribed to patients diagnosed with cancer, or who were under 18 years, were excluded.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

CharacteristicUnitBehavioral Intervention ArmControlTotal
Participants analysed322233555
Age, Customized: Unknown/Not reportedyears322233555
Sex/Gender, Customized: FemaleParticipants199135334
Sex/Gender, Customized: MaleParticipants12192213
Sex/Gender, Customized: Unknown/Not reportedParticipants268
Race and Ethnicity Not CollectedParticipants——0
Region of Enrollment: United StatesParticipants322233555

6. Primary Endpoint

The registry lists the primary outcome as Change in Average Per-clinician Log Total Weekly MME Between Baseline and Intervention for Opioid Naïve and At-Risk Patients (Below 50 MME Daily Dose Rxs), measured over 6-month baseline and 18-month intervention.

ElementRegistry entry
Result typeMean
Measure of spreadStandard Deviation
Unitchg in avg per-clin log total weekly MME
Endpoint classificationContinuous Endpoint

The primary endpoint is the change in average per-clinician log total weekly MME between baseline and intervention for opioid-naïve and at-risk patients with prescriptions below 50 MME daily dose. For each clinician, the measure is based on the weekly total MME and then expressed on the natural-log scale. The registry defines the change as the intervention-period average minus the baseline average. The 6-month baseline and 18-month intervention periods therefore contribute to a comparison of prescribing patterns over time.

MME is a standardized measure based on prescription strength, number of pills, and an opioid conversion factor. The endpoint uses the log of total MME because totals may vary widely; taking a logarithm is a way of representing such measurements on a compressed scale. In general, a change on a log scale is not the same as a change in raw MME. The posted endpoint is an average clinician-level measure, so it describes a group-level prescribing pattern rather than an effect for each patient.

This measure is used to evaluate whether prescribing patterns changed differently between clinics assigned to EHR nudges and clinics assigned to guideline education. It does not directly measure whether a prescription was clinically necessary, nor does the registry endpoint establish a patient’s health outcome. The precise rationale for selecting this endpoint, beyond the stated trial objective, is not reported in the registry record.

7. Results Posted in the Registry

Mean difference: Behavioral Intervention Arm

-0.09

95% CI: -0.09 to -0.09

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals-0.2-0.1-0.1-0.1-0.1-0.0Behavioral Intervention Arm (n=275)Control (n=207)chg in avg per-clin log total weekly MME (95% CI)
Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)Behavioral Intervention Arm-0.10-0.08-0.06-0.04-0.010.01mean difference vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
Change in Average Per-clinician Weekly Proportion of Chronic Patients Prescribed a High Dose Opioid (=> 50 MME Daily Dose Rxs) Between Baseline and InterventionContinuous EndpointMean difference-0.08 (-0.08 to -0.08)1 per arm

Change in Average Per-clinician Weekly Proportion of Chronic Patients Prescribed a High Dose Opioid (=> 50 MME Daily Dose Rxs) Between Baseline and Intervention

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)Behavioral Intervention Arm-0.09-0.07-0.05-0.03-0.010.01mean difference vs control (95% CI)

The registry posts one secondary result: change in the average per-clinician weekly proportion of chronic patients prescribed a high-dose opioid. The direction of its reported group contrast is consistent with the primary endpoint’s pattern. This offers a related view of prescribing, but it concerns a different measure and should not be read as a direct replication of the primary result.

Only one secondary endpoint is posted, while the record lists six outcome measures and six analyses overall. The other outcome results are not reported in the registry record. The posted secondary analysis uses a mixed-model method and reports a statistically significant result, but the record does not state whether secondary outcomes were prespecified in a hierarchy or how the family of analyses was handled.

Unadjusted secondary results generally carry less evidential weight when multiple outcomes or analyses may have been considered. In general, testing more outcomes creates more opportunities for chance findings, unless the analysis plan accounts for that multiplicity. The registry does not report a multiplicity adjustment or a strategy for interpreting secondary endpoints alongside the primary endpoint. Accordingly, the secondary result is best treated as supportive context rather than as an independent, definitive confirmation.

The primary and secondary outcomes also differ in their populations and measurement definitions. The primary outcome uses log total weekly MME for opioid-naïve and at-risk patients below 50 MME daily dose; the secondary concerns the weekly proportion of chronic patients prescribed at least 50 MME daily dose. Because these are distinct prescribing measures, agreement in direction does not imply that the same mechanism or patient-level change explains both results.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

The posted primary analysis reports a net mean difference of -0.09 for the change in average per-clinician log total weekly MME, with a 95% confidence interval from -0.12 to -0.07 and p < 0.001. The group mean changes were -0.14 in the behavioral intervention arm and -0.05 in control. Because the endpoint is defined as intervention average minus baseline average, each negative arm value represents a lower average on the log-scale measure during intervention than at baseline. The net comparison indicates a more negative change in the intervention arm than in control.

This is a comparison of changes across randomized groups, not simply a comparison of intervention-period values. It describes the outcome measure and population specified in the registry; it should not be interpreted as a direct estimate of effects for every clinician or patient. Nor does a difference in this prescription measure, by itself, establish whether prescribing was appropriate or what patient consequences followed.

The sponsor’s reported analysis uses mixed models and labels the result a mean difference (net). Its estimate and significance result are consistent in direction with the posted group means. The registry’s reported confidence interval conveys uncertainty around the sponsor’s estimate, while the p-value indicates that the observed result is not readily compatible with a null comparison under the assumptions of that analysis. Neither statistic answers questions beyond the endpoint and analysis population.

A simple unadjusted recalculation from the arm summaries also gives a mean difference of -0.09 and an unadjusted p-value of <0.001. Such a calculation does not reproduce the full mixed-model analysis, which uses a different analytical framework. Details of the model specification, including its covariates and treatment of clinic-level structure, are not reported in the registry record. The recalculated result is therefore a check based on the posted summary statistics, not a substitute for the sponsor’s analysis.

The trial also posts a secondary outcome concerning the weekly proportion of chronic patients prescribed a high-dose opioid. Its reported contrast is in the same general direction as the primary result. That parallel pattern can provide context, but the secondary result is not interchangeable with the primary endpoint: it measures a different prescribing feature and carries the limitations of an unadjusted secondary analysis.

10. Statistical Methods Explained

The primary endpoint is continuous: it is a numerical measure that can take a range of values rather than placing each observation into a category. The registry reports arm means and standard deviations, and the sponsor reports a mean difference. In general, a mean describes the average value in a group, while a standard deviation describes the spread of individual values around that average. Here, the endpoint is already a clinician-level average on a log scale, so neither the mean nor the standard deviation should be read as a patient-level MME result.

What does the change score represent?

The registry defines change as the 18-month intervention average minus the 6-month baseline average. A negative change therefore corresponds to a lower value during the intervention period than during baseline for the measure as defined. The comparison of interest is the difference between the intervention and control groups’ changes. This structure helps distinguish a difference in change from a simple difference between groups at one time point. It does not, on its own, reveal whether other time-varying influences affected the groups.

Why use a logarithm?

The endpoint uses the natural logarithm of total weekly MME. In general, logarithms compress a scale, so large values are represented less extremely than they would be on the original scale. This can be useful when measurements are unevenly distributed or span a broad range. It also changes interpretation: a difference on the log scale is not a raw-MME difference, and translating it back to a ratio or percentage would require a calculation not reported in the registry record. The result should therefore be described in the endpoint’s stated units and scale.

What does a mean difference tell us?

The mean difference is the net contrast between the two groups’ average changes. A negative estimate means that the intervention group’s change is more negative than the control group’s change under the stated contrast. It does not say that every clinician changed in that direction. Individual clinicians may have values above or below the group average, and the posted group summaries do not show that distribution in detail.

How should the p-value be read?

In general, a p-value measures how incompatible the observed data are with a specified null hypothesis under the statistical model. A small p-value is not the probability that the null hypothesis is true, and it does not quantify the size or practical importance of an effect. The reported p-value is <0.001 for the primary analysis. Its interpretation depends on the analysis assumptions and on how the endpoint and analysis were selected. The registry record does not report a complete statistical analysis plan or a multiplicity adjustment strategy.

What does the confidence interval add?

A confidence interval gives a range of values compatible with the estimate under the method used to construct it. In general, a narrower interval indicates greater statistical precision, while a wider interval indicates more uncertainty about the size of the contrast. The sponsor reports a 95% confidence interval for the primary net mean difference. The interval does not include every possible source of uncertainty, such as uncertainty from model choices or limitations in how the analysis population is defined.

How do the analyses differ?

The registry reports a mixed-model analysis by the sponsor. Mixed models can account for structured data, such as observations grouped within clinics or repeated measurements, when those features are represented in the model. The registry names the method but does not report the model formula, the included covariates, the covariance structure, or the handling of missing observations. Those details matter for understanding exactly how the reported standard error and interval were obtained.

The simple recalculation uses posted group-level summaries to estimate an unadjusted difference and its uncertainty. It is not a reconstruction of the mixed model and cannot account for features not contained in those summaries. The sponsor’s reported analysis is the principal reported inferential result; the recalculation is supplementary context. The posted recalculation and sponsor estimate are similar when rounded, while their interval widths differ. The registry does not provide enough model detail to attribute that difference to a specific modeling choice.

Why can over-dispersion matter?

In general, over-dispersion occurs when observed variability is greater than a model expects. If a method assumes less variability than the data contain, its standard errors may be too small and its confidence intervals too narrow. If it accounts for additional variability, uncertainty may be larger. The registry record does not report an over-dispersion assessment for this endpoint, and the primary continuous outcome should not be treated as a count outcome merely because it is based on prescriptions.

Finally, randomization supports comparison of assigned groups, but interpretation still depends on the actual allocation unit, implementation, measured population, and analysis. The registry states that clinics were randomized and that control clinicians who erroneously received alerts were excluded from the primary endpoint population. The record does not provide enough detail to assess all consequences of that exclusion or to determine whether the analysis followed an intention-to-treat principle.

11. Confidence Intervals and Precision

The sponsor’s primary result is a net mean difference of -0.09, with a 95% confidence interval from -0.12 to -0.07. The interval lies on the negative side of zero, and the reported p-value is <0.001. Together, these statistics indicate that the sponsor’s model estimated a negative difference in change and that the estimate is relatively precise within the assumptions of that analysis. Precision is not the same as certainty: the interval describes sampling uncertainty under the fitted method, not every possible source of uncertainty.

In general, a confidence interval gives a range of values that are compatible with the data and model at the stated confidence level. It does not mean that there is a 95% probability that the particular interval contains the true value. The interval’s practical meaning also depends on the scale. Here, the endpoint is a change in a log-transformed clinician-level weekly measure, so the values should not be read as raw MME differences.

The simple unadjusted recalculation reports the same rounded point estimate, -0.09, but its interval is narrower than the sponsor’s reported interval. That difference is a reminder that estimates and uncertainty can depend on the analysis method, not just the arm means. The registry identifies the sponsor method as mixed models but does not report enough detail to explain the interval-width difference. The sponsor’s interval is therefore the reported model-based uncertainty for the trial analysis.

Neither the interval nor the small p-value resolves limitations related to the selected endpoint, the analysis population, missing-data handling, or unreported modeling decisions. The interval quantifies precision for the statistical contrast under the analysis used; it does not establish clinical importance or describe the range of individual clinician responses.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect11
75% of the observed effect11
50% of the observed effect12
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect1240%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, two-sample comparison of means with the pooled standard deviation. Calculated for the comparison: Behavioral Intervention Arm.

The planning figures are participants per arm, not total event counts. They are illustrations calculated from the observed effect, not the trial’s original design calculations. The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation; the effect to detect is the stated fraction of the observed effect. The original sample-size justification and design assumptions are not reported in the registry record.

The 80% power column describes the number of participants per arm that the planning calculation estimates would be needed to detect the selected effect under those assumptions with 80% power. The 90% power column gives the corresponding planning number for 90% power. In general, power is the chance of detecting an effect of a specified size if that effect is present and the assumptions hold; it is not a guarantee that a study will produce a statistically significant result.

For the primary contrast, the listed figures are 1 participant per arm for 80% power and 1 per arm for 90% power when planning for 100% of the observed effect. They remain 1 per arm for both power levels when planning for 75% of the observed effect. When planning for 50% of the observed effect, the listed figures are 1 per arm for 80% power and 2 per arm for 90% power. These unusually small illustrations follow mechanically from the observed estimate and posted summary variability; they should not be mistaken for a realistic recommendation or for the actual trial design.

Smaller effects are generally harder to distinguish from random variation and therefore require more participants under otherwise comparable assumptions. The figures are not observed power, and they do not describe the probability that the completed trial’s result is correct. Their interpretation is limited by the simplified assumptions and by the fact that the trial’s own design justification is not reported in the registry record.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = 2 (z1−α/2 + z1−β)2 σ2 / Δ2

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

  1. Observed difference Δ = -0.090; pooled standard deviation σ = 0.010.
  2. n = 2 × (1.96 + 0.842)2 × 0.0102 / -0.0902 = 0.2, rounded up to 1 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.09                      # observed difference in means
sigma <- 0.01                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(4477304)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 1 (80% power), 1 (90% power)
 75% of observed effect: n per arm = 1 (80% power), 1 (90% power)
 50% of observed effect: n per arm = 1 (80% power), 2 (90% power)
power.t.test: 1.7 
Simulated power at n = 1 per arm: NA (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

Very small required sample. The formula gives a very small requirement because the observed effect is very large relative to the variability. Normal-approximation formulas are unreliable at such sizes, and the simulation above shows the power actually achieved. An exact or simulation-based design would be needed in practice.

14. Analyses Reported by the Sponsor

MethodEstimate95% CIP-value
Mixed Models AnalysisMean Difference (Net) -0.09-0.12 to -0.07< 0.001

The sponsor reports a Mixed Models Analysis with a primary mean difference (net) of -0.09, a 95% confidence interval from -0.12 to -0.07, and p < 0.001. This is the trial’s reported inferential analysis. It compares the groups’ changes using the stated mixed-model method rather than relying only on the arm-level summary statistics.

The simple recalculation from the posted summaries also gives a rounded mean difference of -0.09 and p < 0.001. However, its confidence interval is narrower than the sponsor’s interval. The two analyses should not be treated as identical: a summary-statistic recalculation cannot reproduce model features that are not present in the arm summaries. The registry does not report the mixed model’s formula, covariates, or other details needed to explain the interval difference.

The sponsor’s estimate is not contradicted by the recalculated point estimate when both are rounded to the reported precision. The sponsor’s confidence interval and method remain the appropriate description of the analysis reported by the trial. The registry record does not report a discrepancy flag for the primary result.

15. Multiplicity, Missing Data and Analysis Populations

The registry identifies a primary endpoint and lists six outcome measures and six analyses, but only one secondary endpoint result is posted. The record does not report a complete statistical analysis plan, a prespecified hierarchy among outcomes, or a correction for multiple comparisons. Without those details, it is not possible to determine how the reported secondary p-value was intended to be interpreted in relation to all planned analyses.

In general, multiplicity arises when several hypotheses are tested. As the number of tests grows, the chance of at least one small p-value arising by chance can increase. Methods such as adjustment procedures or a prespecified testing sequence can address that concern, but the registry record does not say whether any such approach was used. The primary analysis is the central result described in the record; secondary findings warrant more caution where multiplicity handling is unknown.

The record does not report how missing endpoint observations were handled, how much data were missing, or whether missingness differed between groups. It also does not report imputation methods or sensitivity analyses for missing data. Those omissions limit assessment of whether the analysis could be influenced by incomplete outcome information.

The endpoint population description includes clinicians enrolled at AltaMed and Northwestern University who prescribed an opioid prescription below 50 MME during the study period. It also lists exclusions, including historical prescriptions, suppositories or injectionables, control clinicians who erroneously received alerts, and prescriptions for patients diagnosed with cancer or under 18 years. The full analysis-population specification and the consequences of these exclusions are not reported in the registry record. It is also not reported whether the primary analysis followed an intention-to-treat approach, whether all randomized clinics were retained according to assignment, or how departures from assigned intervention were handled.

These missing details do not negate the posted result, but they constrain how fully its robustness and generalizability can be assessed from the registry record alone.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
Behavioral Intervention Arm03220.0%
Control02330.0%

Deaths

ArmDeaths (participants affected of at risk)
Behavioral Intervention Arm0 of 322
Control0 of 233

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

ArmParticipants affected of at risk
Behavioral Intervention Arm0 of 322
Control0 of 233
Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The registry reports serious adverse events and other adverse events separately for the behavioral intervention and control groups. For serious adverse events, 322 were at risk in the intervention group and 233 in control; zero were reported as affected in either group. For other adverse events, the corresponding at-risk counts were 322 and 233, with zero affected in each group. The record also reports zero deaths in both groups, and no top serious-event terms are listed.

These figures describe what was posted in the registry record. They do not establish that no adverse events occurred outside the categories, periods, or reporting procedures captured by the record. The registry does not report event definitions, ascertainment procedures, follow-up completeness, or further detail about safety reporting. The at-risk counts are the denominators shown for these safety entries and should not be assumed to represent every possible analysis population. These results are not a comparison of prescribing outcomes and should be interpreted separately from the primary endpoint.

17. What the Result Does, and Does Not, Mean

The primary result describes a more negative change in the intervention group than in control for the average per-clinician log total weekly MME measure among the endpoint population. The sponsor reports a net mean difference of -0.09, with a 95% confidence interval from -0.12 to -0.07 and p < 0.001. The result is evidence of a difference between the assigned groups on this specific outcome as analyzed, not evidence that every clinician’s prescribing changed in the same way.

The endpoint is an average clinician-level measure, expressed on a natural-log scale, for opioid-naïve and at-risk patients with prescriptions below 50 MME daily dose. It is not a direct count of prescriptions, a raw-MME difference, or a measure of patient health. The result does not establish whether a particular prescription was necessary, whether an individual patient’s care improved or worsened, or what the effect would be in settings outside the participating clinics.

The trial compares an EHR nudge strategy with web-based guideline education, not with an absence of educational content. Its result therefore addresses that contrast as described in the registry. The record does not provide full details of the nudges, all aspects of the control module, or enough model information to independently reproduce the sponsor’s mixed-model analysis.

A statistically significant result is not by itself a judgment about clinical importance. The registry does not report a threshold for a meaningful change in this endpoint, and the log-scale estimate should not be converted into an unreported raw-scale interpretation. The posted secondary finding is related but measures a separate prescribing feature, and its interpretation is limited by unreported multiplicity procedures.

Overall, the data describe a statistically clear between-group difference in change on the specified primary measure. They do not support broader claims about all opioid prescribing, all patients, or patient outcomes that were not measured in the posted results.

18. Why This Trial Matters Statistically

This trial is statistically informative because it makes a randomized comparison of two approaches to supporting prescribing decisions: EHR-based behavioral nudges and web-based guideline education. The primary endpoint is defined as a change from baseline to intervention, which focuses the contrast on how the measure shifted over time in each assigned group rather than only comparing a single endpoint value.

The outcome also illustrates why measurement scale matters. The endpoint is a per-clinician average of log total weekly MME, not raw MME and not a patient-level response. The result must be interpreted in the units and population defined by the registry. Statistical significance cannot remove that distinction or supply a clinical threshold that the record does not report.

The posted result also shows why the sponsor’s analysis and a simple recalculation should be kept distinct. The rounded point estimates agree, but the reported confidence intervals differ in width. A model-based analysis can use structure that cannot be recovered from arm-level summaries, although the registry does not give enough detail to assess the specification. Reporting both the estimate and its uncertainty, while stating the analysis method, is important for understanding what the result does and does not establish.

Finally, the trial record raises common interpretation issues: incomplete reporting of analysis populations, missing-data procedures, and multiplicity handling can limit assessment even when a primary result is precise. The secondary endpoint’s direction may add context, but it does not replace a prespecified analysis strategy. These considerations are central to reading clinical trial results carefully rather than treating a small p-value as a complete account of evidence.

19. Limitations

Several trial-specific limitations affect interpretation of the registry results. The study is unmasked, so clinicians and investigators were not reported as blinded to assignment. The primary endpoint is a prescribing measure rather than a patient health outcome, and it is expressed as a clinician-level average on a log scale. Its relationship to the appropriateness of individual prescriptions or to patient outcomes is not reported.

The intervention description identifies the three patient prescribing-history categories that triggered EHR nudges, but the full prompt content is not reported. The control description identifies a web-based educational module and some related guidance, but its complete content is also not reported. This limits the ability to characterize precisely what was compared or to assess implementation fidelity from the registry record.

The endpoint population includes stated exclusions, including control clinicians who erroneously received alerts. The record does not report the number affected by this exclusion, the complete population flow, or whether the primary analysis retained participants according to randomized assignment. The number of clinics and details about clinic-level allocation are also not reported. These gaps make it difficult to assess the impact of exclusions and clustering on the reported analysis.

The sponsor reports a mixed-model analysis, but the model specification and missing-data procedures are not reported. The record also does not report the trial’s original sample-size rationale, a full multiplicity plan, or results for all listed outcomes. Only one secondary result is posted. These reporting gaps limit assessment of robustness, precision under alternative assumptions, and the relative status of secondary findings. The sample-size illustrations based on the observed effect should not be used to fill the gap in the original design justification.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

23. Record Summary

NCT04477304 is a completed, randomized, parallel-group trial with no masking, sponsored by the University of Southern California. Clinics were assigned to EHR behavioral nudges or web-based guideline education. Registry enrollment is 555; the study began on 2020-12-01 and primary completion was 2023-08-06.

The primary endpoint compares change from a 6-month baseline to an 18-month intervention period in average per-clinician log total weekly MME for opioid-naïve and at-risk patients below 50 MME daily dose. The sponsor’s mixed-model analysis reports a net mean difference of -0.09, with a 95% confidence interval from -0.12 to -0.07 and p < 0.001. One secondary result is posted and has a direction consistent with the primary result.

The registry does not report the full analysis plan, model specification, missing-data procedures, multiplicity strategy, or original sample-size justification. The results describe the posted clinician-level prescribing measures and do not directly establish patient-level or health outcomes.

Clinical Biostats