NUDGE-EHR Replication Trial at Mass General Brigham: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for NUDGE-EHR Replication Trial at Mass General Brigham (NCT05538065), a randomized na study. The primary outcome is Change in Inappropriate Prescribing.

2. Clinical Question

3. Trial Design

The NUDGE-EHR Replication Trial at Mass General Brigham is listed as a randomized, parallel-group trial with no masking. The registry lists Brigham and Women's Hospital as sponsor, an enrollment of 201, and a completed status. The recorded start date is 2022-11-10 and the primary completion date is 2024-03-15. The phase is listed as NA.

The study tested EHR-based strategies intended to influence providers’ prescribing of potentially high-risk medications for older adults. Named approaches include Open Encounter + Pre-commitment and Open Encounter + Follow-up Booster, alongside No Alert (Usual Care). In the usual-care arm, providers received no intervention. The booster is described as an option for a provider-directed in-basket message sent 4 weeks after an alert is triggered. The pre-commitment strategy is described as prompting a discussion of medication risks and sharing a handout; its full arm description is not reported in the registry record.

The registry’s statistical notes identify the design as cluster-randomized and caution that an individual-level re-analysis would be misleading. The arm descriptions say providers were randomized, while the primary outcome is measured for eligible patients treated by those providers. This distinction matters: patients treated by the same provider may share exposure to the same assigned strategy, so treating every patient as independently randomized would not reflect the described assignment process.

The primary outcome timeframe began when each patient was identified as eligible and continued until the end of a 16-month follow-up. The timeframe varied by participant. The registry also lists secondary outcomes for quantity of high-risk medication prescribed and cumulative lorazepam milligram equivalents prescribed. Further details on allocation procedures, cluster counts, and the complete analysis plan are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Primary care providers were identified from electronic health record data and cluster randomized to a study arm. Patients who sought care from these primary care providers were identified based on electronic health record data for analyses and are classified as the participants. Providers were not assessed for effectiveness outcomes.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Patients who met eligibility criteria for inclusion in the analysis (adults ≥65 years, who were prescribed ≥90 pills of benzodiazepine, strongly anticholinergic medication, or sedative hypnotic in the last 180 days and had an office or telemedicine visit with a primary care provider who was assigned to a study arm), as measured from EHR system data.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: All of these baseline measures are for the patients included in analyses in the data. Data from primary care providers (the unit of randomization) was not collected at baseline nor in follow-up.

6. Primary Endpoint

The registry lists the primary outcome as Change in Inappropriate Prescribing, measured over Each patient was assessed beginning from the date in which they were identified as eligible (i.e., meeting the inclusion criteria) until the end of a 16-month follow-up; the outcome time frame varied by participant.

The primary endpoint, Change in Inappropriate Prescribing, is a binary measure of whether a qualifying prescribing action occurred for an eligible patient. It counts a change if the primary care provider discontinued a study high-risk medication, made no order for it during follow-up, or ordered a dose taper for a benzodiazepine or sedative hypnotic. The record defines discontinuation to include active discontinuation or lack of an order during follow-up. The outcome was assessed using EHR data.

This type of composite endpoint can capture more than one way prescribing might change. It allows the analysis to count a discontinuation and a taper as relevant actions, rather than limiting the outcome to a single medication or action. Because the outcome is binary, it indicates whether a qualifying action was recorded, not how many actions occurred or how large a medication change was.

The endpoint is used to represent a change in prescribing practice within the study’s target population. It does not directly measure adverse health consequences, patient preferences, or whether a prescribing decision was clinically appropriate for an individual. The follow-up period varied by participant and extended to the end of a 16-month follow-up beginning at eligibility. The record does not report a more detailed operational definition of how each type of action was classified beyond the description above.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The reported primary analyses use logistic regression and present their results as risk ratios. Logistic regression is a model for a binary outcome, such as whether a patient had a recorded change in prescribing. The registry does not describe the exact model formula, covariates, or method used to express the fitted results as risk ratios. Those details are not reported in the registry record, so the specific estimation procedure cannot be reconstructed from the posted description.

What does a risk ratio describe?

General statistical explanation: a risk ratio compares the probability of an outcome in one group with the probability in another group. A value of 1 represents equal risks under the chosen numerator and reference coding. Values on either side of 1 indicate different risks, with direction depending on which group is in the numerator. Here, the registry reports 1.40 for OG000 versus OG001 and 1.26 for OG000 versus OG002. It does not identify the named arms corresponding to those group codes or specify the coding direction in the posted analysis description. The numerical values alone therefore do not identify which named strategy had more recorded changes.

How should the confidence intervals be read?

General statistical explanation: a confidence interval shows the uncertainty around an estimate under the model and its assumptions. A narrower interval indicates greater precision than a wider interval, while an interval spanning a particular null value may be compatible with a range of effects that includes no difference. The registry reports 95% confidence intervals of 1.10 to 1.78 and 1.01 to 1.57. They are intervals for the risk-ratio estimates, not ranges containing a stated proportion of individual patient outcomes. Their interpretation also depends on the validity of the model and the handling of the cluster-randomized design.

What do the p-values add?

General statistical explanation: a p-value assesses how unusual the data would be under a specified null hypothesis and model. It does not measure the probability that the null hypothesis is true, nor does it measure clinical importance. The registry reports p = 0.002 and p = 0.039. These values accompany the estimated comparisons; they do not show whether a prescribing action was beneficial for an individual patient or whether an intervention is appropriate in a particular clinical situation.

Why does cluster randomization matter?

General statistical explanation: in a cluster-randomized study, groups such as providers or practices, rather than individual patients, are assigned to study conditions. Outcomes among patients within a cluster may be more alike than outcomes among unrelated patients because they share a provider or care environment. This dependence can affect the uncertainty of estimates. An analysis that treats all patients as independently randomized may understate or otherwise mischaracterize uncertainty. The registry specifically says an individual-level re-analysis would be misleading and that only the sponsor’s own analysis is shown. The cluster definition, number of clusters, and exact adjustment method are not reported in the registry record.

What do planning illustrations mean?

General statistical explanation: sample-size planning illustrations use assumptions about the effect to detect and the statistical procedure to estimate a number of participants or events needed for a future study. They are not measures of the probability that the completed trial detected an effect. The registry’s planning assumptions specify a two-sided significance level of 0.05, equal allocation between two arms, a normal approximation, and effects expressed as stated fractions of the observed effect. For time-to-event outcomes, the stated output is total events; for other outcomes, it is participants per arm. The primary outcome is binary, not described as time-to-event. Numerical planning results are not included in the registry record, and these illustrations should not be mistaken for the trial’s original design justification or observed power.

The registry lists six analyses and three outcome measures, but does not describe how the reported tests were selected, whether adjustments for multiple comparisons were used, or how secondary outcomes were analysed. The posted results should therefore be interpreted as the reported analyses, without assuming an unreported statistical plan.

10. Exact Estimates for Each Arm

No two-arm comparison is shown for this trial, but each arm's result can still be summarized with exact interval estimates computed from the posted counts. The Clopper-Pearson interval is exact and conservative; the Wilson interval is a common closer-to-nominal alternative; the Bayesian credible interval uses a uniform Beta(1, 1) prior. For small groups the three can differ noticeably.

R code and output

arm    <- c("No Alert (Usual Care)", "Open Encounter + Pre-commitment", "Open Encounter + Follow-up Booster")
n      <- c(396, 394, 356)
events <- c(106, 145, 122)
for (i in seq_along(n)) {
  ci <- binom.test(events[i], n[i])$conf.int          # Clopper-Pearson exact interval
  cr <- qbeta(c(0.025, 0.975), 1 + events[i], 1 + n[i] - events[i])   # Bayesian, uniform Beta(1,1) prior
  wi <- (function(x, n) { p <- x / n; z <- qnorm(0.975); d <- 1 + z^2 / n
    c((p + z^2 / (2 * n) - z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d, (p + z^2 / (2 * n) + z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d) })(events[i], n[i])
  cat(sprintf("ARM|%s|%d|%d|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f\n", arm[i], n[i], events[i], 100 * events[i] / n[i],
              100 * ci[1], 100 * ci[2], 100 * wi[1], 100 * wi[2], 100 * cr[1], 100 * cr[2]))
}

ARM  No Alert (Usual Care)  396  106  26.8  22.5  31.4  22.6  31.3  22.6  31.3
ARM  Open Encounter + Pre-commitment  394  145  36.8  32.0  41.8  32.2  41.7  32.2  41.7
ARM  Open Encounter + Follow-up Booster  356  122  34.3  29.3  39.5  29.5  39.3  29.5  39.3

11. Analyses Reported by the Sponsor

The registry’s analysis note identifies the trial as cluster-randomized and says a simple individual-level re-analysis would be misleading. It therefore presents only the sponsor’s own analysis for these comparisons. No independent recalculated estimates are reported, and the cluster design means that treating patient records as if each patient had been randomized independently would not be an appropriate substitute.

The sponsor’s reported primary analyses use logistic regression and give risk ratios, 95% confidence intervals, and p-values. The two reported comparisons are OG000 versus OG001 and OG000 versus OG002. Their estimates are 1.40 and 1.26, respectively; the reported intervals are 1.10 to 1.78 and 1.01 to 1.57, and the p-values are 0.002 and 0.039. The registry labels both analyses SUPERIORITY. The mapping from those group codes to the named arms, and the coding direction of the ratios, are not reported in the registry record, limiting interpretation by intervention name.

Because no recalculation is presented, there is no independent estimate to compare with the sponsor’s figures. The registry also does not report enough model detail to reproduce the analysis or assess every aspect of its treatment of clusters. The sponsor’s estimates should be understood as the reported analysis, with those interpretive limits, rather than as a result verified by an individual-level recalculation.

12. Multiplicity, Missing Data and Analysis Populations

The registry lists three outcome measures and six analyses. It reports two primary-outcome comparisons, but the posted information does not explain how the full set of analyses was organized, which analyses were prespecified, or whether the p-values were adjusted for multiple comparisons. A multiplicity procedure is not reported in the registry record. When several outcomes or contrasts are tested, the chance of at least one apparently notable result can increase; adjustment methods are one way researchers may address that issue. Without a reported plan, the two p-values should be read as the values attached to the stated analyses, not as evidence that any multiplicity adjustment was or was not applied.

Missing-data handling is not reported in the registry record. The outcome description says that patients were followed from eligibility until the end of a 16-month follow-up, with the timeframe varying by participant. It does not describe how incomplete EHR records, missing outcome classifications, loss to follow-up, or other unavailable information were handled. The registry also does not state whether all eligible patients were included in each analysis or whether exclusions occurred after eligibility was established.

The primary outcome population is described as eligible adults with recent prescribing of specified high-risk medication classes and a visit with a primary care provider assigned to a study arm. The posted eligibility text is truncated, and the complete analysis-population definition is not reported in the registry record. The record does not describe an intention-to-treat population, a per-protocol population, or another formal analysis set.

Because this is cluster-randomized, the way clusters and patients entered the analysis is important. The registry cautions against simple individual-level re-analysis but does not provide the number of clusters, cluster-level exclusions, or detailed methods for handling dependence within clusters. These omissions limit independent assessment of the analysis population and uncertainty estimates.

13. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry’s safety figures list zero serious adverse events and zero other adverse events in each of the three named groups. The numbers at risk are 396 for No Alert (Usual Care), 394 for Open Encounter + Pre-commitment, and 356 for Open Encounter + Follow-up Booster. Deaths are reported as 14, 7, and 6, respectively. These are separate counts in the safety record; the registry does not state that deaths were caused by a study strategy.

The safety denominators do not match the trial enrollment of 201. The record does not explain the difference between the enrollment figure and the numbers at risk in the safety results. These figures should therefore be read as the reported safety counts and denominators, not treated as interchangeable with enrollment or as evidence of a specific reason for the difference.

Zero recorded events does not establish that no adverse events occurred outside the registry’s reporting process, and the record does not provide additional detail on event collection, attribution, or follow-up for these figures. No serious adverse-event terms are listed. The safety data also do not establish whether a prescribing change caused or prevented an adverse event. They are descriptive counts, and the registry provides no further comparative safety analysis in the posted results.

14. What the Result Does, and Does Not, Mean

The reported analyses address whether the study strategies were associated with a binary change in prescribing for eligible older adults. The endpoint counts a recorded discontinuation, lack of an order during follow-up, or qualifying dose taper. It is therefore a measure of prescribing action recorded in the EHR, not a direct measure of medication exposure over time, patient health, or the quality of an individual decision.

The registry reports two risk-ratio estimates, both labelled SUPERIORITY, with 95% confidence intervals and p-values. The intervals exclude 1, but the registry does not identify the named arms corresponding to the comparison codes or explain the ratios’ numerator and reference coding. These details prevent a confident translation of the estimates into a direction of effect for each named strategy. Statistical evidence from these comparisons also does not establish that a particular strategy would produce the same result in another setting or population.

The results do not show whether medication quantities or cumulative lorazepam milligram equivalents changed, even though those outcomes are listed as secondary measures. Results for those secondary outcomes are not reported in the registry record. Nor do the primary analyses establish whether any observed prescribing action led to fewer adverse drug events, hospitalizations, falls, or other patient outcomes. Such outcomes are not part of the stated primary endpoint.

The trial is described as cluster-randomized, and the registry cautions that individual-level recalculation would be misleading. The sponsor’s reported analysis is therefore the relevant statistical result available in the record, although details needed for full independent assessment are not reported. The estimates should be read within those limits: they quantify the reported comparisons for the measured prescribing outcome and do not constitute medical advice or a recommendation for an individual patient.

15. Why This Trial Matters Statistically

This trial raises a statistical question common to evaluations of clinical decision support: how to assess a provider-facing intervention when assignment occurs at the provider or cluster level but outcomes are measured for patients. That distinction affects both the interpretation of the comparison and the uncertainty around the estimated results. The registry’s explicit warning against an individual-level re-analysis is important because it identifies a design feature that cannot be ignored when interpreting the reported estimates.

The outcome also illustrates the choices involved in defining a behavioral endpoint. The registry combines discontinuation, absence of an order during follow-up, and dose tapering for specified medication classes into a binary measure. That definition captures several possible prescribing actions, but it does not convey their magnitude, duration, clinical rationale, or patient-level consequences. Reading the estimate as a change in recorded prescribing, rather than as a direct measure of health benefit, keeps the statistical claim aligned with what was measured.

The posted analyses report risk ratios with confidence intervals and p-values, but group-code mapping and ratio coding are not stated. This shows why an effect estimate needs a clearly named comparison and direction to be readily interpreted. The record’s incomplete model and analysis-population details also matter: design-aware estimation, eligibility definitions, missing-data handling, and multiplicity decisions shape what can be concluded from a reported result.

Finally, the safety counts and enrollment figure use different denominators, with no explanation in the record. Keeping those figures distinct rather than combining them avoids an unsupported interpretation. Together, these features make the trial useful for statistical reading: the headline estimate is only one part of the evidence, and design, endpoint definition, coding, and reporting completeness all determine how far that estimate can be taken.

16. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record leaves several details needed for a fuller assessment unreported. Although it identifies the study as cluster-randomized and cautions against individual-level re-analysis, it does not state the number or definition of clusters, the allocation procedures, or the precise method used to account for clustering. It also does not provide the complete logistic-regression specification, covariates, or the method used to report risk ratios from the model.

The group codes in the reported comparisons are not mapped to the named arms in the posted analyses. The numerator and reference coding for the reported risk ratios are not stated there either. This limits interpretation of which intervention is associated with a higher or lower outcome risk. In addition, the pre-commitment arm description is truncated, and the complete eligibility criteria are not available in the record.

The registry lists three outcome measures and six analyses but does not report whether the tests were adjusted for multiple comparisons or how primary and secondary analyses were designated. Missing-data methods, analysis-population definitions, and handling of incomplete follow-up are not reported in the registry record. These omissions limit assessment of the analysis choices and the robustness of the reported estimates.

The primary endpoint is a composite measure of recorded prescribing actions. It does not directly measure medication exposure, clinical appropriateness for each patient, patient preferences, or health outcomes. Results for the listed secondary outcomes are not reported in the available analyses. The safety results list no adverse events but do not describe event ascertainment or attribution. The numbers at risk also differ from the recorded enrollment, without an explanation in the record. These limitations do not invalidate the reported estimates, but they define what can be interpreted from them.

17. Related Tutorials

18. Related Statistical Calculators

19. Sources

• ClinicalTrials.gov: NUDGE-EHR Replication Trial at Mass General Brigham — NCT05538065.
• Registry record last updated 2025-05-21; status last verified 2025-05.
• Results first posted 2025-04-01.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

20. Record Summary

NCT05538065, NUDGE-EHR Replication Trial at Mass General Brigham, is listed as a completed, randomized, parallel-group trial with no masking and enrollment of 201. Brigham and Women's Hospital is the sponsor. The primary outcome is a binary measure of change in inappropriate prescribing, based on EHR-recorded discontinuation or lack of an order, or a dose taper for specified medication classes, during a participant-varying follow-up ending at 16 months.

The registry reports two sponsor analyses using logistic regression and risk ratios: 1.40 (95% confidence interval 1.10 to 1.78; p = 0.002) and 1.26 (95% confidence interval 1.01 to 1.57; p = 0.039). Both are labelled SUPERIORITY. The comparison-code mapping to named arms and ratio coding are not reported in the registry record. The record identifies cluster randomization and cautions that individual-level recalculation would be misleading. Details on multiplicity, missing data, and full model specification are also not reported.