Optimizing Electronic Health Record Prompts With Behavioral Economics to Improve Prescribing for Older Adults: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Optimizing Electronic Health Record Prompts With Behavioral Economics to Improve Prescribing for Older Adults (NCT04284553), a randomized na study. The primary outcome is Change in Inappropriate Prescribing by Study Arm: Descriptive Comparison Across Study Arms.

2. Clinical Question

3. Trial Design

This completed trial was randomized, had a parallel design, and used no masking. The registry lists enrollment as 216 and gives a start date of 2020-10-13 and a primary completion date of 2022-08-01. The study was sponsored by Brigham and Women's Hospital. Its phase is listed as NA.

The intervention was delivered through electronic health record alerts, with study arms representing different alert timings and behavioral features. Listed approaches include order-entry and open-encounter alerts, follow-up boosters, cold-state outreach, simplified alerts, sign-off alerts, pre-commitment, and different risk framings. The registry also lists a standard Epic basic alert and no-alert usual care. The arm descriptions themselves are blank in the record, so further operational detail about each alert is not reported in the registry record.

The primary outcome was assessed across both stages of the adaptive trial at Atrius Health rather than reported separately by stage. The record states that this pooling followed the prespecified analytic plan. It also identifies the design as cluster-randomized and cautions that simple individual-level recalculation would be misleading. The particular randomization unit and the details of how clustering was incorporated into the sponsor's analysis are not reported in the registry record.

The study addressed prescribing of benzodiazepines and sedative hypnotics among older adults. The registry background describes potential concerns associated with these medicines and proposes electronic health record decision support, including behavioral approaches, as a way to support prescribing decisions. These background statements explain the rationale for the trial; they are not findings from its primary outcome analysis.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Primary care providers were identified from electronic health record data and cluster randomized to a study arm in Stage 1. Patients who sought care from these primary care providers were identified based on electronic health record data for analyses. Providers were not assessed for effectiveness outcomes. After the interim analysis, primary care providers were re-randomized to a study arm in Stage 2.

Only the first 6 of 16 groups are shown.

Stage 1: October 2020-August 2021

Stage 2: September 2021-August 2022

Analysis population for the primary outcome, as defined in the registry: Patients who met eligibility criteria for inclusion in the analysis (adults ≥65 years, who were prescribed ≥90 pills of benzodiazepine or sedative hypnotic in the last 180 days and had an office or telemedicine visit with a primary care provider who was assigned to a study arm), as measured from EHR system data across Stages 1 and 2

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: All of these baseline measures are for patients included in analyses across Stages 1 and 2 of the trial (n=3063 total, 2909 in Stage 1 and 154 in Stage 2). Data from primary care providers was not collected at baseline nor in follow-up. Baseline patient characteristics are shown for Stages 1 and 2 combined, as the analyses were pre-specified to be conducted combining outcomes across both Stages.

Only the first 6 of 17 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as Change in Inappropriate Prescribing by Study Arm: Descriptive Comparison Across Study Arms, measured over 22 months (2 stages).

The primary endpoint is a binary measure of whether an eligible patient had a change in prescribing during follow-up. A patient counted as having a change if the primary care provider either discontinued a study high-risk medication or ordered a dose taper for a benzodiazepine or sedative hypnotic. Discontinuation included active discontinuation or the absence of an order during follow-up. The outcome was measured using electronic health record data.

The endpoint is reported as a percentage of patients. A binary outcome gives each eligible patient a classification based on whether at least one of the defined prescribing actions occurred; it does not describe the amount of medication change or distinguish between the different actions in the composite. Because the outcome combines discontinuation and tapering, a group-level percentage would not by itself show which component contributed to the result.

The registry specifies a timeframe of 22 months across 2 stages and states that results were measured and presented across both stages rather than separately. This composite provides a consistent, observable indicator of prescribing change within the study’s stated definition. It is a measure of prescribing behavior, not a direct measure of patient health or adverse clinical outcomes. The primary outcome results are not reported in the registry record, so the recorded endpoint definition cannot be used to determine the observed percentage or differences between study arms.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The endpoint is binary at the patient level: a patient either met the definition of prescribing change or did not. Reporting this outcome as a percentage expresses the frequency of that classification within an arm. This is distinct from a continuous outcome, such as a measured quantity that can take a range of values, and from a count of repeated events. The registry identifies the primary endpoint as a percentage of patients and describes it as a binary outcome.

Why does the cluster-randomized design matter?

In a cluster-randomized design, the assignment is made at a group level rather than treating every patient as an independent randomization unit. Patients sharing a cluster can have more similar outcomes than patients in different clusters. General statistical methods for clustered data account for that correlation when estimating uncertainty. If dependence is ignored, standard errors and confidence intervals can be too small, making results look more precise than warranted. The registry notes that this study was cluster-randomized and that individual-level re-analysis would be misleading. The randomization unit, the cluster structure, and the exact sponsor analysis are not reported in the registry record.

What does a confidence interval convey?

As a general statistical explanation, a confidence interval expresses uncertainty around an estimated effect under the assumptions of the analysis. It helps readers distinguish a relatively precise estimate from one with substantial uncertainty. A point estimate alone does not show that uncertainty. The registry record does not report a confidence interval or other precision estimate for the primary endpoint, so the precision of the comparison cannot be assessed from the record.

How should the composite endpoint be understood?

The primary outcome combines discontinuation of a study high-risk medication, including no order during follow-up, with ordering a dose taper for benzodiazepines or sedative hypnotics. Combining components can create a single outcome for analysis, but the overall percentage does not show the contribution of each component. The record does not report separate component results. Nor does this prescribing measure directly capture whether a patient experienced a health event.

What analysis details are available?

The outcome description says that data were measured and presented across both stages of the adaptive trial rather than separately, as part of the prespecified analytic plan. The record does not report the statistical model, the treatment of clustering in that model, any covariate adjustment, the method for constructing uncertainty intervals, or the test used for comparisons. These details are not reported in the registry record and should not be inferred from the endpoint definition.

The registry gives 4 outcome measures and 14 analyses. Those counts indicate that the record includes multiple outcomes or analyses, but do not identify a multiplicity adjustment or establish how the analyses were prioritized beyond the stated primary outcome. The record lists secondary outcomes concerning prescribing quantity across study arms and across seven behavioral intervention factors. Results for those secondary outcomes are not reported in the registry record.

The sample-size assumptions describe a planning illustration based on the observed effect, with a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation; the effect to detect is described as a stated fraction of the observed effect. Such calculations are planning illustrations, not observed power. The record does not report the numerical sample-size outputs or the trial's own sample-size justification. These assumptions should not be treated as the actual analysis plan or as evidence about the precision of the trial's primary result.

10. Exact Estimates for Each Arm

No two-arm comparison is shown for this trial, but each arm's result can still be summarized with exact interval estimates computed from the posted counts. The Clopper-Pearson interval is exact and conservative; the Wilson interval is a common closer-to-nominal alternative; the Bayesian credible interval uses a uniform Beta(1, 1) prior. For small groups the three can differ noticeably.

R code and output

arm    <- c("Base Order Entry Alert", "Base Open Encounter Alert", "Order Entry + Follow-up Booster Alert", "Open Encounter + Follow-up Booster Alert", "Order Entry + Cold State Outreach", "Open Encounter + Cold State Outreach")
n      <- c(91, 106, 87, 175, 174, 102)
events <- c(32, 35, 31, 71, 45, 40)
for (i in seq_along(n)) {
  ci <- binom.test(events[i], n[i])$conf.int          # Clopper-Pearson exact interval
  cr <- qbeta(c(0.025, 0.975), 1 + events[i], 1 + n[i] - events[i])   # Bayesian, uniform Beta(1,1) prior
  wi <- (function(x, n) { p <- x / n; z <- qnorm(0.975); d <- 1 + z^2 / n
    c((p + z^2 / (2 * n) - z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d, (p + z^2 / (2 * n) + z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d) })(events[i], n[i])
  cat(sprintf("ARM|%s|%d|%d|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f\n", arm[i], n[i], events[i], 100 * events[i] / n[i],
              100 * ci[1], 100 * ci[2], 100 * wi[1], 100 * wi[2], 100 * cr[1], 100 * cr[2]))
}

ARM  Base Order Entry Alert  91  32  35.2  25.4  45.9  26.1  45.4  26.1  45.4
ARM  Base Open Encounter Alert  106  35  33.0  24.2  42.8  24.8  42.4  24.8  42.5
ARM  Order Entry + Follow-up Booster Alert  87  31  35.6  25.6  46.6  26.4  46.1  26.4  46.1
ARM  Open Encounter + Follow-up Booster Alert  175  71  40.6  33.2  48.2  33.6  48.0  33.6  48.0
ARM  Order Entry + Cold State Outreach  174  45  25.9  19.5  33.0  19.9  32.8  19.9  32.9
ARM  Open Encounter + Cold State Outreach  102  40  39.2  29.7  49.4  30.3  48.9  30.3  48.9

11. Multiplicity, Missing Data and Analysis Populations

The registry lists 4 outcome measures and 14 analyses. It also identifies secondary prescribing outcomes: descriptive comparisons across study arms and an analysis across seven behavioral intervention factors. Having multiple outcomes and comparisons creates the general statistical issue of multiplicity: as more hypotheses are examined, the chance of at least one apparently notable result can increase. Whether or how that issue was addressed depends on a prespecified analysis strategy. A multiplicity adjustment or other control strategy is not reported in the registry record.

The primary outcome description says that results were pooled across both stages of the adaptive trial rather than presented separately, and that this followed the prespecified analytic plan. Other details of that plan, including how the primary comparison was chosen and how any secondary analyses were handled, are not reported in the registry record. The record does not provide outcome estimates or test results with which to assess the consequences of multiple analyses.

The endpoint population is described as eligible patients aged 65 years or older who met the medication and visit criteria and whose primary care provider was assigned to a study arm. The registry does not report the number of patients analyzed for the primary endpoint by arm, the handling of patients with incomplete electronic health record follow-up, or any imputation or missing-data sensitivity analyses. It also does not report whether the analysis followed an intention-to-treat principle or another analysis-population definition.

Because the trial is identified as cluster-randomized, the analysis population and missingness at both patient and cluster levels could matter to interpretation. The specific methods used to account for clustering and missing data are not reported in the registry record. The absence of those details does not establish that a particular method was or was not used; it means the record does not permit assessment of those choices.

12. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: intentional overdose (0.6%).

The adverse-event record lists serious and other events by study arm, along with the number at risk and death counts. The serious-event field is 0 in every listed arm except Open Encounter + Pre-commitment, which records 1. The other-event field is 0 for every listed arm. These are the posted counts; they are not the primary prescribing endpoint and do not indicate the frequency of prescribing change.

The death counts are recorded separately from the serious and other event counts. For example, the No Alert (Usual Care) group lists 18 deaths, while Open Encounter + Sign-off Alert lists 11. The record does not provide a causal assessment linking these deaths to an intervention or to a particular adverse event. Counts alone do not establish whether events were caused by an alert strategy, and the record does not report comparative safety estimates or uncertainty intervals.

The top serious-event term is Intentional overdose, reported as 0.6%. The record does not provide further detail about the event's relationship to treatment assignment. These safety entries should therefore be read as descriptive registry counts, not as evidence of a causal safety difference. They also do not answer the primary question, which concerns the percentage of eligible patients with a defined prescribing change.

13. What the Result Does, and Does Not, Mean

The registry describes a randomized trial testing electronic health record alerts with behavioral features and defines a primary outcome based on discontinuation or tapering of selected high-risk medications. It also reports that the outcome was measured across both stages of the adaptive trial. These registry entries clarify what the study set out to measure and how the primary outcome was constructed.

The registry record does not report the primary outcome percentages, comparison estimates, or precision measures. It therefore does not show whether prescribing change was more or less frequent in any particular arm, how large any observed contrast was, or how uncertain it was. The listed adverse-event counts cannot fill that gap: they concern safety reporting and deaths, not the primary prescribing outcome.

Even if a primary outcome percentage were available, the composite would represent the occurrence of a defined prescribing action, not a direct measure of health benefit or harm. It would not, by itself, establish why prescribing changed, distinguish the contribution of discontinuation from tapering, or show that a particular alert feature caused the change. Those interpretations would require appropriate arm comparisons, analysis that accounts for the cluster-randomized design, and relevant outcome detail.

The study's background rationale describes concerns about benzodiazepines and sedative hypnotics in older adults and the potential role of decision support. That rationale should not be mistaken for a trial result. The registry information supports an account of the research question, intervention categories, design, and outcome definition; it does not support a conclusion about the effectiveness or ineffectiveness of a treatment or alert strategy.

14. Why This Trial Matters Statistically

This trial raises a statistical question that matters for clinical research: whether electronic health record decision support can change prescribing practices, and how behavioral features of alerts should be compared. The study includes different alert timings and features, along with standard-alert and no-alert groups. Its outcome definition translates prescribing behavior into a binary measure that can be reported as a percentage.

The cluster-randomized design is especially relevant to inference. When assignment occurs by cluster, observations within a cluster may be correlated. Accounting for that structure is important to estimating uncertainty and avoiding misleading patient-level re-analysis. The registry explicitly warns against a simple individual-level recalculation, underscoring that the analysis must respect the design rather than rely only on the count of patient records.

The trial also illustrates how a composite endpoint can make a practical outcome measurable while combining distinct actions. Discontinuation, no order during follow-up, and dose tapering all meet the stated definition, but a single composite percentage would not reveal how each action contributed. The outcome is useful for describing prescribing change, yet it should not be confused with patient-centered health outcomes.

Finally, the registry record demonstrates the difference between a clear endpoint definition and a reportable result. Without arm-level estimates and uncertainty measures, readers cannot judge the size, direction, or precision of a comparison. A transparent account of the analysis population, missing data, multiplicity, and cluster-adjusted methods would also be important for evaluating the evidence. Those outcome and analysis details are not reported in the registry record.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The main interpretive limitation is that the registry record does not report primary outcome results. Arm-specific percentages, comparative effect estimates, and precision measures are absent, preventing assessment of the direction, size, or uncertainty of the prescribing outcome.

The record identifies a cluster-randomized design and cautions that individual-level recalculation would be misleading, but does not describe the randomization unit or the sponsor's statistical model. Details of how clustering was handled, whether covariates were included, and how uncertainty was calculated are not reported in the registry record. That limits independent evaluation of the primary comparison.

The primary endpoint combines several qualifying actions: active discontinuation, lack of an order during follow-up, or a dose-taper order. The registry does not provide separate results for those components. Thus, even if the composite result were reported, it would not show which action drove the overall percentage. The outcome captures prescribing behavior rather than patient health outcomes.

Although the record identifies the eligible population and states that results were pooled across both stages, it does not report the analyzed population by arm, missing-data handling, or sensitivity analyses. It lists multiple outcomes and analyses but does not report a multiplicity strategy. Arm descriptions are blank, leaving the precise implementation of each alert strategy unclear. These gaps limit assessment of reproducibility and robustness; they should not be taken as evidence that a particular method was not used.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Optimizing Electronic Health Record Prompts With Behavioral Economics to Improve Prescribing for Older Adults — NCT04284553.
• Registry record last updated 2024-11-07; status last verified 2024-10.
• Results first posted 2023-10-26.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

NCT04284553 is a completed, randomized, parallel trial with no masking, sponsored by Brigham and Women's Hospital. The registry lists enrollment as 216, a start date of 2020-10-13, and a primary completion date of 2022-08-01. The trial evaluated electronic health record alerts with behavioral features, alongside a standard alert and no-alert usual care.

The primary endpoint was the percentage of eligible patients with a defined prescribing change: discontinuation or lack of an order for a study high-risk medication, or a dose-taper order for a benzodiazepine or sedative hypnotic. The outcome was measured across 22 months and 2 stages. The design is identified as cluster-randomized, and the record cautions against simple individual-level re-analysis.

Primary outcome estimates and details of the statistical analysis, missing-data handling, and multiplicity strategy are not reported in the registry record. The adverse-event entries are descriptive and do not supply the missing prescribing results.