Effect of Behavioral Nudges on Serious Illness Conversation Documentation: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Effect of Behavioral Nudges on Serious Illness Conversation Documentation (NCT04867850), a randomized na study. The primary outcome is Number of High Risk Patients With Documentation of a Serious Illness Conversation (SIC).

2. Clinical Question

3. Trial Design

The registry describes a randomized, single-masked, factorial, pragmatic cluster trial with four arms: usual care, clinician nudge, patient nudge, and clinician and patient nudge. The planned enrollment was 4450, and the record lists the study as completed. The study start date was 2021-09-09, and the primary completion date was 2022-09-09. The phase is listed as NA.

The strategies varied whether a nudge was directed to clinicians, patients, or both. The patient strategy used a normalizing message prompting completion of an electronic questionnaire designed to prime a serious illness conversation. The clinician strategy added peer-comparison performance feedback to targeted messages identifying patients at high risk of predicted 6-month mortality. Usual care already included the targeted clinician messages, so it was not an absence-of-contact condition.

The brief summary describes the trial as cluster randomized, but the unit of cluster allocation and the number of clusters are not reported in the registry record. The party or parties masked under the single-masking designation are also not reported. The record identifies the design as factorial but does not report the sponsor’s statistical model, the planned contrasts, or whether an interaction between the clinician and patient strategies was assessed.

The primary endpoint concerned high-risk patients and documentation within 6 months of the index visit. The record also lists outcomes concerning documentation among all patients, palliative care referrals among high-risk patients, and aggressive end-of-life care among decedent high-risk patients. Although the trial is marked completed, outcome estimates and the sponsor’s analysis are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: A total of 4,450 patients were included in this study on serious illness conversations (SICs). These patients were seen by 163 clinicians across 65 oncologist-advanced practice provider (APP) clusters. Baseline measures and outcome data were collected for patients only.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Baseline measures, participant counts, and outcome data all correspond to details for patients only. Baseline and outcome data were not collected for clinicians.

6. Primary Endpoint

The registry lists the primary outcome as Number of High Risk Patients With Documentation of a Serious Illness Conversation (SIC), measured over Within 6 months of the Index Visit (baseline).

The primary endpoint was the number of high-risk patients with documentation of a serious illness conversation (SIC). The outcome was measured at the patient level as binary: a patient either had qualifying documentation or did not. The registry defines qualifying documentation by a documented note including the SIC template in the Advanced Care Planning section of the electronic medical record, entered by any provider. The timeframe was within 6 months of the index visit, described as baseline.

This endpoint measures recorded documentation, not every conversation that may have taken place. A conversation could occur without the specified template being documented, while a qualifying note records that the documentation criterion was met. The registry therefore operationalizes the primary question through a record-based measure of documentation rather than a direct assessment of conversation quality, content, or patient experience.

Using a binary outcome makes it possible to count how many patients met the documentation criterion in each arm. It does not, on its own, show why documentation occurred or whether the conversation changed care. The registry record does not report the primary outcome counts, the arm-specific proportions, or an estimated comparison between arms. It also does not report further details about how missing or ambiguous documentation was handled.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

General: binary outcomes. The primary outcome is binary at the patient level: qualifying SIC documentation was either present or absent within the specified period. A binary endpoint can be described with counts and proportions, then compared across study arms using an analysis suited to the design. The registry names the endpoint and its measurement rule, but it does not report arm-specific primary outcome counts, an effect estimate, a confidence interval, or a test result. No magnitude or direction of effect can therefore be interpreted from this record.

General: uncertainty. A confidence interval expresses the range of effect values compatible with the data under the assumptions of the analysis. A narrow interval indicates greater precision than a wide interval; neither width nor statistical significance alone establishes clinical importance. Since no interval for the primary endpoint is reported in the registry record, its precision cannot be assessed. The record also does not report a primary outcome p-value or the sponsor’s analytic model.

General: cluster randomization. In cluster-randomized studies, groups rather than individual patients are assigned to study conditions. Patients in the same group can have correlated outcomes because they may share clinicians, clinic workflows, or other contextual features. Statistical methods ordinarily need to allow for this dependence; ignoring it can understate uncertainty. The trial summary identifies a cluster-randomized design, and the record warns that a simple individual-level re-analysis would be misleading. The specific clustering unit, number of clusters, and sponsor’s method for accounting for clustering are not reported in the registry record.

General: factorial structure. A factorial design varies more than one intervention component across arms. This trial has conditions corresponding to clinician nudges and patient nudges, including a condition with both. In general, factorial analyses may estimate the contribution of each component and assess whether the combined components act differently from what would be expected based on their separate effects. Such interpretations depend on the specified contrasts and model. The registry labels the design factorial but does not report the sponsor’s planned or completed factorial contrasts, or whether an interaction was estimated.

General: comparator and intervention definition. Intervention effects are interpreted relative to the comparator actually used. Here, usual care included targeted text messages identifying patients at high risk of predicted 6-month mortality. The clinician-nudge arm added peer-comparison performance feedback. As a result, any comparison involving those arms would be about the strategies as described, rather than a simple comparison of messaging against no messaging. The record does not report fidelity, exposure, or adherence measures that would clarify how consistently the strategies were delivered or received.

General: planning versus results. Sample-size planning is distinct from observed evidence. The registry’s planning assumptions describe a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation; the illustrative effect is expressed as a fraction of the observed effect. The record states that the planning illustration is not the trial’s own design. Such an illustration cannot replace the sponsor’s analysis, and a planned sample size is not an observed effect or a measure of observed power. The actual planning justification and the sponsor’s sample-size rationale are not reported in the registry record.

General: outcome reporting. A useful analysis would normally make clear which participants were included, how the endpoint was handled when documentation status could not be determined, and how estimates and uncertainty were calculated. The primary outcome description says outcomes were measured for patients only. The registry record does not report analysis populations, missing-data methods, exclusions from analysis, or the sponsor’s statistical analysis plan. These omissions limit assessment of how the intended endpoint was translated into an analysis.

The record lists four outcome measures: the primary high-risk SIC documentation outcome and secondary outcomes concerning SIC documentation among all patients, palliative care referral among high-risk patients, and aggressive end-of-life care among decedent high-risk patients. The registry lists no analyses. The reported outcome definitions and study design can explain what was intended to be evaluated, but without sponsor results they cannot answer whether any intervention condition changed an outcome.

10. Exact Estimates for Each Arm

No two-arm comparison is shown for this trial, but each arm's result can still be summarized with exact interval estimates computed from the posted counts. The Clopper-Pearson interval is exact and conservative; the Wilson interval is a common closer-to-nominal alternative; the Bayesian credible interval uses a uniform Beta(1, 1) prior. For small groups the three can differ noticeably.

R code and output

arm    <- c("Usual Care", "Clinician Nudge", "Patient Nudge", "Clinician and Patient Nudge")
n      <- c(1004, 1179, 997, 1270)
events <- c(112, 136, 115, 179)
for (i in seq_along(n)) {
  ci <- binom.test(events[i], n[i])$conf.int          # Clopper-Pearson exact interval
  cr <- qbeta(c(0.025, 0.975), 1 + events[i], 1 + n[i] - events[i])   # Bayesian, uniform Beta(1,1) prior
  wi <- (function(x, n) { p <- x / n; z <- qnorm(0.975); d <- 1 + z^2 / n
    c((p + z^2 / (2 * n) - z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d, (p + z^2 / (2 * n) + z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d) })(events[i], n[i])
  cat(sprintf("ARM|%s|%d|%d|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f\n", arm[i], n[i], events[i], 100 * events[i] / n[i],
              100 * ci[1], 100 * ci[2], 100 * wi[1], 100 * wi[2], 100 * cr[1], 100 * cr[2]))
}

ARM  Usual Care  1004  112  11.2  9.3  13.3  9.4  13.3  9.4  13.3
ARM  Clinician Nudge  1179  136  11.5  9.8  13.5  9.8  13.5  9.8  13.5
ARM  Patient Nudge  997  115  11.5  9.6  13.7  9.7  13.7  9.7  13.7
ARM  Clinician and Patient Nudge  1270  179  14.1  12.2  16.1  12.3  16.1  12.3  16.1

11. Multiplicity, Missing Data and Analysis Populations

The registry lists a primary endpoint and three secondary outcomes. When several outcomes or contrasts are tested, the chance of at least one apparently notable result can increase unless the analysis plan addresses multiplicity. A plan might distinguish a single primary comparison from exploratory comparisons, or use an adjustment for multiple testing. The registry record does not report a multiplicity strategy, a hierarchy among comparisons, or which contrasts were designated primary.

The four-arm factorial structure also allows multiple possible comparisons: each intervention condition can be compared with usual care, and component or combined-strategy effects may be examined. Which of these comparisons the investigators intended to emphasize, and how any multiplicity was handled, are not reported in the registry record. The existence of multiple possible comparisons should not be mistaken for evidence that each was conducted.

Missing-data procedures and analysis populations are also not reported. The endpoint is based on electronic record documentation, but the record does not explain how patients with incomplete records, uncertain documentation status, or unavailable follow-up were classified. It does not report participant flow by arm, exclusions, or whether the sponsor used an intention-to-treat or another analysis population.

The description says outcomes were measured for patients only, but that does not establish how the cluster assignment and analysis were connected, or how missing outcomes were handled. These details are important for assessing whether the analysis preserved the randomized comparison and whether uncertainty appropriately reflected clustering. Without the sponsor’s analysis and relevant protocol details, the effects of missingness and multiplicity cannot be evaluated from the registry record.

12. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry records zero serious adverse events for usual care, clinician nudge, patient nudge, and clinician and patient nudge. The corresponding numbers at risk are 1004, 1179, 997, and 1270, respectively. The record also lists zero in the other adverse-event count for each arm. No top serious-event terms are listed.

These are registry event counts, not an analysis of comparative safety. The record does not report event definitions, collection procedures, follow-up completeness, or a statistical comparison across arms. A zero count in the posted record does not show that no event of any kind occurred; it indicates that no events are recorded in these categories in the registry results.

Deaths are reported separately: 162 for usual care, 227 for clinician nudge, 157 for patient nudge, and 227 for clinician and patient nudge. The registry does not specify in these entries whether deaths were attributed to an intervention or how they relate to serious adverse-event reporting. They should therefore be read as separate reported counts, not added to or treated as serious adverse events, and not interpreted as evidence about the primary documentation outcome.

13. What the Result Does, and Does Not, Mean

This trial addresses an implementation question: whether nudges directed to clinicians, patients, or both are associated with documentation of serious illness conversations for high-risk patients with cancer. Its primary endpoint is clearly tied to a specified electronic-record note and a 6-month timeframe. The design also recognizes that implementation strategies may involve different recipients and may be combined.

The registry record does not report the primary outcome result or the sponsor’s analysis. Therefore, it does not show whether documentation increased, decreased, or was similar across the arms. It also does not establish whether the combined strategy was more effective than either individual strategy. The stated hypotheses are expectations, not results.

Even if an outcome estimate were available, documentation would remain a record-based endpoint. It would not by itself establish the number or quality of conversations that took place, patient understanding, or the effects of conversations on care. The secondary outcomes could address other aspects of care, but their results are not reported in the registry record.

The comparator also shapes what any eventual result would mean. Usual care already included targeted clinician messages about high-risk patients, while the clinician-nudge arm added peer-comparison feedback. The trial therefore does not represent a simple test of clinician alerts versus no alerts. The patient-facing message and combined strategy are also specific interventions, so any findings would apply most directly to those approaches and the trial’s setting.

Finally, cluster randomization makes the analysis approach consequential. The registry itself cautions against a simple patient-level re-analysis, while not reporting the sponsor’s results. Without the sponsor’s cluster-aware analysis, estimates and uncertainty cannot be evaluated. The trial’s completion status and planned enrollment are not substitutes for those results, and the safety counts do not answer the efficacy question.

14. Why This Trial Matters Statistically

The trial is statistically notable for combining a factorial intervention structure with cluster randomization. It examines nudges directed to two different audiences—clinicians and patients—and includes a combined condition. This structure can help distinguish questions about separate components from questions about using them together, provided the analysis specifies and estimates the relevant contrasts.

Cluster allocation is important because patient outcomes may not be independent when patients share clinical teams or workflows. The registry explicitly cautions that a simple individual-level re-analysis would be misleading. This highlights a general issue in implementation research: the unit that receives an intervention and the unit used to measure an outcome may differ, so the analysis must reflect the design.

The comparator is also meaningful. Usual care already included targeted text messages identifying high-risk patients, while the clinician-nudge condition added performance feedback. Recognizing this distinction prevents overgeneralizing a potential contrast as the effect of all clinician prompting. The patient and combined conditions likewise represent defined strategies rather than abstract categories of behavioral intervention.

At the same time, statistical interpretation depends on reported estimates, uncertainty, and analysis details. The registry record does not report the sponsor’s outcome analysis, the cluster allocation unit, or the multiplicity and missing-data plans. As a result, the design raises useful methodological questions, but the posted record does not establish the size, precision, or direction of an intervention effect. The distinction between a carefully structured trial and a reported, interpretable result is central to reading this record.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The main limitation of the registry record for interpretation is the absence of primary and secondary outcome results and the sponsor’s analysis. Although the study is marked completed, no effect estimates or uncertainty measures are reported. The brief summary gives hypotheses but cannot substitute for outcome findings.

The cluster-randomized design is identified, but the registry does not report the cluster unit, number of clusters, or the sponsor’s method for accounting for within-cluster correlation. These details are central to understanding precision and the validity of comparisons. The record specifically cautions that simple individual-level re-analysis would be misleading.

The record identifies a factorial model but does not describe the planned contrasts or whether the interaction between clinician and patient nudges was assessed. It also does not report a multiplicity approach despite listing several outcomes and possible arm comparisons. Missing-data handling, participant flow, and analysis populations are not reported, limiting assessment of how complete outcome data and exclusions were addressed.

Intervention delivery and receipt are not described through fidelity or adherence results. The registry also does not identify who was masked under the single-masking designation. The patient-level endpoint is based on documentation of a template in the medical record; the record does not establish whether all conversations were documented in that form or whether documentation reflects conversation quality.

Finally, usual care included targeted clinician messages, which is important context for interpreting the clinician strategy. The registry describes planned sample-size assumptions as a planning illustration rather than the trial’s own design, but does not report the actual sample-size justification. These limitations concern what can be concluded from the registry record and do not establish that the trial itself failed to collect or analyze the relevant information.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Effect of Behavioral Nudges on Serious Illness Conversation Documentation — NCT04867850.
• Registry record last updated 2024-04-18; status last verified 2024-03.
• Results first posted 2024-04-18.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

NCT04867850 is a completed, randomized, single-masked, factorial, pragmatic cluster trial sponsored by Abramson Cancer Center at Penn Medicine. The registry lists planned enrollment of 4450, with a study start date of 2021-09-09 and a primary completion date of 2022-09-09. The condition is cancer, and the phase is listed as NA.

The four study arms were usual care, clinician nudge, patient nudge, and clinician and patient nudge. The primary outcome was the number of high-risk patients with documented SIC within 6 months of the index visit, measured through a qualifying electronic-record note. Three secondary outcomes concerned documentation among all patients, high-risk patients’ palliative care referrals, and aggressive end-of-life care among decedent high-risk patients.

The registry record does not report the sponsor’s outcome analysis, primary endpoint results, or statistical analysis details. It lists zero serious adverse events in each arm and separately reports deaths by arm. These entries do not replace the missing outcome analysis.