Project to Improve Communication About Serious Illness - Pilot Study: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Project to Improve Communication About Serious Illness - Pilot Study (NCT03746392), a randomized na study. The primary outcome is Documentation of Goals of Care.

2. Clinical Question

3. Trial Design

This was a randomized, parallel-group pilot study with single masking. The registry lists the phase as NA and the status as completed. The study title is Project to Improve Communication About Serious Illness - Pilot Study; its official title is Using the Electronic Health Record to Identify and Promote Goals-of-Care Communication. The University of Washington is listed as the sponsor.

The registry records enrollment of 150. The posted arm counts are 75 for the Jumpstart Intervention and 75 for usual care. Allocation is described as randomized, but the sequence-generation method, allocation concealment, and practical details of assignment are not reported in the registry record. Single masking is recorded, but the identity of the masked party or parties and how masking was maintained are not reported.

The intervention was a behavioral intervention centered on the one-page Jumpstart Form. According to the registry description, the form used questionnaire information about preferences for goals-of-care communication and care, and included guidance intended to help patients or surrogates talk with clinicians. The comparator was usual inpatient hospital care; details beyond that label are not reported.

The listed study dates are a start date of 2018-07-01 and a primary completion date of 2020-06-30. The registry describes the project as a two-year pilot and says it was based on prior use of the form in an outpatient clinic setting. The registry record does not report the original sample-size justification, design assumptions, or whether the pilot was intended to estimate a particular effect size. It also does not describe site count, recruitment procedures, or additional trial procedures.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Documentation of Goals of Care, measured over At hospital discharge, an average of 2 weeks.

The primary endpoint was the presence of documentation of goals-of-care discussions in the patient’s electronic health record during the index hospitalization. It was recorded as a count of participants with the event. The stated assessment time was hospital discharge, described as an average of 2 weeks.

This endpoint captures whether a discussion was documented, rather than independently establishing whether a conversation occurred, how fully it addressed the patient’s preferences, or whether care later matched those preferences. Documentation is a practical process measure: it can indicate whether goals-of-care communication is represented in the clinical record and may support communication among care teams. Its interpretation is nevertheless limited to the recorded outcome defined by the registry.

The posted event counts are 16 in the Jumpstart Intervention arm and 6 in the Usual Care arm, with 75 participants listed in each arm. These are participant counts, not scores or repeated-event totals. The record does not report a secondary outcome measure. It also does not describe how documentation was identified, whether assessors were masked to assignment, or how incomplete or unavailable records were handled. Those details matter when judging how consistently this endpoint was ascertained.

7. Results Posted in the Registry

• Jumpstart Intervention: 21.3% (16/75) versus 8.0% (6/75) on control; risk difference 13.3 percentage points (95% CI 2.2 to 24.5), risk ratio 2.67 (1.10–6.44); unadjusted two-proportion z-test p = 0.021.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary outcome is binary: for each participant, the record indicates whether documentation of a goals-of-care discussion was present during the index hospitalization. The posted arm summaries report event counts and arm sizes. For such an outcome, a group comparison can be expressed as a difference in risks, a ratio of risks, or a ratio of odds. These measures answer related but different questions, so their numerical values are not interchangeable.

What does the risk difference describe?

The recalculated risk difference is 0.133. It is the unadjusted difference between the groups’ event risks on the scale used for the calculation. A positive value means the estimated event risk is higher in the intervention group than in the usual-care group. The interval reported for this estimate runs from 0.022 to 0.244. The registry record does not state the confidence level attached to this interval, so it is best read as the posted uncertainty interval without assigning it an unreported level.

As a general statistical explanation, a risk difference is often useful because it expresses the contrast in absolute terms on the outcome’s probability scale. Its interpretation depends on how the event is defined and on the population to which the estimate applies. Here the event is documentation in the electronic health record, not a broader measure of communication quality.

How do relative measures differ?

The recalculated risk ratio is 2.667, with an interval from 1.104 to 6.442. The risk ratio compares the event risk in one group with the event risk in the other. The recalculated odds ratio is 3.119, with an interval from 1.147 to 8.482. An odds ratio compares odds rather than risks. When an outcome is not rare, these measures can differ in magnitude; that is a general property of the measures, not a further result about this trial. Both are unadjusted comparisons from the posted arm summaries.

The registry does not report an adjusted effect estimate or covariates for adjustment. The recalculated estimates should therefore be understood as unadjusted comparisons, not as estimates controlling for participant or clinical characteristics. Random allocation is recorded, but details of randomization and the analysis population are not available in the registry record.

What does a p-value tell the reader?

The sponsor’s reported method is Fisher Exact, with p = 0.036. The recalculated unadjusted result is p = 0.021. Fisher’s exact test is a method for assessing evidence against a null association in a two-group table of categorical outcomes. The registry labels its analysis as superiority. The record does not provide additional methodological detail about the sponsor’s test beyond the method name and p-value.

As a general statistical explanation, a p-value is calculated under a null model and describes how unusual data at least as incompatible with that model would be if the model and test assumptions held. It is not the probability that a treatment effect is real, and it is not a measure of the effect’s size. A small p-value alone does not show that a finding is clinically important or free of bias. The effect estimate and its interval address magnitude and precision; design and measurement details inform the credibility of the comparison.

Why might two analyses produce different p-values?

The recalculated p-value and registry p-value are not identical: the recalculation is described as unadjusted and uses the posted summary statistics, while the registry reports Fisher Exact. The record does not explain the calculation details behind the recalculated result or provide further specifications for the sponsor analysis. The difference should therefore be reported plainly rather than attributed to an undocumented cause. The registry’s result remains the sponsor’s reported analysis; the recalculation is a separate summary-statistic-based comparison.

For binary outcomes, over-dispersion is not the central issue in the way it can be for count outcomes modeled with a Poisson distribution. More generally, over-dispersion means observed variability exceeds what a model assumes, which can make model-based standard errors too small if ignored. That general consideration does not establish that over-dispersion occurred here, and the registry does not report a model requiring that assessment.

All recalculated measures are unadjusted. No additional primary or secondary outcome results are posted in the registry record, and no analysis plan, multiplicity adjustment, missing-data strategy, or analysis-population definition is reported. These omissions constrain interpretation of the inferential results, even though the endpoint counts and the sponsor’s test are available.

10. Confidence Intervals and Precision

The risk-difference estimate is 0.133, with a reported interval from 0.022 to 0.244. The interval does not include zero, the value that would represent no difference on the risk-difference scale. This is a feature of the posted interval and does not, by itself, resolve questions about measurement, bias, or how broadly the result applies. The confidence level for this interval is not reported in the registry record.

The relative estimates also have intervals that remain above the no-association value of one: the risk ratio is 2.667, with an interval from 1.104 to 6.442; the odds ratio is 3.119, with an interval from 1.147 to 8.482. The intervals are wide enough to indicate uncertainty about the magnitude of the relative contrast. Their scale differs from the risk-difference scale, so the endpoints should not be compared as though they represented the same quantity.

As a general statistical explanation, an interval describes a range of effect estimates compatible with the observed data and the method’s assumptions at a stated confidence level. A narrower interval indicates greater precision under those assumptions; a wider interval indicates less precision. An interval is not a guarantee that the true value lies within its limits, nor does it account automatically for bias, outcome misclassification, missing data, or limitations in the study design.

The registry’s own Fisher Exact analysis reports p = 0.036 but does not report an effect estimate or interval for that analysis. The unadjusted recalculation supplies estimates from the posted counts, but it does not replace the sponsor’s result. The registry record does not report a confidence level for the recalculated intervals, so precision should be described using the interval limits without asserting a specific confidence percentage.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are illustrations based on the observed effect, not the original trial’s documented design assumptions. The listed assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is set to a stated fraction of the observed effect. The registry record does not report the trial’s own original sample-size assumptions or justification.

For this binary endpoint, the planning figures are participants per arm, not total events. The distinction matters: the registry notes that planning figures for time-to-event outcomes are expressed as total events, but this is not a time-to-event outcome. The curve and table should be read as planning illustrations for different target effect sizes, not as descriptions of the number of people actually enrolled or outcomes observed.

The 80% power column gives the illustrated participants per arm for a design targeting an 80% chance of detecting the specified effect under the stated assumptions, if that effect and those assumptions were correct. The 90% power column gives the corresponding planning figure for a 90% chance. These are design probabilities under assumptions, not guarantees about a future study and not estimates of the chance that the intervention’s effect is real.

The rows use 100% of the observed effect, 75% of the observed effect, and 50% of the observed effect. At 80% power, the listed figures are 110 participants per arm, 177 participants per arm, and 354 participants per arm, respectively. At 90% power, they are 146 participants per arm, 236 participants per arm, and 474 participants per arm, respectively. Smaller effects need more participants to detect under the same planning framework, which is why the figures rise as the target effect becomes a smaller fraction of the observed effect.

These are not observed power calculations. They are planning illustrations derived from the observed result and the stated assumptions. Because the observed effect itself is uncertain, planning around it can produce an overly optimistic or otherwise unstable target. The registry does not report a separate design-stage effect assumption, and these figures should not be presented as evidence about the trial’s original power.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.080; treatment proportion p₁ = 0.213; average p̄ = 0.147.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.5003 = 0.9806.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.4913 = 0.4135.
4. n = (0.9806 + 0.4135)² / (0.1333)² = 109.3, rounded up to 110 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.08                          # control proportion
p1 <- 0.213333333333333                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3746392)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 110 (80% power), 146 (90% power)
 75% of observed effect: n per arm = 177 (80% power), 236 (90% power)
 50% of observed effect: n per arm = 354 (80% power), 474 (90% power)
power.prop.test: 109.3 
Simulated power at n = 110 per arm: 0.822 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Analyses Reported by the Sponsor

The registry reports the sponsor’s primary analysis as a Fisher Exact test, with p = 0.036, labeled as a superiority analysis. It does not provide an effect estimate or interval for that reported test. The separate recalculation from the posted group counts gives a risk difference of 0.133, with an interval from 0.022 to 0.244, and an unadjusted p-value of 0.021.

The two p-values are different, and the record does not give enough detail to attribute the difference to a particular computational choice beyond identifying the sponsor’s test as Fisher Exact and the recalculation as an unadjusted calculation from the posted summaries. They should be presented as separate results. The recalculation adds an effect-size description, but it is not a replacement for the sponsor’s reported analysis. The registry record does not report further specifications of the sponsor’s analysis or an explanation for the difference.

14. Multiplicity, Missing Data and Analysis Populations

The registry lists one primary outcome measure and one reported analysis. It lists no posted secondary outcomes. This limits the multiplicity questions that can be assessed from the record, but it does not establish that no additional outcomes, analyses, or hypotheses were considered during the study. Whether any adjustment for multiple testing was planned or applied is not reported in the registry record.

The record does not report a statistical analysis plan, a prespecified analysis population, or whether the primary analysis followed an intention-to-treat approach. It also does not state whether participants were analyzed according to assignment, whether any participants were excluded from analysis, or how deviations from assigned intervention were handled. Random allocation is recorded, but that information alone does not establish the analysis approach.

Missing-data procedures are not reported. The registry does not state whether any electronic health records or outcome assessments were unavailable, whether missing outcome status was possible, or how incomplete data would be treated. For a documentation endpoint, absence of a recorded discussion could mean no discussion was documented; it should not automatically be interpreted as proof that no conversation occurred. The registry description defines the endpoint by the presence of documentation, but does not give additional rules for classifying records.

The process used to locate or verify documentation, the role of assessors, and whether outcome ascertainment was blinded to assignment are also not reported. Single masking is listed for the study, but the masked party is not identified. These gaps make it difficult to assess the extent to which ascertainment could have differed between groups.

As a general statistical point, multiplicity can increase the chance of obtaining at least one apparently unusual result when many outcomes or analyses are tested. Adjustments or prespecified testing strategies can address that concern, but the registry record does not say whether either was used. With one listed primary outcome and one posted analysis, the available record supports describing the reported result, not making broader claims about the full analysis process.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The serious adverse event summary lists 0 serious adverse events in the Jumpstart Intervention group and 0 in the Usual Care group. The listed numbers at risk for serious events are 75 in each group. The registry also lists 0 other adverse events and 0 deaths in each group, with 75 at risk for other adverse events in each group. No serious adverse event terms are listed.

These entries should be read as the registry’s posted counts for the adverse-event fields, not as proof that no harms of any kind occurred. The record does not describe the adverse-event collection period, event definitions, ascertainment procedures, or whether events outside the listed categories were collected. Zero reported events in these fields does not establish equivalent safety between groups or rule out uncommon events. The posted information is limited to the listed counts and does not support a broader safety conclusion.

16. What the Result Does, and Does Not, Mean

The result concerns a specific process outcome: whether a goals-of-care discussion was documented in the electronic health record during the index hospitalization. The posted counts and the unadjusted estimates show a higher recorded event frequency in the intervention group than in the usual-care group. The registry’s own Fisher Exact analysis reports p = 0.036, while the summary-statistic recalculation reports p = 0.021. The recalculated risk difference and its interval describe the magnitude and uncertainty of that unadjusted comparison.

These findings do not show whether the discussions were more complete, whether patients’ preferences were accurately captured, or whether subsequent care aligned with those preferences. The endpoint records documentation, not the content or quality of the conversation. It also does not measure patient experience, health status, or other outcomes that are not posted in the registry record.

The study is described as a randomized pilot, but details about sequence generation, concealment, masking roles, outcome ascertainment, missing data, and the analysis population are not reported. Those details affect how confidently a group difference can be attributed to the intervention and how widely it can be applied. The conditions listed in the registry indicate the clinical context, but the record does not report the distribution of participants across those conditions or establish that the results apply to every listed group.

The planning figures based on the observed effect are not evidence of observed power and do not establish what a future study would find. Likewise, the absence of reported serious adverse events in the listed fields should not be interpreted as proof of no risk. The registry result is best understood as evidence about the recorded documentation outcome in this pilot, interpreted alongside uncertainty in the effect estimate and gaps in the available methodological detail.

17. Why This Trial Matters Statistically

This trial is statistically notable because its primary endpoint is a documented process of care rather than a clinical event such as disease progression or survival. That distinction shapes the interpretation: the measured outcome is whether a goals-of-care discussion appeared in the electronic health record during the index hospitalization. The result therefore bears on recorded communication, while leaving the content and consequences of communication outside the measured endpoint.

The posted counts allow the comparison to be expressed in both absolute and relative terms. The risk difference, risk ratio, and odds ratio each describe a different scale, and the interval estimates make clear that the estimated contrast is uncertain. Reporting more than a p-value helps readers distinguish evidence against a null model from the size and precision of the observed difference.

The pilot context also matters. Planning illustrations based on the observed effect show how sample-size requirements vary when a study targets a smaller fraction of that effect. They are not a substitute for a design-stage assumption, which is not reported in the registry record, and are not observed power. This is a useful distinction when reading preliminary estimates that may be imprecise.

Finally, a record-based endpoint depends on documentation and ascertainment. The registry does not report detailed procedures for identifying discussions, handling missing information, or defining the analysis population. Those omissions do not negate the posted result, but they limit what can be learned from it. A careful statistical reading keeps the outcome definition, uncertainty, sponsor analysis, and reporting gaps in view rather than treating a single p-value as a complete account.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record does not report detailed eligibility criteria, recruitment procedures, site count, or the distribution of participants across the listed conditions. Although the study is described as randomized and parallel, sequence generation and allocation concealment are not described. Single masking is recorded, but the masked party and masking procedures are not identified.

The endpoint is defined through electronic health record documentation. The record does not explain how documentation was located, classified, or verified, whether assessors were masked, or how an absent or incomplete record was handled. Documentation status may not capture every conversation, and the endpoint does not assess the quality or effect of a discussion.

The registry does not report a statistical analysis plan, original sample-size assumptions, analysis population, missing-data approach, or any adjustment for multiple testing. It reports one primary outcome and one analysis, with no secondary outcomes posted. The sponsor’s test is identified as Fisher Exact, but further analysis specifications and an effect interval for that test are not reported.

The study is identified as a pilot, and the recalculated planning illustrations rely on the observed effect. Such illustrations depend on assumptions and should not be taken as a design justification or a forecast of a future result. The record also provides only the listed adverse-event counts, without detailed definitions or ascertainment periods. These trial-specific limitations leave uncertainty about measurement, implementation, and generalizability beyond the posted outcome comparison.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: Project to Improve Communication About Serious Illness - Pilot Study — NCT03746392.
• Registry record last updated 2025-08-22; status last verified 2025-08.
• Results first posted 2025-08-22.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

NCT03746392 is a completed randomized, single-masked, parallel-group pilot sponsored by the University of Washington. It compares the Jumpstart Intervention with Usual Care for documentation of goals-of-care discussions in the electronic health record during the index hospitalization. Enrollment is recorded as 150, with 75 participants listed in each arm.

The primary outcome counts are 16 in the intervention arm and 6 in the usual-care arm. The registry reports a Fisher Exact analysis with p = 0.036. An unadjusted recalculation from the posted summaries gives a risk difference of 0.133, with an interval from 0.022 to 0.244, and p = 0.021. The endpoint measures documentation, and the registry does not report several details about outcome ascertainment, missing data, or analysis populations.

The registry lists 0 serious adverse events, 0 other adverse events, and 0 deaths in each group. These counts describe the posted adverse-event fields and do not establish the absence of all harms.