This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for Evaluation of Methods for Implementation of a Comfort Care Order Set (NCT02383173), a randomized na study. The primary outcome is Presence of an Active Opioid Order at Time of Death.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT02383173 |
| Phase | NA |
| Condition | Terminal Conditions, End of Life |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | None (open label) |
| Enrollment | 134 |
| Arms in the primary outcome | 2 |
| Lead sponsor | VA Office of Research and Development |
| Results posted | Yes |
2. Clinical Question
Population
The primary outcome population was Veterans who died as inpatients in acute care units of VA Medical Centers. The outcome timeframe was death among Veterans who died in the 9 months following the intervention period. The registry enrollment is 134, while outcome denominators are 1519 in the Basic Implementation Approach and 1687 in the Enhanced Implementation Approach. The registry record does not explain the relationship between these figures or identify the unit randomized.
Intervention
The Enhanced Implementation Approach used the infrastructure of Palliative Care Consult Teams and included in-person train-the-champion workshops. The intended role of the workshops was to prepare team members to lead and train others at their home sites.
Comparator
The Basic Implementation Approach also used Palliative Care Consult Teams, with a teleconference to review educational materials and activate the teams to educate other providers. The comparison is between implementation strategies, not a comparison of an education intervention against no education.
Primary question
Among Veterans in the stated inpatient death population, did the implementation approaches differ in the presence of an active opioid order at the time of death? The posted primary outcome counts and the recalculated unadjusted comparison suggest a difference, but the registry’s reported GEE analysis has a different p-value. The sponsor’s reported analysis takes precedence over the recalculation when interpreting that disagreement.
3. Trial Design
The registry describes a randomized, parallel trial with no masking. The two approaches were the Basic Implementation Approach and the Enhanced Implementation Approach. Both were designed to implement a Comfort Care Education Intervention through the established infrastructure of Palliative Care Consult Teams. The stated contrast is therefore between two ways of supporting implementation: a teleconference-based approach and an approach that added in-person train-the-champion workshops.
The record identifies the study as completed, with a start date of 2015-03-01 and a primary completion date of 2019-04-30. It lists enrollment as 134. For the primary outcome, however, the registry reports outcome denominators of 1519 for the Basic Implementation Approach and 1687 for the Enhanced Implementation Approach. Those denominators refer to Veterans in the outcome counts, not the listed enrollment figure. The registry record does not explain how the enrollment figure relates to the outcome population or why those denominators are larger.
The primary outcome was assessed among Veterans who died as inpatients in acute care units of VA Medical Centers, with the timeframe defined as death in the 9 months following the intervention period. The unit of randomization, allocation procedures, and whether the outcome population was defined before or after randomization are not reported in the registry record. No further detail about masking is listed beyond the record’s designation of none.
The brief description frames the program as an effort to transfer home hospice and palliative-care practices for the last days and hours of life into the inpatient setting. It also describes the intent to evaluate implementation processes and refine the intervention and strategies for possible wider use within the VA Healthcare System. The registry lists secondary outcomes concerning medication orders, care orders, location of death, devices, restraints, and pastoral care. Numerical results are posted for only some of those outcomes.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: Eligibility of VAMCs and Palliative Care Consult Team members were determined before enrollment. All those enrolled teams were randomized.
Overall Study
| Stage | Basic Implementation Approach | Enhanced Implementation Approach |
|---|---|---|
| STARTED | 88 | 46 |
| COMPLETED | 88 | 44 |
| NOT COMPLETED | 0 | 2 |
Reasons for not completing this period:
| Reason | Basic Implementation Approach | Enhanced Implementation Approach |
|---|---|---|
| Palliative Care Team withdrew | 0 | 2 |
Analysis population for the primary outcome, as defined in the registry: Veterans who died as inpatients in acute care units of VA Medical Centers
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
Population: The Overall Number of Baseline Participants is greater than the number Started in the Participant Flow. The participant flow numbers describe the study participants, VA providers who participated in the education intervention. Data in Baseline Characteristics describe the patients whose medical records were abstracted to measure outcomes.
| Characteristic | Unit | Basic Implementation Approach | Enhanced Implementation Approach | Total |
|---|---|---|---|---|
| Participants analysed | 3078 | 3413 | 6491 | |
| Age, Categorical: <=18 years | Participants | 0 | 0 | 0 |
| Age, Categorical: Between 18 and 65 years | Participants | 645 | 557 | 1202 |
| Age, Categorical: >=65 years | Participants | 2433 | 2855 | 5288 |
| Age, Continuous | years | 74.2 (SD 11.6) | 75.6 (SD 11.4) | 75.0 (SD 11.5) |
| Sex: Female, Male: Female | Participants | 77 | 88 | 165 |
| Sex: Female, Male: Male | Participants | 3001 | 3325 | 6326 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 11 | 20 | 31 |
| Race (NIH/OMB): Asian | Participants | 15 | 14 | 29 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 33 | 16 | 49 |
| Race (NIH/OMB): Black or African American | Participants | 659 | 751 | 1410 |
| Race (NIH/OMB): White | Participants | 2096 | 2415 | 4511 |
| Race (NIH/OMB): More than one race | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 264 | 197 | 461 |
| Region of Enrollment: United States | Participants | 3078 | 3413 | 6491 |
6. Primary Endpoint
The registry lists the primary outcome as Presence of an Active Opioid Order at Time of Death, measured over at time of death for veterans who died in the 9 months following the intervention period.
| Element | Registry entry |
|---|---|
| Result type | Count Of Participants |
| Measure of spread | not applicable |
| Unit | Participants |
| Endpoint classification | Binary Endpoint |
The primary endpoint is the presence of an active opioid order at the time of death, recorded in the Veteran’s medical record. It counts participants with an active order; it does not measure opioid administration, dose, symptom relief, or the quality of the order. Its stated population is Veterans who died as inpatients in acute care units of VA Medical Centers, and its timeframe is death in the 9 months following the intervention period.
This endpoint is relevant to the implementation question because an order in the medical record is an observable marker of whether a specified element of care was documented. It can be counted across the two implementation approaches and compared as a binary outcome: a participant either had the order recorded at death or did not. That makes the endpoint directly interpretable as a difference in the share of participants with the recorded order, while leaving the actual clinical context and use of the medication unmeasured.
The endpoint should therefore be read narrowly. A higher recorded prevalence does not, by itself, establish how an order was used, whether the Veteran received the medication, or whether care was better for an individual. The registry describes the measure and population, but further details about record abstraction, assessment reliability, and handling of records with unavailable or ambiguous information are not reported in the registry record.
7. Results Posted in the Registry
- Enhanced Implementation Approach: 91.5% (1543/1687) versus 87.6% (1330/1519) on control; risk difference 3.9 percentage points (95% CI 1.8 to 6.0), risk ratio 1.04 (1.02–1.07); unadjusted two-proportion z-test p = <0.001.
Risk difference: Enhanced Implementation Approach
95% CI: 1.8 to 6.0 percentage points
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Presence of an Order for Benzodiazepine Medication | Binary Endpoint | Risk difference | -1.3 pts (-4.5 to 2.0) | 21,692 per arm |
| Presence of a Do-not-resuscitate Order | Binary Endpoint | Risk difference | +1.2 pts (-1.0 to 3.4) | 11,055 per arm |
| Presence of a Palliative Care Consult Order | Binary Endpoint | Risk difference | +5.0 pts (2.3 to 7.7) | 924 per arm |
Presence of an Order for Benzodiazepine Medication
- Endpoint type: Binary: the number of participants with the event, compared as proportions.
- Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
- Result:
- Enhanced Implementation Approach: 66.2% (1117/1687) versus 67.5% (1025/1519) on control; risk difference -1.3 percentage points (95% CI -4.5 to 2.0), risk ratio 0.98 (0.93–1.03); unadjusted two-proportion z-test p = 0.447.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Enhanced Implementation Approach and the control. The interval gives the range of effects that remain compatible with the data.
Presence of a Do-not-resuscitate Order
- Endpoint type: Binary: the number of participants with the event, compared as proportions.
- Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
- Result:
- Enhanced Implementation Approach: 89.2% (1504/1687) versus 88.0% (1336/1519) on control; risk difference 1.2 percentage points (95% CI -1.0 to 3.4), risk ratio 1.01 (0.99–1.04); unadjusted two-proportion z-test p = 0.286.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Enhanced Implementation Approach and the control. The interval gives the range of effects that remain compatible with the data.
Presence of a Palliative Care Consult Order
- Endpoint type: Binary: the number of participants with the event, compared as proportions.
- Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
- Result:
- Enhanced Implementation Approach: 84.5% (1426/1687) versus 79.5% (1208/1519) on control; risk difference 5.0 percentage points (95% CI 2.3 to 7.7), risk ratio 1.06 (1.03–1.10); unadjusted two-proportion z-test p = <0.001.
- Interpretation:
- The 95% confidence interval excludes no difference, so the proportion with the event is statistically higher for Enhanced Implementation Approach than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
The posted numerical secondary results show a mixed pattern rather than a uniform shift across every care-process measure. The recalculated comparisons indicate a positive difference for one order-related outcome, while the other two detailed comparisons do not show comparably clear evidence of a difference. This broad pattern is not identical to the primary recalculation, but it is compatible with the possibility that different recorded care processes need not move together.
These secondary findings provide context for the primary endpoint, but they do not resolve the disagreement between the primary unadjusted recalculation and the sponsor’s GEE result. The secondary analyses are recalculations from posted counts, and no sponsor analyses are listed for them. They should not be treated as independent confirmation of the primary comparison or as a substitute for the sponsor’s primary analysis.
Multiplicity is important when interpreting several endpoints. As a general statistical concept, examining multiple outcomes creates more opportunities for a nominally notable result to occur by chance, even when no multiplicity adjustment is used. The registry does not report an adjustment or a prespecified hierarchy for these secondary outcomes. Consequently, unadjusted secondary p-values carry less weight than a result supported by a prespecified primary analysis, particularly when several outcomes are being considered together.
The registry lists additional secondary outcomes concerning location of death, a nasogastric tube, an intravenous line, restraints, and a pastoral care visit. Numerical results for those outcomes are not reported in the registry record. Accordingly, the overall pattern across all listed secondary outcomes is incomplete, and no broad conclusion about the full set of measures can be drawn from the posted numerical results.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Presence of an Order for Benzodiazepine Medication
- Presence of a Do-not-resuscitate Order
- Presence of a Palliative Care Consult Order
- Location of Death
- Presence of a Nasogastric Tube
- Presence of an Intravenous Line
- Presence of Restraints
- Presence of a Pastoral Care Visit
9. Statistical Interpretation
For the primary outcome, the Basic Implementation Approach had 1330 participants with an active opioid order among 1519, and the Enhanced Implementation Approach had 1543 among 1687. The recalculated unadjusted risk difference is 0.039, with a 95% confidence interval from 0.018 to 0.060 and p < 0.001. In this simple comparison, the estimate is positive, meaning the recorded outcome was more common in the Enhanced Implementation Approach group.
That recalculation does not match the sponsor’s reported analysis. The registry reports a Generalized Estimating Equation analysis with p = 0.445. Because the registry record marks a discrepancy between the recalculation and the sponsor analysis, the sponsor’s own analysis takes precedence. The posted GEE result does not show evidence of a statistically significant difference at conventional thresholds, while the recalculation treats the counts as an unadjusted comparison. The registry does not report enough details about the GEE analysis to determine why its result differs, so the two results should not be blended into a single conclusion.
The recalculation is based on the posted counts and is not a replacement for the sponsor’s analysis. Its interval quantifies uncertainty under the simple unadjusted calculation, not necessarily the uncertainty under the sponsor’s analysis. The GEE p-value is reported without an associated estimate or confidence interval in the registry record, limiting the ability to assess the magnitude and precision of that analysis from the posted information alone.
For the secondary outcomes with numerical results, the overall pattern is mixed. The recalculated comparisons show a positive difference for the presence of a Palliative Care Consult Order, while the comparisons for a benzodiazepine order and a do-not-resuscitate order do not provide similarly clear evidence of a difference. These are unadjusted secondary comparisons. The registry does not list sponsor analyses for those outcomes, and the multiplicity of outcomes makes isolated unadjusted findings less conclusive.
The outcomes are documentation or care-process measures, and their interpretation depends on the population and record-based definitions. They do not establish that the implementation strategy caused a change in individual care, nor do they establish effects on patient-centered outcomes. The registry does not report estimates for all listed secondary outcomes, so the overall pattern across the full outcome set cannot be fully characterized from the record.
10. Statistical Methods Explained
The primary outcome is binary: each participant is counted according to whether an active opioid order was present at death. The registry posts the number with the order and the number in each approach group. A straightforward comparison of the resulting proportions can be expressed as a risk difference, which is the difference between the proportion in the Enhanced Implementation Approach and the proportion in the Basic Implementation Approach. The recalculated risk difference is 0.039. A positive value indicates a larger observed proportion in the enhanced group in this unadjusted comparison.
What does the risk difference convey?
As a general statistical concept, the risk difference keeps the comparison on the original probability scale. It describes the absolute separation between two group proportions, rather than how many times as large one proportion is relative to the other. Its confidence interval gives a range of values compatible with the observed data under the assumptions of the calculation. The primary recalculated interval is 0.018 to 0.060. Because it does not include zero, it corresponds to the recalculated p-value of p < 0.001 for this simple comparison. This is not the sponsor’s GEE estimate or interval; neither an estimate nor confidence interval for the sponsor’s analysis is reported in the registry record.
How do relative measures differ?
The recalculated comparison also includes relative measures: a risk ratio of 1.045 with an interval from 1.02 to 1.07, and an odds ratio of 1.523 with an interval from 1.211 to 1.914. As a general statistical concept, a risk ratio compares proportions as a ratio, while an odds ratio compares odds. These measures use different scales from the risk difference, so their numerical values are not directly interchangeable. For a common outcome such as the primary outcome, the odds ratio can be farther from 1 than the risk ratio is from 1. The registry’s sponsor analysis is reported as a GEE p-value, not as one of these recalculated measures.
What does GEE add?
The registry identifies Generalized Estimating Equation (GEE) as the sponsor’s method for the primary comparison. As a general statistical concept, GEE can be used for outcomes that may be correlated within groups or clusters, and it estimates an average association while accounting for a chosen correlation structure. The record does not report the correlation structure, working model, covariates, effect measure, standard error method, or analysis population for this GEE result. It therefore is not possible to reconstruct its estimate or explain exactly why its p-value differs from the unadjusted recalculation.
The distinction matters because the posted recalculation treats the arm-level counts as a simple, unadjusted comparison. It does not account for any clustering, covariate adjustment, or other features that might have been incorporated in the sponsor’s model. These differences can affect both the estimate and its uncertainty. Given the recorded discrepancy, the sponsor’s own analysis takes precedence, while the recalculation is useful only as a separate description of what the arm-level summary counts imply under a simple analysis.
How should secondary comparisons be read?
The registry posts recalculated binary comparisons for the presence of a benzodiazepine medication order, a do-not-resuscitate order, and a Palliative Care Consult Order. Each is expressed as a risk difference with a confidence interval and an unadjusted p-value. As a general statistical concept, an unadjusted p-value evaluates one comparison without correcting for the number of outcomes or tests considered. When multiple endpoints are examined, the chance of finding at least one apparently unusual result can increase. The registry does not report a multiplicity adjustment or sponsor analyses for these secondary results.
All recalculated results depend on the posted counts and the assumptions of the unadjusted method. The registry does not report missing-data procedures, endpoint adjudication details, or a full analysis plan. The trial’s original statistical plan, including its planned model specifications and any planned interim analyses, is not reported in the registry record. Those omissions limit assessment of whether the posted outcomes and analyses correspond to a prespecified plan.
11. Confidence Intervals and Precision
The recalculated primary risk difference is 0.039, with a 95% confidence interval from 0.018 to 0.060. As a general statistical concept, a confidence interval describes the range of effect values reasonably compatible with the data and the calculation’s assumptions. It is not a guarantee that the true effect lies in the interval, and it does not describe uncertainty from design or measurement issues that the calculation does not model.
The primary interval lies on the positive side of zero, which is consistent with the small unadjusted p-value. This interval belongs to the recalculated comparison, not to the sponsor’s GEE analysis. The sponsor result is reported as p = 0.445, but the corresponding estimate and confidence interval are not reported in the registry record. Thus, the posted information does not permit a direct comparison of precision between the sponsor’s analysis and the recalculation.
The recalculated relative-measure intervals also lie above 1, but these measures use different scales and should not be read as if they were the risk difference. For the secondary comparisons, the intervals vary in whether they include zero, reflecting differing levels of precision and evidence in the unadjusted calculations. Their interpretation is further limited by the number of outcomes considered and the absence of a reported multiplicity procedure.
Precision is not the same as importance. Even a relatively narrow interval around a record-based difference cannot answer whether the difference reflects a meaningful change in the care experience or outcome for an individual Veteran. Conversely, the sponsor’s p-value alone does not convey the size or direction of its estimate. The lack of a reported sponsor estimate and interval leaves a substantial gap in assessing the primary result’s magnitude under the sponsor’s method.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 965 | 1,291 |
| 75% of the observed effect | 1,786 | 2,391 |
| 50% of the observed effect | 4,176 | 5,590 |
The sample-size table and curve are planning illustrations based on the observed effect, not the original design calculations. The figures are participants per arm because these are binary participant outcomes. For the primary outcome, the planning illustration uses the observed effect and also shows planning for 75% and 50% of the observed effect. Smaller effects require more participants to detect under the stated assumptions.
The planning assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, a normal approximation, and an effect to detect equal to the stated fraction of the observed effect. The figures in the 80% power column describe the participants per arm for a design intended to have an 80% chance of detecting an effect of that specified size, if the assumptions hold. The 90% power column makes the same kind of planning statement for a 90% chance. Higher target power and smaller target effects generally require larger planned samples.
For the primary outcome, planning for 100% of the observed effect gives 965 participants per arm for 80% power and 1,291 participants per arm for 90% power. Planning for 75% of the observed effect gives 1,786 participants per arm for 80% power and 2,391 participants per arm for 90% power. Planning for 50% of the observed effect gives 4,176 participants per arm for 80% power and 5,590 participants per arm for 90% power.
These values are illustrations for future planning and are not observed power. The registry’s assumptions for the original trial design, including its original sample-size justification, are not reported in the registry record. The planning figures should not be used to infer that the completed trial was designed around the recalculated effect or that the original study had a particular power.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.
Worked calculation for this trial
- Control proportion p0 = 0.876; treatment proportion p1 = 0.915; average p̄ = 0.895.
- First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.4333 = 0.8493.
- Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.4325 = 0.3640.
- n = (0.8493 + 0.3640)2 / (0.0391)2 = 964.6, rounded up to 965 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
p0 <- 0.875576036866359 # control proportion
p1 <- 0.914641375222288 # treatment proportion
n_per_arm <- function(power, frac = 1) {
q1 <- p0 + (p1 - p0) * frac # smaller effect = fraction of the observed difference
pbar <- (p0 + q1) / 2
z_b <- qnorm(power)
ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")
# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(2383173)
n <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))
Output from running the code
100% of observed effect: n per arm = 965 (80% power), 1,291 (90% power)
75% of observed effect: n per arm = 1,786 (80% power), 2,391 (90% power)
50% of observed effect: n per arm = 4,176 (80% power), 5,590 (90% power)
power.prop.test: 964.6
Simulated power at n = 965 per arm: 0.800 (20000 replicates)
The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| Generalized Estimating Equation (GEE) | not reported | 0.445 |
The registry reports a GEE analysis comparing the Basic and Enhanced Implementation Approaches, with p = 0.445. It does not report the sponsor’s effect estimate or confidence interval, and the GEE details needed to reproduce the analysis are not reported in the registry record. The reported p-value is the sponsor’s own analysis and takes precedence over the recalculated result because the record indicates a discrepancy.
The recalculated risk difference of 0.039, its confidence interval from 0.018 to 0.060, and p < 0.001 come from a simple unadjusted comparison of the posted counts. They do not reproduce the GEE analysis. The model could have incorporated features not represented in the arm-level recalculation, but the registry record does not specify which features or explain the discrepancy. It would therefore be inappropriate to treat the recalculation as a correction of the sponsor’s result.
The two results answer related but not necessarily identical statistical questions. The recalculation describes the separation between the posted group proportions under a simple method. The sponsor’s GEE p-value reflects the analysis method named in the registry, although the associated estimate and uncertainty are unavailable. Readers should retain that distinction rather than combine the recalculated estimate with the sponsor’s p-value or infer the sponsor’s effect size from it.
15. Multiplicity, Missing Data and Analysis Populations
The registry identifies one primary outcome and lists secondary outcomes covering medication orders, care orders, location of death, devices, restraints, and pastoral care. The record does not report whether the investigators used a multiplicity adjustment, a testing hierarchy, or another strategy for controlling the chance of false-positive findings across multiple outcomes. The secondary recalculations are unadjusted, so their p-values should be treated as exploratory rather than as a collection of confirmatory tests.
The registry record also does not report the original statistical analysis plan, covariates for the sponsor’s GEE model, the GEE correlation structure, or details of the model’s standard error calculation. It does not identify whether interim analyses were planned or conducted. These details cannot be inferred from the reported p-value or from the arm-level counts.
Information about missing data is also limited. The record does not describe how unavailable medical records, missing outcome values, or incomplete follow-up were handled. It does not specify whether the outcome counts reflect all eligible Veterans, a complete-case population, or another analysis population. The relationship between the registry enrollment of 134 and the primary outcome denominators of 1519 and 1687 is not explained. The unit randomized is not reported, so the degree to which the outcome data may be clustered by site or team cannot be assessed from the record.
These gaps matter for evaluating both the analysis and its generalizability. A model that accounts for clustering can produce different uncertainty estimates from a simple comparison, and inclusion or exclusion of participants with unavailable outcomes can affect the estimated group proportions. Without the original analysis details, population definitions, and missing-data procedures, it is not possible to establish whether the posted counts and the sponsor’s GEE result use the same underlying population or assumptions.
16. What the Result Does, and Does Not, Mean
The posted counts show that active opioid orders at death were recorded for 1330 participants in the Basic Implementation Approach and 1543 in the Enhanced Implementation Approach. The unadjusted recalculation estimates a positive risk difference, but the sponsor’s reported GEE analysis has p = 0.445. Because the registry records a discrepancy, the sponsor’s analysis takes precedence; the recalculation should be read as a separate, simple description of the arm-level counts rather than as the definitive primary analysis.
The outcome is a record-based indicator of an active order at death. It does not establish whether the medication was administered, how it was used, or what effect it had on symptoms or the experience of care. The trial compares two approaches for implementing education through Palliative Care Consult Teams; it does not compare an education program with no program. The primary endpoint therefore speaks to documentation of one care-process measure in the specified population, not to all aspects of end-of-life care.
The record does not report an estimate or confidence interval for the sponsor’s GEE analysis. That prevents a complete assessment of the direction and magnitude of the sponsor’s estimated effect, even though its p-value is available. The recalculated interval describes uncertainty only for the unadjusted calculation and cannot fill that gap.
Nor do these results establish whether either approach improved overall care or outcomes for individual Veterans. The registry does not report numerical findings for all listed secondary outcomes, does not provide enough detail to assess missing data or the analysis population, and does not explain the relationship between the listed enrollment and outcome denominators. Such limits make broad claims about implementation success or clinical impact unsupported by the posted record.
17. Why This Trial Matters Statistically
This trial is statistically interesting because it compares two active implementation strategies rather than an intervention with no implementation support. That design directs attention toward how an established clinical team infrastructure might be used to spread education, and it makes the contrast relevant to implementation research. The primary outcome is a measurable record-based process indicator, allowing a comparison between the two approaches.
The primary result also illustrates why statistical method and reporting detail matter. The posted arm counts yield a positive unadjusted risk difference with a confidence interval that excludes zero, while the registry reports a GEE p-value of 0.445. The two results cannot be reconciled from the information in the record. A reader therefore needs to distinguish a transparent recalculation from the sponsor’s named analysis, and avoid treating either as a complete account of the other.
The record highlights additional practical issues: the listed enrollment does not align transparently with the outcome denominators, the unit of randomization is not identified, and the sponsor’s model details are sparse. These are not merely technical omissions. They affect whether readers can understand the population represented by the outcome counts, reproduce the analysis, and judge the uncertainty appropriately.
Finally, several related care-process outcomes are listed, while only some have numerical results. Reading multiple endpoints without a stated multiplicity strategy can make isolated findings appear more definitive than they are. The trial therefore offers a useful case for separating a primary analysis from exploratory comparisons, distinguishing documentation outcomes from clinical outcomes, and recognizing when registry reporting is insufficient for a complete statistical assessment.
18. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
Several limitations are specific to the information reported for this trial. The registry lists enrollment as 134, whereas the primary outcome denominators are 1519 and 1687. The record does not explain whether enrollment refers to a different unit or population, or how it relates to the Veterans included in the endpoint counts. The unit of randomization is also not reported. Because the intervention approaches were delivered through Palliative Care Consult Teams, the missing randomization-unit detail makes it difficult to assess whether the reported participant counts reflect clustered assignment or outcomes.
The sponsor’s GEE result is reported only as p = 0.445. Its effect estimate, confidence interval, covariates, correlation structure, and analysis population are not reported in the registry record. This makes the discrepancy with the unadjusted recalculation impossible to investigate from the posted information. The record also does not describe missing-data handling, outcome abstraction procedures, or whether the recorded order was assessed consistently across sites.
Only three secondary outcomes have numerical results detailed in the record, although additional secondary outcomes are listed. No sponsor analyses are listed for the posted secondary results, and no multiplicity strategy is reported. These limitations constrain interpretation of the secondary pattern and leave the possibility of selective or incomplete outcome reporting unassessed.
Finally, the primary outcome is a documented order at death rather than a direct measure of medication administration or patient experience. The registry record does not establish how the implementation strategies affected broader care processes or clinical outcomes. These limitations concern what can be concluded from the registry record; they do not establish that the underlying trial lacked procedures or analyses that were not recorded.
19. Related Tutorials
20. Related Statistical Calculators
21. Sources
- ClinicalTrials.gov: Evaluation of Methods for Implementation of a Comfort Care Order Set — NCT02383173.
- Registry record last updated 2023-07-27; status last verified 2023-07.
- Results first posted 2020-12-08.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
22. Record Summary
The registry describes a completed randomized, parallel comparison of Basic and Enhanced Implementation Approaches for a Comfort Care Education Intervention using Palliative Care Consult Teams. The primary outcome is an active opioid order at death among Veterans who died as inpatients in acute care units of VA Medical Centers, within the stated timeframe following the intervention period.
The recalculated unadjusted comparison yields a positive risk difference, while the sponsor’s reported GEE analysis gives p = 0.445. The registry marks a discrepancy, so the sponsor’s own analysis takes precedence. The record does not report the sponsor estimate or confidence interval, key model details, the unit randomized, missing-data procedures, or the relation between listed enrollment and outcome denominators. The secondary results available in the record are incomplete and unadjusted.