Effective Communication to Improve Decision Making About Health Care Plans: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Effective Communication to Improve Decision Making About Health Care Plans (NCT01986790), a randomized na study. The primary outcome is Knowledge.

2. Clinical Question

3. Trial Design

The registry describes a randomized, single-masked, parallel study sponsored by Washington University School of Medicine. The listed phase is NA. The record reports enrollment of 343 and a completed study. The start is recorded as 2012-04, and primary completion as 2014-03.

Participants were assigned to one of three behavioral communication approaches. The Plain Language group received a table describing plan features and costs, with definitions of insurance terms incorporated into the table. The Plain Language + Visuals group received that table plus visuals focused on particular plan features; participants could view one feature at a time and choose the order. The Plain Language + Narratives group received the table plus narratives about how others might use and rate the plans.

The primary outcome was knowledge, assessed immediately after participants saw their assigned intervention. The registry describes the outcome as the mean and standard deviation of a calculated knowledge score. It reports group sizes of 91 for Plain Language, 154 for Plain Language + Visuals, and 98 for Plain Language + Narratives. These group sizes account for the posted primary-outcome summaries, but the analysis population definition is not reported in the registry record.

Single masking is recorded, but who was masked is not reported in the registry record. The allocation procedure beyond randomization, details of implementation, and the trial’s original sample-size assumptions are also not reported in the registry record. The posted results include two comparisons with Plain Language as the reference group.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Analysis is represented as mean (standard deviation) of calculated knowledge.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Knowledge, measured over 1 day (Immediately following showing the participant the assigned intervention (plain language table, plain language table + visuals, or plain language table + narratives).

The primary endpoint is knowledge about health insurance plans. The registry says the score represents understanding of plan details and is based on the number of correct answers to 7 items. The scale ranges from 0 to 7, and a higher score is described as a better outcome. Scores are reported as group means with standard deviations.

This measure is relevant to the stated study aim: examining whether communication strategies help people understand health insurance plan information. The endpoint therefore captures participants’ performance on questions about the information they had just viewed, rather than a later decision, insurance choice, or health outcome. The recorded timeframe is 1 day, immediately following presentation of the assigned intervention.

A score can summarize performance across the items, while the mean describes the group’s average score and the standard deviation describes variation among participants. Those summaries do not show how participants answered each individual item or whether particular terminology was easier to understand than other plan details. Item-level results and information about the score’s measurement properties are not reported in the registry record.

7. Results Posted in the Registry

• Plain Language + Visuals: mean 5.70 versus 6.00 on control; difference -0.30 (95% CI -0.62 to 0.02), standardized effect d = -0.24; z-test p = 0.067.
• Plain Language + Narratives: mean 5.50 versus 6.00 on control; difference -0.50 (95% CI -0.89 to -0.11), standardized effect d = -0.37; z-test p = 0.011.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary outcome is continuous: each participant has a numerical knowledge score, and the registry reports the mean and standard deviation in each group. A mean difference expresses the difference between group averages. In these comparisons, the reference is Plain Language, and the contrasts are reported as the enhanced-approach mean relative to that reference. A negative estimate indicates a lower observed average in the named group than in Plain Language.

What does the standard deviation describe?

The standard deviation describes how spread out individual scores are around their group mean. It is not the uncertainty of the mean itself. Uncertainty in a mean difference also depends on the group sizes and the variability of the scores. The registry reports standard deviations of 1.2 for Plain Language, 1.3 for Plain Language + Visuals, and 1.5 for Plain Language + Narratives. The group means and standard deviations provide a compact description of the observed score distributions, but they do not reveal the full distributions or individual responses.

How should a confidence interval be read?

A confidence interval gives a range of values compatible with the estimate under the statistical procedure used. In general, it communicates precision: narrower intervals indicate less uncertainty about the contrast than wider intervals, while an interval spanning zero includes no mean difference among the values it covers. The Visuals comparison interval extends from -0.621 to 0.021 and spans zero. The Narratives comparison interval extends from -0.886 to -0.114 and does not span zero. The registry record does not identify the interval construction method in enough detail to assess its assumptions.

What does the p-value add?

A p-value describes how unusual data at least as incompatible with a specified null hypothesis would be under the assumptions of the statistical test. It is not the probability that the null hypothesis is true, nor does it measure the size or practical importance of a difference. The posted unadjusted p-values are 0.067 for Visuals versus Plain Language and 0.011 for Narratives versus Plain Language. The analysis test and any model adjustments are not reported in the registry record. The p-values should be considered alongside the estimated differences and intervals, rather than as standalone measures.

Why can over-dispersion matter?

General explanation: over-dispersion occurs when observed variation is greater than a model assumes. If a model underestimates variability, its standard errors may be too small and its uncertainty intervals too narrow. This concept is particularly relevant for count or binary outcomes. The posted primary outcome is a continuous score summarized by means and standard deviations, and the registry record does not report a model-based over-dispersion analysis. It would not be appropriate to infer that over-dispersion was assessed or present.

What can be inferred from the randomized design?

Random assignment is intended to create groups that can be compared without systematically choosing the intervention based on participant characteristics. The registry identifies the study as randomized and parallel, but it does not describe the sequence-generation or allocation-concealment procedures. Nor does it report the masked party. These missing details limit assessment of how randomization and masking were implemented. The posted estimates remain comparisons of the reported groups; the record does not provide enough information to evaluate all potential sources of bias or the exact analysis population.

Why does the reference group matter?

Both contrasts use Plain Language as the reference, so each estimate answers a specific relative question: how the mean score in one enhanced-communication group compares with the mean in the plain-language group. These contrasts do not directly compare Visuals with Narratives. No direct comparison between those two groups is reported. A difference between two separate reference-group estimates does not itself provide a formally tested comparison between the enhanced approaches.

10. Confidence Intervals and Precision

Precision concerns the uncertainty around an estimated contrast, not just the numerical value of the estimate. For Plain Language + Visuals versus Plain Language, the mean difference is -0.3, with a reported interval from -0.621 to 0.021. Because the interval includes zero, the data are compatible with a range that includes no average difference, as well as values on either side of zero. The estimate is therefore not sharply separated from zero by this interval.

For Plain Language + Narratives versus Plain Language, the mean difference is -0.5, with an interval from -0.886 to -0.114. This interval lies below zero, indicating that the posted range of compatible differences is negative for this contrast. It still covers a range of possible magnitudes, so the point estimate should not be treated as an exact value.

In general, a confidence interval reflects both the estimated contrast and the uncertainty in that estimate. The standard deviations and group sizes contribute to precision, but the interval also depends on the statistical procedure and its assumptions. The registry record does not report enough analysis detail to independently evaluate the interval method. The confidence intervals are not bounds on every participant’s score, and they do not describe the range of individual outcomes.

Precision is distinct from importance. The scale runs from 0 to 7, but the registry does not report a threshold for a meaningful difference or evidence about how score differences relate to subsequent decisions. The intervals therefore describe uncertainty in mean score contrasts, not the practical importance of those contrasts. Unadjusted intervals also do not account for multiplicity unless an adjustment is specified; no such adjustment is reported in the registry record.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total events. They are illustrations based on the observed effect and assume a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is set to the stated fraction of the observed effect. These are planning assumptions for the illustrations; the trial’s original design assumptions are not reported in the registry record.

For each comparison, the 80% power column gives the estimated number of participants per arm for a future study to have an 80% chance of detecting the specified effect if that effect were the true one and the assumptions held. The 90% power column gives the corresponding number per arm for a 90% chance. These percentages describe a study-design target, not the probability that the observed result is correct. The figures are not observed power.

Planning for 100% of the observed effect gives 279 participants per arm for 80% power and 373 for 90% power for Visuals versus Plain Language. Planning for 75% of the observed effect gives 496 per arm for 80% power and 664 for 90% power; planning for 50% gives 1,115 and 1,492 per arm, respectively.

For Narratives versus Plain Language, planning for 100% of the observed effect gives 117 participants per arm for 80% power and 157 for 90% power. Planning for 75% gives 208 and 278 per arm; planning for 50% gives 468 and 626 per arm. Smaller effects require more participants to detect under the stated assumptions. These figures are not a reconstruction of the trial’s original sample-size rationale.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -0.300; pooled standard deviation σ = 1.264.
2. n = 2 × (1.96 + 0.842)² × 1.264² / -0.300² = 278.6, rounded up to 279 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.3                      # observed difference in means
sigma <- 1.26388583638215                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(1986790)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 279 (80% power), 373 (90% power)
 75% of observed effect: n per arm = 496 (80% power), 664 (90% power)
 50% of observed effect: n per arm = 1,115 (80% power), 1,492 (90% power)
power.t.test: 279.6 
Simulated power at n = 279 per arm: 0.801 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry posts two primary-outcome contrasts, each using Plain Language as the reference group. Their reported p-values are unadjusted. An adjustment for multiple comparisons is not reported in the registry record, and no multiplicity strategy or prespecified hierarchy is described. When several comparisons are considered, the chance of at least one apparently unusual result can differ from that for a single test. General statistical practice may address this through prespecification or adjustment, but the record does not establish whether either approach was used outside the posted results.

The record lists three outcome measures, but no secondary results are posted in the registry record for this analysis, and the number of analyses listed is 0. This does not establish that no other analyses were conducted; analysis details beyond the posted comparisons are not reported. The registry record does not state whether the primary outcome and contrasts were prespecified in a protocol or statistical analysis plan.

Missing-data handling is not reported in the registry record. The group sizes accompanying the outcome summaries are available, but the record does not explain whether those are all randomized participants, participants with complete outcome data, or another analysis set. Reasons for missing measurements, imputation procedures, and sensitivity analyses are not reported. Without those details, it is not possible to assess whether missing knowledge scores could have influenced the estimated group means or contrasts.

The analysis population is likewise not described. The record does not state whether results follow intention-to-treat principles, whether participants were excluded after randomization, or how deviations from assigned interventions were handled. The method used to estimate the comparisons, including any covariate adjustment, is not reported. These omissions make it difficult to evaluate the relation between random assignment, the analyzed participants, and the posted estimates.

14. What the Result Does, and Does Not, Mean

The posted results describe knowledge scores measured immediately after participants saw one of three formats for presenting health insurance plan information. The Plain Language group had a mean of 6.0. The means were 5.7 for Plain Language + Visuals and 5.5 for Plain Language + Narratives. The estimated contrasts relative to Plain Language were -0.3 and -0.5, respectively.

The Visuals contrast has an interval that includes zero, while the Narratives contrast has an interval wholly below zero. The corresponding unadjusted p-values are 0.067 and 0.011. These are statistical summaries of the posted comparisons. They do not indicate the probability that either communication approach is beneficial or harmful, and they do not establish the practical importance of a score difference.

The outcome measures correct answers about insurance plan details, not whether participants made a better insurance choice, enrolled in a plan, or experienced a later health or financial outcome. The record does not report these downstream outcomes. Nor does an immediate assessment show whether knowledge persisted beyond the stated timeframe. The results therefore speak to performance on the knowledge measure at the assessment, not to lasting understanding or real-world consequences.

The study compares communication formats, but the record does not report item-level scores, participant preferences, or analyses explaining why scores differed. The findings cannot identify which visual elements or narrative features may have contributed to the group summaries. Further, no direct statistical contrast between the Visuals and Narratives groups is posted.

Finally, the conclusions depend on the reported analysis and its uncertainty. The registry record does not describe missing-data handling, the analysis population, the test used, or multiplicity adjustment. These limits mean the posted results should be interpreted as estimates for the described outcome and comparisons, rather than as a complete account of all evidence about communication and health insurance decisions.

15. Why This Trial Matters Statistically

This trial is statistically interesting because its primary outcome is closely aligned with a specific communication question: whether ways of presenting plan information are associated with differences in participants’ immediate knowledge scores. The three approaches share a plain-language table, while two add either visuals or narratives. The posted contrasts therefore make clear how each augmented format compares with that common reference approach.

The results also illustrate why effect estimates and uncertainty matter together. The Visuals estimate is -0.3, and its interval includes zero; the Narratives estimate is -0.5, and its interval lies below zero. A p-value alone would not communicate the direction and range of plausible values. The score means, standard deviations, group sizes, contrasts, and intervals each contribute different information about the result.

The two comparisons also raise an important issue in interpreting multiple tests. Both use the same reference group, and the reported p-values are unadjusted. Without a reported multiplicity plan, readers should distinguish the posted contrast-specific results from a confirmatory claim that accounts for the full set of outcomes and comparisons.

Finally, the record shows how statistical evidence can be narrower than a study’s broad motivation. Immediate knowledge scores are relevant to understanding plan information, but they are not equivalent to decision quality or later behavior. Clear separation of measured outcomes from broader goals helps prevent overinterpretation. Missing reporting about analysis populations, missing data, and model details likewise identifies what additional information would support a fuller statistical appraisal.

16. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to the available record. It identifies the randomized, single-masked, parallel design but does not identify who was masked or explain how randomization and allocation were implemented. These details matter when assessing the possibility of selection or measurement biases, yet they cannot be evaluated from the registry record.

The primary outcome is described as a score based on 7 correct-answer items on a 0-to-7 scale. Item-level results, score validation information, and the handling of any unanswered items are not reported. Means and standard deviations cannot show whether score distributions were skewed, clustered near the scale limits, or driven by particular questions. The record also does not provide participant characteristics, so the groups’ composition and the applicability of the results to other populations cannot be assessed from these results.

The results concern an immediate assessment. Later knowledge retention, plan selection, enrollment, and other consequences are not reported. The descriptions distinguish the communication formats, but the record does not report how consistently participants engaged with the materials or whether the intended features were understood as designed.

Analysis reporting is limited: the statistical test or model, analysis population, missing-data approach, and multiplicity handling are not reported. The record lists zero analyses and includes two unadjusted primary-outcome contrasts, but it does not explain the relationship between those entries or whether other analyses were conducted. Adverse-event fields list zero participants at risk in each group, with deaths recorded as null; this does not provide an interpretable safety assessment or establish that no events occurred.

17. Related Tutorials

18. Related Statistical Calculators

19. Sources

• ClinicalTrials.gov: Effective Communication to Improve Decision Making About Health Care Plans — NCT01986790.
• Registry record last updated 2016-06-09; status last verified 2016-05.
• Results first posted 2016-06-09.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

20. Record Summary

The registry record describes a completed randomized, single-masked, parallel study sponsored by Washington University School of Medicine, with enrollment of 343. Participants were assigned to Plain Language, Plain Language + Visuals, or Plain Language + Narratives. The primary outcome was knowledge about health insurance plan details, measured immediately after the assigned information was shown and scored from 0 to 7.

Mean scores were 6.0, 5.7, and 5.5, respectively. Compared with Plain Language, the estimated mean differences were -0.3 for Visuals and -0.5 for Narratives. The Visuals interval included zero; the Narratives interval was below zero. The posted p-values are unadjusted. Details about missing data, analysis populations, statistical models, multiplicity, and the trial’s original sample-size assumptions are not reported in the registry record.