Can Exposure to Brief Messages Correct Misperceptions?: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Can Exposure to Brief Messages Correct Misperceptions? (NCT05129592), a randomized na study. The primary outcome is Change in Accuracy of Belief That Nicotine Causes Cancer.

2. Clinical Question

The study focused on a specific misconception: that nicotine itself causes cancer. Its central question was whether adding explanations to a factual correction could change the accuracy of that belief. The message content was organized around two components of narrative coherence. One component explained what causes tobacco-related disease; the other offered an account of why the mistaken nicotine-cancer connection might be believed. The factorial design included messages with each component separately and with both together, alongside the factual control message.

The registry also describes questions about beliefs concerning other tobacco products, including products not explicitly discussed in the messages. Those questions were intended to assess whether message exposure affected related beliefs and inferences. However, secondary results are not reported in the registry record. The primary outcome therefore provides the available posted evidence for the specific belief that nicotine causes cancer. It does not, by itself, establish whether messages changed beliefs about other products or broader reasoning.

3. Trial Design

This was a randomized, single-masked, factorial behavioral study. It tested four factual-message conditions: a control correction without explanations, a correction with a causal explanation, a correction explaining why the misperception may have arisen, and a correction containing both explanatory components. The registry identifies the intervention as coherent corrective messages. It does not report which party was masked, so the meaning of single masking cannot be specified further.

The study was conducted as an online experiment. The registry lists enrollment of 193 and reports that the study is completed. The recorded start date is 2021-11-22, and the primary completion date is 2022-01-30. Participant demographics, recruitment methods, eligibility criteria, and the exact process used to assign participants to message conditions are not reported in the registry record. The registry describes randomized allocation but does not provide further detail about sequence generation or concealment.

The primary outcome was measured at baseline and up to 5 minutes after participants received a corrective message. The registry reports group summaries for the four conditions and also reports sponsor analyses comparing selected groups using ANCOVA on final values. These are different kinds of summaries: the simple recalculations compare posted group means, while the sponsor’s analyses are adjusted comparisons of final values. The registry does not report a complete statistical analysis plan, a factorial interaction estimate, or the original design assumptions. Although the study description names additional belief questions, secondary results are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in Accuracy of Belief That Nicotine Causes Cancer, measured over Baseline and up to 5 minutes after receiving the corrective message.

The primary endpoint is titled Change in Accuracy of Belief That Nicotine Causes Cancer. It captures the extent to which participants agree with the statement that the substance nicotine causes cancer. Responses use a continuous scale from 0, labeled I do not agree at all, to 100, labeled I completely agree. The outcome is described as measured at baseline and up to 5 minutes after the corrective message.

This measure is relevant to the study question because the messages are intended to correct the particular belief that nicotine causes cancer. A lower agreement score corresponds to less agreement with that statement, while a higher score corresponds to more agreement. The scale gives a graded response rather than limiting answers to a yes-or-no choice. The posted arm summaries include a mean and standard deviation for each message condition.

The endpoint is a self-reported belief measure, not a measure of behavior, tobacco use, disease, or a clinical health outcome. The short interval after the message captures an immediate response; whether any belief change persisted beyond that interval is not reported in the registry record. The record does not provide details about the exact calculation of the change score, how baseline values were incorporated into the posted arm summaries, or how missing endpoint responses were handled. Those uncertainties matter when interpreting the summaries and comparing them with the sponsor’s adjusted analyses.

7. Results Posted in the Registry

• Nicotine Corrective With Causal Explanation: mean 54.00 versus 52.60 on control; difference 1.40 (95% CI -12.14 to 14.94), standardized effect d = 0.04; z-test p = 0.839.
• Nicotine Corrective With Reason for Misperception: mean 39.20 versus 52.60 on control; difference -13.40 (95% CI -26.55 to -0.25), standardized effect d = -0.40; z-test p = 0.046.
• Nicotine Corrective With Both Components of Coherence: mean 36.10 versus 52.60 on control; difference -16.50 (95% CI -30.91 to -2.09), standardized effect d = -0.49; z-test p = 0.025.

8. Statistical Interpretation

9. Statistical Methods Explained

What does a mean difference describe? A mean difference expresses how far apart two group averages are on the outcome scale, using the stated group order as its reference. For the simple recalculations, the posted mean for a message condition is compared with the posted mean for control. A negative difference means the message-condition mean is lower than the control mean under that comparison. Because lower agreement corresponds to less agreement that nicotine causes cancer, the direction is interpretable in relation to the wording of the scale. A mean difference does not describe the range of individual responses or imply that every participant’s response changed by that amount.

What do the spread estimates describe? Each arm has a reported standard deviation: 31.9 in control, 35.3 for the causal explanation, 34.1 for the reason for misperception, and 35.4 for both components. In general, a standard deviation describes the dispersion of individual measurements around a group mean. These estimates indicate that individual scores vary around their respective averages. They are not standard errors or confidence intervals for the group means. The recalculations use pooled standard deviations reported for each comparison; pooling is a way of summarizing within-group variability for a two-group contrast. The registry does not report a separate assumption assessment for the recalculated contrasts.

What does a confidence interval communicate? In general, a confidence interval describes uncertainty around an estimated effect under the method used to calculate it. A wide interval indicates less precision than a narrow interval. When an interval includes zero for a difference, the data are compatible with no difference as well as with effects in either direction covered by that interval. An interval that excludes zero is evidence of a nonzero difference under the corresponding unadjusted calculation, but it does not by itself establish practical importance, persistence, or causation beyond the randomized comparison. Interpretation also depends on the analysis being appropriate and on how many outcomes or comparisons were examined.

What do the p-values mean? In general, a p-value measures how incompatible the observed data are with a specified null hypothesis under the assumptions of the statistical test. It is not the probability that the null hypothesis is true, and it is not a measure of the size or importance of an effect. The recalculated p-values are explicitly unadjusted. The registry does not report whether the sponsor’s p-values were adjusted for multiple comparisons. Since the record notes 36 analyses, the possibility of multiple testing is relevant; a nominal p-value should be read in that context rather than in isolation.

How does ANCOVA differ from a simple comparison? In general, analysis of covariance (ANCOVA) estimates a group contrast while accounting for one or more covariates. In studies with baseline and follow-up measures, baseline outcome values may be included as a covariate, but the registry does not specify the covariates used in the sponsor analyses. The posted sponsor results are labeled ANCOVA and describe mean differences in final values. The recalculated contrasts instead use the posted group summary means and standard deviations and are unadjusted. Because these analyses use different information and methods, estimates and p-values need not match. The record flags a discrepancy, so the sponsor’s own analysis takes precedence for the reported trial conclusion.

What does the factorial design allow? In general, a factorial design can evaluate components separately and in combination, and can assess whether the effect of one component depends on the presence of another. The four message conditions correspond to the explanatory components being absent or present in different combinations. However, having a factorial structure does not by itself establish that an interaction was tested or reported. The registry lists selected pairwise ANCOVA comparisons, but an interaction estimate and a full factorial analysis are not reported in the registry record. The posted comparisons therefore should not be treated as a reported test of whether the two components work synergistically.

How should the continuous scale be understood? The measure ranges from 0 to 100 and asks about degree of agreement. In general, treating a response scale as continuous allows summaries such as means and standard deviations and comparisons of average scores. It also compresses responses into a single value, which can hide differences in response distributions or variation in how participants interpret scale points. The registry does not report distributional diagnostics, alternative analyses, or evidence about the scale’s measurement properties in this sample. These features limit how much can be inferred from the group averages alone.

For these reasons, the results should be read as comparisons of short-term average self-reported belief scores across randomized message conditions. The registry does not report the full analysis plan, the covariates in ANCOVA, missing-data procedures, or adjustment for multiplicity. Those omissions prevent a complete independent assessment of how the sponsor’s reported analyses were specified and how robust the conclusions are to other reasonable analytical choices.

10. Confidence Intervals and Precision

The recalculated estimates are not equally precise. For the causal-explanation condition versus control, the estimated difference is 1.4 points and its 95% confidence interval ranges from -12.14 to 14.94. This interval is broad and includes zero, so the posted summaries are compatible with a difference in either direction. For the reason-for-misperception condition, the estimate is -13.4 points; its interval extends from -26.55 to very near zero. Its upper bound is close to the null value, making the estimate sensitive to uncertainty around that boundary. For both components versus control, the estimated difference is -16.5 points, with a 95% confidence interval from -30.91 to -2.088. This interval excludes zero in the unadjusted recalculation, but still spans a range of possible effect sizes.

In general, a confidence interval is an estimate of precision, not a guarantee that the true effect lies within its limits. The intervals depend on the statistical method and assumptions used. Here, the recalculated intervals are based on posted group summaries and are unadjusted; they are not the same as the sponsor’s ANCOVA intervals. The registry flags a discrepancy, so the sponsor’s own analysis takes precedence. The sponsor’s reported intervals also vary in relation to zero: the causal-explanation versus both-components comparison has an interval from 2.02 to 35.33, whereas the control versus both-components interval ranges from -1.87 to 33.30, and the reason-for-misperception versus both-components interval ranges from -12.76 to 20.15.

The differences in interval width and position reflect both estimated contrasts and uncertainty. The record does not report adjusted intervals accounting for the full set of analyses, so the posted intervals should not be read as multiplicity-adjusted. Precision for an immediate self-reported outcome also does not address whether any difference persists over time; follow-up beyond up to 5 minutes is not reported in the registry record.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total events. They are illustrations calculated from the observed effect rather than the trial’s own original design assumptions; those original assumptions are not reported in the registry record. The illustration uses a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. Each planning scenario asks what enrollment would be needed to detect the stated fraction of the observed effect. These figures are not observed power.

The 80% power column represents the illustrative number of participants per arm for a design with an 80% chance of detecting the specified effect under the stated assumptions, if that effect is real and the assumptions hold. The 90% power column gives the corresponding planning target for a higher probability of detecting that same specified effect. Neither column describes the probability that the trial’s result is correct, and neither changes the interpretation of the observed trial estimates.

The rows use the full observed effect, 75% of the observed effect, and 50% of the observed effect. Smaller effects need more participants to detect with the same planned power and significance level. For example, for the reason-for-misperception comparison, planning for 100% of the observed effect gives 97 participants per arm for 80% power and 129 for 90% power; planning for 50% gives 385 and 516, respectively. These are planning illustrations, not enrollment targets recommended for a future study. They depend on the observed effect, which is uncertain and may not represent the effect a future study would observe. The registry’s own sample-size justification is not reported.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = 1.400; pooled standard deviation σ = 33.843.
2. n = 2 × (1.96 + 0.842)² × 33.843² / 1.400² = 9,173.3, rounded up to 9,174 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- 1.4                      # observed difference in means
sigma <- 33.8432010611366                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(5129592)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 9,174 (80% power), 12,281 (90% power)
 75% of observed effect: n per arm = 16,309 (80% power), 21,832 (90% power)
 50% of observed effect: n per arm = 36,694 (80% power), 49,122 (90% power)
power.t.test: 9174.2 
Simulated power at n = 9,174 per arm: 0.802 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Analyses Reported by the Sponsor

The sponsor’s reported analyses are ANCOVA comparisons of final values, while the recalculations are unadjusted comparisons based on posted group summary statistics. The sponsor reports control versus both components as a mean difference of 15.71, with a 95% confidence interval from -1.87 to 33.30 and p = 0.095. For causal explanation versus both components, the reported difference is 18.67, with a 95% confidence interval from 2.02 to 35.33 and p = 0.022. For reason for misperception versus both components, it is 3.69, with a 95% confidence interval from -12.76 to 20.15 and p = 0.931.

The record marks a discrepancy between the sponsor analysis and the recalculated comparisons. The methods and quantities differ: the recalculations compare posted means without adjustment, whereas the sponsor results are labeled ANCOVA and refer to final values. The registry does not specify all ANCOVA covariates or explain the discrepancy. Accordingly, the sponsor’s own analysis takes precedence when reporting the trial’s results. The unadjusted recalculations can help readers understand how simple contrasts of the summaries behave, but they should not replace or overrule the sponsor’s analysis.

14. Multiplicity, Missing Data and Analysis Populations

The registry reports 36 analyses and 7 outcome measures, while no secondary results are posted. These counts indicate that the primary comparison is not the only analysis contemplated or listed in the record. When many outcomes or comparisons are examined, the chance of obtaining at least one apparently notable result by chance can increase. In general, methods such as prespecifying a primary contrast or adjusting for multiple comparisons can address this concern, but the registry does not report whether such methods were used. The recalculated p-values are unadjusted, and the adjustment status of the sponsor’s reported p-values is not reported in the registry record.

The registry does not report an analysis plan describing which comparisons were primary, how the factorial components were to be tested, or how the 36 analyses were organized. The posted sponsor results include selected pairwise comparisons, but an interaction test for the two message components is not reported. It is therefore not possible to determine from the registry whether these pairwise comparisons were prespecified, selected after examining the data, or adjusted to account for the broader set of analyses.

Missing-data handling is also not reported in the registry record. The record does not describe how incomplete baseline or follow-up responses were treated, whether participants with missing responses were excluded, or whether any imputation method was used. The reported arm summaries give group sizes, but they do not by themselves establish that every randomized participant contributed to the primary endpoint analysis. Analysis populations, such as an intention-to-treat population or a per-protocol population, are not defined in the posted information.

These gaps matter because missing responses and multiple testing can affect estimates, uncertainty, and the interpretation of nominal statistical significance. They do not establish that the reported analysis is invalid; they mean that the registry does not contain enough detail to assess those aspects fully. The results should therefore be interpreted with the stated distinction between sponsor analyses and simple recalculations, and with the unreported analysis details in mind.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry lists zero serious adverse events and zero other adverse events for each of the four message conditions. It also lists zero deaths in each condition. These are the posted event counts for the reported groups and should be read as event counts, not as evidence that no participant experienced any unrecorded or unreported event.

This was an online behavioral experiment involving brief educational messages. The registry does not report detailed adverse-event definitions, the method of event collection, or further safety-monitoring procedures. It also does not provide serious adverse-event terms, consistent with the empty list of top serious-event terms. The figures therefore describe what is recorded in the registry, but they cannot establish the completeness of event ascertainment or characterize risks beyond the reported observation and reporting procedures.

16. What the Result Does, and Does Not, Mean

The posted results describe immediate responses to brief factual messages about nicotine. The outcome was agreement with the statement that nicotine causes cancer, measured from 0 to 100 at baseline and up to 5 minutes after message exposure. The reported means were lower in the two conditions that included an explanation for the misperception, either alone or together with a causal explanation, than in the control condition. The causal-explanation-only mean was close to the control mean. These are descriptive group averages and do not reveal the full pattern of individual responses.

The sponsor’s ANCOVA results are the trial’s reported adjusted analyses and take precedence over the unadjusted recalculations because the registry flags a discrepancy. Those sponsor comparisons do not all give the same statistical evidence: one reported interval excludes zero, while the other two include zero. The comparisons are between selected message conditions, and the registry does not report a factorial interaction test. It is not possible to infer from these results alone whether combining the components produces an effect beyond the components considered separately.

The findings do not establish that any message caused a durable change in belief. The available endpoint is a short-term self-reported score, and later follow-up is not reported in the registry record. Nor do the primary results establish effects on tobacco use, product switching, health outcomes, or beliefs about other products. Although the study description mentions related questions, their results are not posted.

Statistical evidence and practical meaning are distinct. A p-value does not measure the size or importance of a difference, and a confidence interval conveys uncertainty rather than a guarantee. The reported comparisons also sit within a record listing 36 analyses, with multiplicity procedures not reported. The most defensible reading is therefore limited to the short-term belief measure, the specific message conditions, and the analyses the registry reports.

17. Why This Trial Matters Statistically

This trial raises a useful statistical question about how message content can be separated into components. The four conditions distinguish a factual correction from explanations about disease causation and about the origin of a mistaken belief. A factorial design can, in principle, help researchers assess component effects and whether components interact. The registry’s selected comparisons offer some pairwise information, but do not report an interaction estimate. That distinction matters: a result for one combined message compared with another condition is not automatically evidence that two components reinforce one another.

The endpoint also illustrates the difference between an immediate response and a lasting outcome. A continuous agreement scale can detect graded differences in stated belief, and the baseline-to-post-message timeframe makes the outcome relevant to the immediate educational exposure. Yet a change in a short-term self-report is not equivalent to a change in behavior or long-term belief. Keeping the measurement interval and construct explicit prevents statistical findings from being extended beyond what was measured.

Finally, this record shows why analysis details and precision matter alongside average differences. The recalculated unadjusted contrasts and the sponsor’s ANCOVA results are not interchangeable, and the registry flags a discrepancy. The sponsor analysis takes precedence, while the recalculations make the dependence on analysis method visible. The number of analyses, unreported multiplicity methods, and absent secondary results further limit how confidently individual p-values can be interpreted. These are important considerations when evaluating a randomized experiment with several message conditions and outcomes.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record leaves important features of the sample unspecified. It reports enrollment and arm-level group sizes but does not report participant demographics, eligibility details, recruitment methods, or how representative the online sample was. These omissions limit assessment of how well the results may apply beyond the participants studied. The registry says the experiment was single-masked, but does not identify who was masked.

The outcome concerns self-reported agreement and was measured only at baseline and up to 5 minutes after exposure. Later persistence of any response is not reported. The record also does not provide secondary outcome results, despite describing questions about beliefs concerning other tobacco products and inferential beliefs. Those questions cannot be evaluated from the posted results.

There is a marked discrepancy between the sponsor analyses and the unadjusted recalculations. The sponsor’s ANCOVA analyses take precedence, but the registry does not specify all covariates, explain the discrepancy, or provide a complete statistical analysis plan. It also does not report an interaction analysis for the factorial components, multiplicity adjustment, missing-data methods, or analysis populations. These omissions make it difficult to assess the full analysis strategy or determine how sensitive results may be to other choices.

Finally, the arm summaries and short-term outcome do not provide information about individual response distributions beyond the reported standard deviations. A group mean cannot show whether effects were consistent across participants. No inference about behavior, long-term belief, or health outcomes is supported by the primary endpoint alone.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: Can Exposure to Brief Messages Correct Misperceptions? — NCT05129592.
• Registry record last updated 2024-03-12; status last verified 2024-03.
• Results first posted 2024-03-12.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

NCT05129592, titled Can Exposure to Brief Messages Correct Misperceptions?, is a completed randomized, single-masked factorial behavioral study sponsored by Johns Hopkins Bloomberg School of Public Health. It compared four brief factual nicotine-correction messages: a control, a message with a causal explanation, a message explaining the reason for the misperception, and a message containing both components. The registry lists enrollment of 193.

The primary outcome measured agreement that nicotine causes cancer on a continuous scale from 0 to 100, at baseline and up to 5 minutes after the message. The registry reports group means and standard deviations, selected sponsor ANCOVA comparisons of final values, and zero serious adverse events, other adverse events, and deaths in each condition. It marks a discrepancy between the sponsor analysis and unadjusted recalculations; the sponsor’s own analysis takes precedence.

Secondary results, multiplicity methods, missing-data procedures, analysis populations, and the original sample-size assumptions are not reported in the registry record.