A Smartphone Application (ACT on Vaping) for Vaping Cessation in Young Adults: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for A Smartphone Application (ACT on Vaping) for Vaping Cessation in Young Adults (NCT05897242), a randomized na study. The primary outcome is Change in Readiness to Quit.

2. Clinical Question

3. Trial Design

This was a randomized, single-masked, parallel-group trial sponsored by Fred Hutchinson Cancer Center. The registry lists the study as completed, with enrollment of 61. The listed start date is 2024-01-04 and the primary completion date is 2024-04-26. The phase is recorded as NA.

The two arms differed in whether participants had access to the ACT on Vaping app. Arm I combined app use and supportive messages with incentivized text-message check-ins about vaping status. Arm II received the incentivized check-ins without the app. Random allocation is intended to create groups that are comparable on average at the start of a study, although randomization does not guarantee that every participant characteristic is balanced in a trial of this size. The method used to generate the allocation sequence and the identity of the masked party are not reported in the registry record.

The primary outcome was a continuous score, summarized as a mean and standard deviation. The posted outcome summaries include 24 participants in Arm I and 29 in Arm II. The registry's total enrollment and the outcome-specific arm counts describe different parts of the record; the reasons for the difference are not reported. The registry describes the primary population as being used for descriptive statistics in the ACT on Vaping group and for comparisons between arms on other acceptability and efficacy indicators. A detailed analysis-population definition is not reported in the registry record.

The primary completion date marks the stated completion of primary outcome collection, not necessarily the date of every activity in the study. The registry lists 4 outcome measures, but no secondary results are posted and no analyses are listed. Further protocol details, including the original design assumptions and full statistical analysis plan, are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: We calculated descriptive statistics (e.g., means, proportions) for the ACT on Vaping treatment group to evaluate the go/no-go criteria and to compare the two arms on other indicators of treatment acceptability and efficacy. Given that pilot and feasibility trials are not appropriate designs for testing hypotheses regarding the efficacy of interventions, we focused on descriptive differences as an alternative to hypothesis testing.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in Readiness to Quit, measured over From baseline to 3 months post-randomization.

The primary outcome is change in readiness to quit e-cigarette use from baseline to 3 months post-randomization. Readiness was assessed with the 1-item Contemplation Ladder, a scale ranging from 0-10. Higher scores represent greater readiness to make a change. The registry identifies a cut-off score of 5 for high versus low quit readiness and gives its anchor as “Think I should quit but not quite ready.”

Readiness is an intermediate self-reported attitude, rather than a direct measure of whether someone quit vaping or maintained abstinence. It can be useful for describing motivation and for evaluating a program designed to engage people at different stages of readiness. A continuous score retains gradations across the scale, so the analysis can compare average scores or changes rather than only separating participants into high- and low-readiness categories. That distinction matters: an average difference does not directly indicate how many people crossed the cut-off or stopped vaping.

The registry says the outcome evaluates differences in Contemplation Ladder scores, but it does not report the detailed scoring procedure used to define change. The outcome is summarized with a mean and standard deviation. Those summaries describe the average and variation in recorded scores; they do not show each participant's trajectory. The posted comparison therefore addresses average readiness-score change between the randomized arms, not all possible aspects of acceptability, engagement, or vaping behavior.

7. Results Posted in the Registry

• Arm II (Incentivized Text Message Check-ins): mean 0.72 versus 0.96 on control; difference -0.24 (95% CI -1.25 to 0.77), standardized effect d = -0.13; z-test p = 0.642.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary measure is continuous: the Contemplation Ladder assigns a score on a 0-10 scale, and the registry reports arm-level means and standard deviations. A mean describes the arithmetic center of the observed scores; a standard deviation describes how spread out those scores are around the mean. These summaries are compact descriptions, not a complete picture of the score distribution. They do not reveal whether scores are symmetric, clustered, or influenced by a small number of unusual observations.

What does a mean difference represent?

The posted comparison is a mean difference, expressed in the outcome's units. Its direction depends on the order of the arms: the estimate is Arm II relative to Arm I. The estimate of -0.24 means the observed Arm II average was lower by 0.24 units than the Arm I average, according to the posted comparison. This is an unadjusted summary comparison, not a model-based adjustment for baseline characteristics. The registry does not report an adjusted estimate.

A mean difference is useful because it remains on the same scale as the outcome and can be read alongside the ladder's range and meaning. It does not describe the proportion of participants whose readiness increased, nor does it indicate how many crossed the cut-off score of 5. Those require different summaries. A standardized effect size can express a difference relative to variability, but the posted standardized estimate is not needed to interpret the raw-unit result and is not reproduced.

How should the confidence interval be understood?

A confidence interval gives a range of values compatible with the data under the statistical method used. It communicates uncertainty around an estimate, rather than just whether the estimate is above or below zero. Here, the interval includes zero and reaches into both negative and positive values. Thus, the posted comparison does not sharply distinguish a difference favoring one arm from a difference in the other direction. The interval is also a reminder that the point estimate is not known without uncertainty.

The interval's interpretation depends on the model and assumptions used to calculate it. The detailed analysis procedure is not reported in the registry record. The posted result is therefore best described using the stated mean difference, interval, and unadjusted p-value, without assuming undocumented adjustments or methods.

What does the p-value contribute?

A p-value describes how surprising results at least as inconsistent with a specified null hypothesis would be if that hypothesis and the statistical model were true. It is not the probability that the null hypothesis is true, and it is not a measure of the clinical importance of a difference. The unadjusted p-value of 0.642 does not, on its own, establish equivalence or prove that there is no difference. Its interpretation belongs with the estimate and confidence interval, the study size, and the quality and completeness of the outcome data.

The word “unadjusted” matters because the registry does not report a multiplicity correction for this comparison. It also does not describe whether the comparison accounts for other outcomes, repeated measurements, or other features of the analysis. No analyses are listed in the registry, while the primary comparison is posted. The analysis plan and any distinctions between planned and reported analyses are not reported in the registry record.

Why can variation matter?

When scores vary considerably among participants, the group averages are less precise than they would be if participants' scores were more similar, all else equal. This is one reason standard deviations and confidence intervals should accompany an average comparison. Over-dispersion is a general concern when data vary more than a statistical model expects: uncertainty estimates can be too narrow if a model understates that variation. The registry does not describe a specific over-dispersion assessment, and no such assessment should be inferred from the posted summaries alone.

For a continuous outcome, a normal approximation is one possible basis for inference, but the analysis model is not reported in detail. The posted means and standard deviations do not establish that scores follow a normal distribution. With a bounded scale, values are constrained by the scale endpoints, and the underlying distribution may be important to the suitability of an analysis. Individual scores and diagnostic checks are not reported in the registry record.

What is not established by the summary statistics?

The outcome-specific counts are 24 in Arm I and 29 in Arm II, while total enrollment is 61. The record does not explain the difference, so it cannot be attributed to withdrawals, missing measurements, exclusions, or any other cause. The registry does not report the analysis population, missing-data method, baseline-adjustment method, or sensitivity analyses. These omissions limit the ability to independently assess whether the posted estimate could be affected by incomplete outcome data or analytic choices.

Randomization supports a between-arm comparison, but the interpretation still depends on follow-up, measurement, and analysis procedures. The registry's record does not report enough detail to reconstruct every step of the analysis. The result can be read as a posted comparison of average readiness-score change, while broader causal and behavioral interpretations require information that is not reported in the registry record.

10. Confidence Intervals and Precision

The mean-difference estimate is -0.24 units, with a confidence interval extending from -1.251 to 0.771. The interval includes zero and includes values in both directions. This indicates that the estimate is imprecise enough that the posted data do not isolate a single direction of difference. The point estimate is close to zero, but the interval is more informative than the point estimate alone because it displays the uncertainty around it.

In general, a confidence interval is a range generated by a statistical procedure that, under its assumptions, reflects uncertainty in the estimate. It is not a range containing a known proportion of individual participants' scores. Nor does inclusion of zero show that the arms are equivalent: equivalence requires a prespecified margin and an analysis designed to assess whether differences fall within that margin. An equivalence margin is not reported in the registry record.

The outcome-specific counts were 24 for Arm I and 29 for Arm II. Smaller groups generally provide less information about an average difference than larger groups, particularly when participant scores vary. The reported standard deviations were 2.0 and 1.7, respectively. These measures show that individual scores varied around the arm means; they do not reveal the full distribution or the precision of each participant's measurement.

The registry does not report the confidence-interval method, any baseline adjustment, or a missing-data analysis. Those details can affect uncertainty, so the interval should be understood as the one posted rather than as a complete account of all possible analytic uncertainty. The unadjusted p-value of 0.642 is consistent with uncertainty about the difference, but it is not a substitute for considering the interval and the scale's interpretation.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning estimates are participants per arm, not total events. They illustrate sample sizes under assumptions based on the observed effect; they are not the actual enrollment or outcome counts. For planning at 100% of the observed effect, the estimates are 925 participants per arm for 80% power and 1,238 participants per arm for 90% power. For planning at 75% of the observed effect, they are 1,643 participants per arm for 80% power and 2,200 participants per arm for 90% power. For planning at 50% of the observed effect, they are 3,697 participants per arm for 80% power and 4,949 participants per arm for 90% power.

The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. Each planning scenario sets the effect to detect as the stated fraction of the observed effect. The 80% power column describes a planning target under which the assumed study design would have an 80% chance of detecting an effect of that size if the assumptions and effect were true. The 90% column uses a higher chance-of-detection target. These are planning quantities, not guarantees that a future study will detect an effect.

The pattern is expected: smaller effects require more participants to distinguish them from random variation with the same target power and significance level. A sample-size curve, when read alongside these estimates, represents the relationship between the effect targeted and the required sample size under the stated assumptions. These calculations are illustrations based on the observed effect, not the trial's own design assumptions; those original assumptions are not reported in the registry record. This is not observed power.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -0.240; pooled standard deviation σ = 1.841.
2. n = 2 × (1.96 + 0.842)² × 1.841² / -0.240² = 924.0, rounded up to 925 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.24                      # observed difference in means
sigma <- 1.84135499980154                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(5897242)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 925 (80% power), 1,238 (90% power)
 75% of observed effect: n per arm = 1,643 (80% power), 2,200 (90% power)
 50% of observed effect: n per arm = 3,697 (80% power), 4,949 (90% power)
power.t.test: 925 
Simulated power at n = 925 per arm: 0.801 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry lists 4 outcome measures, but no secondary results are posted and no analyses are listed. One primary outcome comparison is posted, with an unadjusted p-value. The record does not describe whether the study tested additional outcomes, multiple time points, subgroups, or alternative models, or whether any adjustment for multiple comparisons was planned or applied. It is therefore not possible to assess the overall family of statistical tests from the registry record.

Multiplicity is a general issue when many hypotheses are tested: the chance of finding at least one apparently notable result can increase as the number of tests grows. Methods that control or otherwise address multiplicity can be prespecified, but the registry record does not report a multiplicity strategy. The absence of posted secondary results is not evidence that no other outcomes were collected or examined; it means those results are not reported in the registry record.

The record describes the primary outcome as change from baseline to 3 months post-randomization, but does not report how missing scores were handled. It does not state whether participants with incomplete follow-up were excluded, whether any imputation method was used, or whether sensitivity analyses explored alternative assumptions about missing data. The outcome counts differ from total enrollment, but the reason is not reported and should not be inferred from the counts alone.

The analysis population is also not specified in enough detail to determine whether the posted comparison followed an intention-to-treat, per-protocol, or other definition. The registry notes descriptive statistics for the ACT on Vaping treatment group and comparisons between arms on other indicators, but it does not provide the analysis-population rules. Whether the analysis adjusted for baseline readiness or other participant characteristics is not reported in the registry record.

These omissions matter because choices about missing outcomes, analysis populations, and multiple testing can influence estimates and their interpretation. The posted comparison remains interpretable as the reported unadjusted result, but a full appraisal of robustness and selective reporting would require analysis details that are not reported in the registry record.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent non-serious adverse event terms

The registry reports serious adverse events for groups with 31 participants at risk in Arm I and 30 in Arm II. No serious adverse events affected participants in either group, and no deaths were reported. These are the serious-event counts recorded for the listed groups; they should not be read as proof that no other type of event occurred.

For other adverse events, 1 of the 31 participants at risk in Arm I was affected, and 3 of the 30 participants at risk in Arm II were affected. The registry does not list serious adverse-event terms. These counts describe participants affected, not necessarily the total number of event episodes. The record does not provide further information about event types, severity, timing, attribution, or how events were collected.

The adverse-event denominators differ from the outcome-specific counts for readiness. The registry does not explain the differences, so no reason for them can be assigned. Safety summaries are descriptive and do not establish whether an event was caused by an intervention. They also do not support a broad comparison of safety without detail about ascertainment, follow-up, and event definitions. The counts should be interpreted within the limits of the registry's reporting.

15. What the Result Does, and Does Not, Mean

The posted data show a mean readiness-score change of 0.96 units in the ACT on Vaping app arm and 0.72 units in the incentivized text-message check-in arm. The estimated difference, defined as Arm II relative to Arm I, was -0.24 units. The confidence interval spans values on both sides of zero, and the unadjusted p-value was 0.642. Together, these statistics describe an uncertain average difference in the measured readiness outcome.

The result does not establish that the approaches have the same effect, nor does it establish that either approach changed readiness by a particular amount for every participant. It is not an equivalence analysis. The mean summarizes a group; individual responses may differ, and individual score changes are not reported. The available outcome counts also do not represent the total enrollment, and the reason for the difference is not reported in the registry record.

Readiness to quit is not the same as quitting. The primary outcome does not directly report abstinence, vaping frequency, or long-term cessation. Although the registry describes ACT on Vaping as a program for young adult e-cigarette users and lists several intervention components, the posted primary comparison cannot establish the program's effect on outcomes that are not reported. Secondary results are not posted.

The trial's randomized parallel-group design supports comparing the assigned arms, but interpretation depends on how outcomes were collected and analyzed. Details about missing data, analysis populations, baseline adjustment, and the statistical plan are not reported in the registry record. The posted summaries are therefore best treated as a limited description of the observed readiness-score results, not as a complete evaluation of every aspect of the intervention.

In general, a non-significant p-value does not show that a meaningful difference is impossible. Conversely, a numerical difference between sample means does not establish that a true difference exists. The interval and the modest outcome-specific counts illustrate the uncertainty in this comparison. Claims about participants beyond those represented in the study would also depend on eligibility and population details, which are not reported in the registry record.

16. Why This Trial Matters Statistically

This trial raises a statistical question that is easy to blur: whether an intervention changes a readiness measure is different from whether it changes vaping behavior. Defining readiness as the primary outcome makes the target of the comparison explicit. The Contemplation Ladder is continuous, while the registry also describes a cut-off for high versus low readiness. Those two ways of describing the measure answer different questions; the posted mean comparison does not reveal how many participants moved across the cut-off.

The randomized comparison also separates the app from a check-in-only comparator more clearly than a single-arm description would. Still, the result depends on precision as well as design. The confidence interval covers differences in both directions, illustrating why a point estimate should not be treated as a settled answer. The arm-level variability and outcome counts are relevant to how much information the comparison contains.

Planning calculations based on an observed effect can help readers understand how assumed effect size and target power relate to required enrollment. In this record, the planning estimates increase as the target effect becomes a smaller fraction of the observed effect. This is a general feature of power calculations, not evidence that the original trial was designed around those estimates. The original design assumptions are not reported in the registry record.

Finally, the record illustrates the value of transparent reporting beyond a single outcome estimate. Missing-data procedures, analysis populations, multiplicity plans, and secondary results can determine how a finding should be interpreted. Their absence does not erase the posted comparison, but it limits independent assessment of its robustness and scope.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several trial-specific limitations constrain interpretation. The outcome-specific summaries include 24 participants in Arm I and 29 in Arm II, while total enrollment was 61. The registry does not explain this difference. Without a participant flow account or missing-data description, the relationship between enrollment, follow-up, and the analyzed outcome records is unclear.

The primary measure is self-reported readiness on a 1-item ladder. The registry gives the scale range and cut-off anchor, but does not report measurement properties, the detailed calculation of change, or individual score distributions. A mean change can obscure participants with different directions or magnitudes of change. The posted result does not show whether scores crossed the cut-off, nor does it directly measure quitting.

The comparator received incentivized check-ins, so the comparison is not app access versus no contact. That distinction is important for interpreting the contrast: the arms differ in access to the app and supportive messages, while both include check-ins. The registry does not report the content, frequency, or adherence to the messages and app. Engagement and treatment exposure therefore cannot be assessed from the posted record.

The registry does not report the age range, detailed eligibility criteria, participant characteristics, randomization procedure, masking details, or a full statistical analysis plan. It also does not report how missing data were handled, whether baseline readiness was incorporated into the analysis, or whether sensitivity analyses were conducted. These details matter for assessing comparability, potential bias, and robustness.

Finally, no secondary results are posted, despite the record listing 4 outcome measures, and no analyses are listed. The posted evidence is consequently narrow: a primary readiness comparison and limited adverse-event counts. These limitations do not determine the direction of an unreported effect; they describe what cannot be evaluated from the registry record.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: A Smartphone Application (ACT on Vaping) for Vaping Cessation in Young Adults — NCT05897242.
• Registry record last updated 2025-06-24; status last verified 2025-06.
• Results first posted 2025-06-06.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT05897242, titled “A Smartphone Application (ACT on Vaping) for Vaping Cessation in Young Adults,” is a completed, randomized, single-masked, parallel-group trial sponsored by Fred Hutchinson Cancer Center. It enrolled 61 participants. The study compared ACT on Vaping app use with supportive messages and incentivized vaping-status check-ins against incentivized check-ins alone.

The primary outcome was change in Contemplation Ladder readiness-to-quit scores from baseline to 3 months post-randomization. The posted mean changes were 0.96 in the app arm and 0.72 in the check-in arm, with an estimated Arm II relative to Arm I mean difference of -0.24. The confidence interval includes zero, and the unadjusted p-value was 0.642.

The registry lists 4 outcome measures, no analyses, and no posted secondary results. Detailed information about missing-data handling, analysis populations, and the original statistical plan is not reported in the registry record.