Smartphone Based Smoking Cessation Intervention: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Smartphone Based Smoking Cessation Intervention (NCT03740490), a randomized phase 2, phase 3 study. The primary outcome is Biochemically-Verified Smoking Cessation.

2. Clinical Question

3. Trial Design

This completed study was a randomized, open-label, parallel-group trial listed in phase 2 and phase 3. The registry lists an enrollment of 454. Participants were assigned to Smart-T + NRT or NCI QuitGuide + NRT. The posted primary outcome counts include 225 participants in the Smart-T + NRT group and 229 in the NCI QuitGuide + NRT group. The registry lists the University of Oklahoma as sponsor.

The primary outcome was assessed at 27 weeks. It was defined as biochemically confirmed 7-day point-prevalence abstinence using a carbon monoxide cutoff of ≤ 6 ppm. The posted results also include measures of urge to quit smoking, stress, perceived cigarette availability, and motivation to quit smoking. Results are posted at 27 weeks for the first three of these secondary measures; results for change in motivation to quit smoking are not reported in the registry record.

The registry lists the study start as 2019-08-26 and primary completion as 2023-11-17, with status completed. The reported enrollment is the planned or recorded total enrollment entry; it should not be confused with the number contributing to a particular outcome. Outcome-specific denominators are shown with the results, and they can differ across measures.

Details such as the original sample-size justification, eligibility rules, recruitment procedures, allocation concealment, follow-up procedures, and the prespecified statistical analysis plan are not reported in the registry record. The record also does not report whether interim analyses were planned or conducted. Those omissions limit how fully the design and analysis decisions can be reconstructed from the registry alone.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Biochemically-Verified Smoking Cessation, measured over 27 Weeks.

The primary endpoint is biochemically verified 7-day point-prevalence abstinence at 27 weeks. It classifies whether a participant met the stated abstinence definition at the assessment, rather than measuring the full history of smoking behavior across the entire follow-up period. The biochemical criterion was a carbon monoxide cutoff of ≤ 6 ppm, paired with self-reported abstinence over the specified 7-day period.

Biochemical verification is used to supplement self-report with an objective measure related to recent smoking. That can reduce reliance on self-report alone, although it does not make the endpoint a measure of continuous abstinence. A point-prevalence measure describes status at a particular assessment window; it does not establish that a participant remained abstinent throughout the weeks before that assessment.

The registry reports this endpoint as a count of participants meeting the definition. Counts can be interpreted alongside the number assessed in each group, which is why both the event count and denominator matter. The outcome is relevant to the trial’s comparison because the study was designed to compare cessation rates between two smartphone-based app approaches, each paired with NRT. It does not by itself identify which component of either combination accounts for any group difference.

7. Results Posted in the Registry

• NCI QuitGuide + NRT: 10.0% (23/229) versus 16.4% (37/225) on control; risk difference -6.4 percentage points (95% CI -12.6 to -0.2), risk ratio 0.61 (0.38–0.99); unadjusted two-proportion z-test p = 0.044.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Change in Urge to Quit Smoking

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • NCI QuitGuide + NRT: mean 0.02 versus 0.05 on control; difference -0.04 (95% CI -0.08 to 0.01), standardized effect d = -0.15; z-test p = 0.114.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NCI QuitGuide + NRT and the control. The interval gives the range of effects that remain compatible with the data.

Change in Stress

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • NCI QuitGuide + NRT: mean -0.03 versus -0.01 on control; difference -0.03 (95% CI -0.06 to 0.01), standardized effect d = -0.15; z-test p = 0.129.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NCI QuitGuide + NRT and the control. The interval gives the range of effects that remain compatible with the data.

Change in Perceived Cigarette Availability

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • NCI QuitGuide + NRT: mean -0.01 versus 0.01 on control; difference -0.02 (95% CI -0.05 to 0.01), standardized effect d = -0.14; z-test p = 0.150.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NCI QuitGuide + NRT and the control. The interval gives the range of effects that remain compatible with the data.

The secondary outcomes cover several related but distinct dimensions: urge to quit smoking, stress, perceived cigarette availability, and motivation to quit smoking. The registry posts results at 27 weeks for the first three, while results for motivation to quit smoking are not reported in the registry record. Across the posted secondary results, the overall pattern does not show a consistent, clear separation between the randomized groups.

This broad pattern does not directly explain the primary abstinence comparison. The secondary measures are not substitutes for biochemically verified abstinence, and a change in a scale score does not establish why a participant did or did not meet the primary endpoint. Nor does a lack of a clear group difference on a secondary measure demonstrate that the groups were identical on that measure. The estimates and intervals remain the relevant summaries of uncertainty.

Several outcomes and analyses create opportunities for chance findings when each is assessed separately. Unadjusted secondary results carry less weight when the registry does not report a prespecified multiplicity strategy, because repeated testing can increase the chance of at least one apparently notable result by chance. The registry record does not report whether secondary outcomes were designated as confirmatory or exploratory, or whether any adjustment was planned.

The sponsor-reported mixed-model analyses are the registry’s reported inferential results for the posted secondary measures. Simple unadjusted recalculations from summary statistics are useful for orientation but may not match those analyses because methods and estimands can differ. Conclusions should therefore rely on the full set of outcome estimates, intervals, and analysis context, rather than selecting a single secondary finding as decisive evidence about the primary comparison.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Change in Urge to Quit Smoking
• Change in Stress
• Change in Perceived Cigarette Availability
• Change in Motivation to Quit Smoking

9. Statistical Interpretation

10. Statistical Methods Explained

Randomization assigns participants to groups by chance, with the aim of making the groups comparable on average at baseline. It supports a causal comparison of the assigned treatment strategies when the trial is conducted and analyzed as planned. Randomization does not ensure that every characteristic is balanced in a particular trial, and the registry record does not report the allocation procedure or concealment details.

What does the risk difference express?

A risk difference subtracts one group’s event proportion from the other group’s event proportion. Here the comparison is NCI QuitGuide + NRT minus Smart-T + NRT. The posted estimate is -0.064. A negative value indicates a lower observed proportion in the first-named comparison group under that direction. Unlike a ratio, a risk difference is on the absolute probability scale, so it directly describes the size and direction of the contrast in proportions.

The sign depends on which group is subtracted from which. It is therefore important to read the comparison direction as well as the estimate. Reversing the order would reverse the sign without changing the underlying group data. The arm counts and proportions make the direction interpretable without relying on a sign alone.

What do the confidence interval and p-value contribute?

A confidence interval describes a range of effect estimates compatible with the data and the method’s assumptions. It communicates uncertainty about the estimate rather than giving a guarantee that the true effect lies in the particular interval. For the primary recalculation, the interval extends from -0.126 to -0.002. Because the interval is close to zero at one end, the data do not pin down the magnitude with high precision.

A p-value measures how unusual data at least as incompatible with a stated null hypothesis would be under the statistical model, if that null hypothesis and the model assumptions held. It is not the probability that the null hypothesis is true, and it does not quantify clinical importance. The recalculated primary p-value is 0.044. It is unadjusted, and the registry does not report a multiplicity plan for the family of outcomes or analyses.

What does an unadjusted recalculation mean?

The recalculated primary comparison uses the posted event counts and group denominators to form a simple, unadjusted comparison. It is not a reconstruction of an analysis that accounts for covariates, repeated observations, or a missing-data model. No such primary analysis details are reported in the registry record. Consequently, the recalculation is useful for describing the posted summaries, but it cannot substitute for a complete statistical analysis plan or an individual-participant analysis.

For secondary outcomes, the posted group means and standard deviations permit unadjusted comparisons of means. The registry also reports sponsor analyses using mixed models for three secondary measures. A mixed model can account for structured measurements and correlation among observations, depending on its specification. The detailed model specification, covariates, handling of missing observations, and analysis population are not reported in the registry record. The model name alone is not enough to establish exactly what was estimated.

Why can secondary estimates differ by method?

An unadjusted comparison based on group summaries and a sponsor-reported mixed-model estimate need not be identical. They can target different quantities or use different assumptions about measurements, covariates, and incomplete observations. The recalculated mean differences are therefore simple descriptions based on posted summaries, while the mixed-model values are the analyses identified in the registry as reported results. Without the full analysis details, the distinction cannot be resolved further.

For continuous outcomes, the standard deviation describes variation among observed scores; it is not the uncertainty of the group mean itself. For binary outcomes, over-dispersion is a general concern when the observed variation exceeds what a simple model assumes. It can affect standard errors and confidence intervals. The registry does not report an over-dispersion assessment for the primary binary endpoint, so no claim about such an assessment can be made.

The registry lists planning illustrations derived from the observed effect, using a two-sided significance level of 0.05, equal allocation, and a normal approximation. These calculations are design-oriented scenarios. They do not describe the original trial’s design rationale, which is not reported in the registry record.

11. Confidence Intervals and Precision

The primary risk-difference confidence interval runs from -0.126 to -0.002, with the comparison defined as NCI QuitGuide + NRT minus Smart-T + NRT. It gives a range of estimates compatible with the posted counts under the unadjusted recalculation method. All values in that interval are negative, but the endpoint nearest zero is close to no difference. Thus, the direction is more clearly indicated than the exact magnitude.

Precision is not the same as whether an estimate is important. The interval communicates uncertainty in the estimate, while judgments about importance require context not reported in the registry record. The posted result does not establish that the observed contrast will recur at the same size elsewhere. It also does not account for potential differences in populations, implementation, follow-up, or analysis choices.

The relative-measure intervals likewise extend close to their null values: 1 for a ratio. The risk ratio interval is 0.375 to 0.994, and the odds ratio interval is 0.325 to 0.990. These scales can make the comparison appear different from the absolute risk difference, so the scale and direction should be stated when interpreting them.

Confidence intervals reflect sampling uncertainty under their calculation assumptions. They do not account automatically for systematic errors, selective reporting, unmeasured differences, or uncertainty about the registry’s unreported analysis decisions. The registry does not report adjusted primary results or details about missing-data handling. The reported precision should therefore be understood as conditional on the posted summaries and the unadjusted method, rather than as a complete measure of every source of uncertainty.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size figures are planning illustrations based on fractions of the observed effect, not the trial’s original design justification. For this binary primary outcome, the entries are participants per arm. The assumptions listed for these illustrations are a two-sided significance level of 0.05, equal allocation between the arms, and a normal approximation. Each scenario sets the effect to detect at the stated fraction of the observed effect.

The 80% power column gives the estimated participants per arm for a design with an 80% chance of detecting the assumed effect under the planning assumptions, if that effect and the assumptions were correct. The 90% power column gives the corresponding per-arm planning figure for a higher chance of detection under those same assumptions. These are hypothetical design calculations; they are not observed trial results.

The rows use 100% of the observed effect, 75% of the observed effect, and 50% of the observed effect. As the assumed effect becomes smaller, the planning calculation requires more participants per arm to distinguish it from no difference with the stated power. This pattern reflects the greater difficulty of detecting a smaller contrast when other assumptions stay fixed.

These figures are not observed power. They do not estimate the probability that the completed trial “had enough power” after seeing its results. They illustrate sample sizes under specified assumptions, using the observed effect as a starting point. Because observed effects can be uncertain, planning from them is only one scenario; the original sample-size assumptions and justification are not reported in the registry record.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.164; treatment proportion p₁ = 0.100; average p̄ = 0.132.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.4794 = 0.9396.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.4772 = 0.4016.
4. n = (0.9396 + 0.4016)² / (-0.0640)² = 439.1, rounded up to 440 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.164444444444444                          # control proportion
p1 <- 0.100436681222707                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3740490)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 440 (80% power), 588 (90% power)
 75% of observed effect: n per arm = 822 (80% power), 1,099 (90% power)
 50% of observed effect: n per arm = 1,937 (80% power), 2,592 (90% power)
power.prop.test: 439.1 
Simulated power at n = 440 per arm: 0.805 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry lists one primary outcome and several secondary outcomes, but a multiplicity strategy is not reported in the registry record. Multiplicity matters when investigators evaluate multiple outcomes or perform multiple analyses: even if each test is considered separately, the chance of a chance finding somewhere among them can rise. Without a reported plan, the primary p-value and secondary p-values should be treated as unadjusted results rather than as evidence that has been corrected for the full set of tests.

The record reports the numbers included in the posted outcome summaries, but it does not describe the analysis population, such as whether the primary analysis followed intention-to-treat principles. It also does not report how participants with missing outcome assessments were handled, whether any imputation was used, or whether sensitivity analyses assessed alternative missing-data assumptions. Since missingness can affect both the group proportions and their uncertainty, these details are relevant to interpretation.

The registry does not report the full primary statistical model, covariate adjustment, a prespecified analysis plan, or interim analyses. Those omissions do not prove that no such methods were used; they mean the information is not reported in the registry record. The primary comparison discussed in the interpretation is consequently an unadjusted recalculation from the posted group counts, not a substitute for an unpublished or unavailable analysis plan.

For secondary outcomes, mixed-model analyses are identified, but the model specification and missing-data assumptions are not reported in the registry record. An analysis label alone cannot establish the exact estimand or how incomplete repeated observations were handled. These limitations make it important to separate the registry’s reported results from assumptions that cannot be verified from the record.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The serious adverse event figures report zero participants with serious adverse events in each group: 0 among 225 at risk for Smart-T + NRT and 0 among 229 at risk for NCI QuitGuide + NRT. The registry lists no serious adverse event terms. These figures describe the serious adverse event reporting recorded for the groups; they do not establish that no other health events occurred.

The registry also reports 2 deaths in the Smart-T + NRT group and 0 in the NCI QuitGuide + NRT group. The record does not state that these deaths were caused by either intervention, and the figures alone cannot establish a causal relationship. They should be read as reported counts, not as an attribution of cause.

Safety counts are descriptive and should be interpreted in light of the number at risk, duration of observation, and completeness of reporting. The registry does not provide further details about event timing, causes of death, or relationships to treatment. Therefore, the figures do not support a broader comparison of safety beyond the reported counts.

16. What the Result Does, and Does Not, Mean

The primary result describes a difference in observed biochemically verified abstinence proportions between two randomized app-plus-NRT groups at 27 weeks. More participants in the Smart-T + NRT group met the posted outcome definition than in the NCI QuitGuide + NRT group. The unadjusted recalculation estimates the comparison on absolute and relative scales, with a confidence interval that indicates uncertainty in the magnitude.

The result does not compare either app-plus-NRT approach with no treatment, NRT alone, or an app without NRT. Because both groups received NRT, the between-group contrast cannot isolate an effect attributable only to the app. It also does not establish that the Smart-T app’s tailoring algorithm, ecological momentary assessment, or any single app component caused the observed group difference.

The endpoint is point-prevalence abstinence at a specified follow-up, not continuous abstinence across the entire follow-up period. Meeting the endpoint does not document every participant’s smoking history between assessments. The result also does not establish that the same contrast would occur in populations with different characteristics, under different implementation conditions, or at other follow-up times.

The primary recalculation is unadjusted, and the registry does not report the full primary analysis plan, missing-data approach, analysis population, or multiplicity strategy. Those omissions limit assessment of how robust the estimate is to alternative analytic decisions. A p-value does not measure effect importance, and a confidence interval does not remove uncertainty from missing information or systematic limitations.

Accordingly, the posted data support a careful description of the observed group contrast, not a blanket conclusion about treatment effectiveness or clinical choice. They do not show that every participant would have the same outcome, and they do not establish a mechanism. Interpretation should remain specific to the trial’s comparison, endpoint, follow-up, and reported analysis details.

17. Why This Trial Matters Statistically

This trial raises a useful statistical question about comparing two active, technology-based cessation approaches rather than comparing treatment with no intervention. The shared NRT component makes the estimand a contrast between app-plus-NRT strategies. That distinction matters because a group difference cannot be attributed to the app alone when the registry does not include an NRT-only or app-free comparison.

The primary outcome is binary and is reported both as counts and as proportions. Showing the denominators alongside the event counts is essential: a count alone does not describe the chance of meeting the endpoint in each group. Reporting the comparison on absolute and relative scales also helps readers distinguish the size of the difference in proportions from a ratio-based contrast.

The result illustrates why an estimate and its uncertainty should be considered together. The recalculated interval lies just below no difference, while spanning a range of possible negative contrasts. A single p-value cannot communicate that range or replace consideration of the estimate’s precision. The secondary outcomes further illustrate that multiple endpoints require careful attention to prespecification and multiplicity, neither of which is detailed in the registry record.

The planning illustrations connect observed effects to hypothetical future sample-size scenarios, but they are not the original study’s rationale or a measure of realized power. Separating planning calculations from completed-trial evidence prevents a common misreading. Taken together, the record encourages attention to estimands, outcome definitions, uncertainty, multiplicity, and transparent reporting of analysis decisions in app-based clinical trials.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to interpreting this registry record. The primary comparison is an unadjusted recalculation from the posted counts and denominators. A separate sponsor-reported primary analysis, its model, and any covariate adjustment are not reported in the registry record. The result should therefore not be mistaken for a full reconstruction of the trial’s planned analysis.

The registry does not report eligibility criteria in sufficient detail to establish how closely the enrolled participants match the lower-socioeconomic-status population emphasized in the study description. Nor does it report detailed recruitment, adherence, app engagement, or NRT use. Without these details, the applicability of the observed comparison to particular groups or implementation settings is uncertain.

The 27-week point-prevalence endpoint captures abstinence status over the specified assessment window, not continuous abstinence. Details about missing outcome observations, the analysis population, and sensitivity analyses are not reported in the registry record. The posted denominators are useful, but they do not reveal why outcome numbers may differ across measures or how missing observations were treated.

The open-label design means participants and study personnel were not masked according to the registry. The record does not describe how this may have affected reporting or measurement beyond the biochemical confirmation component. The registry also does not report the multiplicity plan, original sample-size justification, interim analyses, or complete analysis specification. These gaps constrain interpretation of nominal p-values, secondary findings, and the stability of the estimates.

Finally, the serious adverse event summary contains no event terms and limited detail about the reported deaths. Timing, cause, and relationship to treatment are not reported in the registry record. The available figures therefore support only a narrow description of reported counts.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: Smartphone Based Smoking Cessation Intervention — NCT03740490.
• Registry record last updated 2025-08-17; status last verified 2025-08.
• Results first posted 2025-07-31.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

The registry record describes a completed, randomized, open-label, parallel-group phase 2 and phase 3 trial sponsored by the University of Oklahoma, with enrollment of 454. It compared Smart-T + NRT with NCI QuitGuide + NRT. The primary endpoint was biochemically verified 7-day point-prevalence abstinence at 27 weeks using a carbon monoxide cutoff of ≤ 6 ppm.

The posted primary counts were 37 of 225 for Smart-T + NRT and 23 of 229 for NCI QuitGuide + NRT. An unadjusted recalculation gives a risk difference of -0.064 for QuitGuide + NRT minus Smart-T + NRT, with a 95% confidence interval from -0.126 to -0.002 and p = 0.044. The registry also reports secondary outcome analyses and serious adverse event counts. Important details about the primary analysis plan, missing data, multiplicity, and original sample-size justification are not reported in the registry record.