Does Switching to Nicotine Containing Electronic Cigarettes Reduce Health Risk Markers: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Does Switching to Nicotine Containing Electronic Cigarettes Reduce Health Risk Markers (NCT03625986), a randomized na study. The primary outcome is Urinary NNAL Concentration.

2. Clinical Question

3. Trial Design

The registry describes a randomized, quadruple-masked, parallel-group trial of electronic cigarettes for current smokers with tobacco dependence. The two groups differed in the nicotine concentration of the liquid: 58 mg/ml in the nicotine-containing group and 0 mg/ml in the non-nicotine group. Both interventions were intended for use over 6 weeks, with participants encouraged to switch completely from combustible cigarettes.

The registry lists enrollment as 105 and the study status as completed. The recorded start date is 2022-04-22 and the primary completion date is 2024-01-26. These registry fields describe the study record; the dates do not provide information about when particular participants enrolled or completed follow-up. Phase is listed as NA.

The primary endpoint was assessed at the 6-week visit after treatment. The posted primary summaries include 33 participants in the nicotine-containing group and 32 in the non-nicotine group. The endpoint description states that four values were missing because samples could not be analyzed. The registry also posts secondary-outcome results, including pulmonary function, exhaled carbon monoxide, urine cotinine, blood pressure, nicotine-dependence measures, cigarettes per day, and abstinence from cigarettes and other tobacco. Details of randomization procedures, the masking procedures, adherence, and the trial’s original sample-size justification are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: One participant was excluded from the study before their assignment to a group because she did not meet the eligibility criteria for blood pressure.

Randomization Period

Reasons for not completing this period:

10 Week Post-treatment

Analysis population for the primary outcome, as defined in the registry: Four missing values because sample could not be analyzed

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Urinary NNAL Concentration, measured over 6-week visit (post-treatment).

The primary endpoint is urinary NNAL concentration, measured in ng/ml at the 6-week post-treatment visit. The registry describes NNAL as a urinary carcinogen biomarker of exposure and states that the concentration was corrected for creatinine. Creatinine correction is used to account for variation in urine concentration when reporting a urinary biomarker. The endpoint is therefore an exposure marker, not a direct clinical event or a complete measure of health.

The registry’s endpoint description makes it relevant to the study’s stated interest in markers of health risk after switching from combustible cigarettes to electronic cigarettes. The primary outcome summaries report a mean and standard deviation for each group at the follow-up visit. The posted comparison is between those groups at that visit. No baseline NNAL values or baseline-to-follow-up change estimates are included in the posted primary results, so the between-group result should not be interpreted as a quantified within-person change from smoking baseline.

Four primary-outcome values were missing because the samples could not be analyzed. The registry does not report further information about the values, such as their distribution across groups or whether additional methods were used to address missingness.

7. Results Posted in the Registry

• Non-Nicotine Electronic Cigarette: mean 338.20 versus 151.60 on control; difference 186.60 (95% CI 63.34 to 309.86), standardized effect d = 0.74; z-test p = 0.003.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Pulmonary Function

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Non-Nicotine Electronic Cigarette: mean 2.70 versus 2.70 on control; difference 0.00 (95% CI -0.38 to 0.38), standardized effect d = 0.00; z-test p = 1.000.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Non-Nicotine Electronic Cigarette and the control. The interval gives the range of effects that remain compatible with the data.

Exhaled Carbon Monoxide (CO)

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Non-Nicotine Electronic Cigarette: mean 13.80 versus 10.70 on control; difference 3.10 (95% CI -4.23 to 10.43), standardized effect d = 0.20; z-test p = 0.407.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Non-Nicotine Electronic Cigarette and the control. The interval gives the range of effects that remain compatible with the data.

Urine Cotinine Concentration

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Non-Nicotine Electronic Cigarette: mean 2,880 versus 5,296 on control; difference -2,416 (95% CI -4,358 to -473.75), standardized effect d = -0.60; z-test p = 0.015.
• Interpretation:
  • The 95% confidence interval excludes no difference, so the mean is statistically lower for Non-Nicotine Electronic Cigarette than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.

The posted secondary outcomes cover different domains: pulmonary function, exhaled carbon monoxide, urine cotinine, blood pressure, nicotine dependence, electronic-cigarette dependence, cigarette consumption, and abstinence from cigarettes and other tobacco. They do not all measure the same underlying construct as urinary NNAL. In particular, an exposure marker, a functional measurement, a dependence score, and a tobacco-use outcome each address a distinct question. Their results therefore provide context rather than a single combined confirmation of the primary comparison.

Across the secondary results, the pattern is not uniformly precise. Some unadjusted comparisons are close to no observed difference, while others show group contrasts with intervals that exclude zero. The posted secondary results should be interpreted by their specific measures and uncertainty, rather than by counting results in one direction or treating each as an independent replication of the primary endpoint. The registry lists 16 posted secondary outcomes, while the broad outcome list includes measures that are not all represented in the posted secondary summaries.

Multiple outcomes create multiple opportunities for apparently noteworthy findings. When many tests are considered, some may appear unusual by chance even if there is no underlying difference for those outcomes. The registry does not report a multiplicity adjustment or a hierarchy for interpreting secondary outcomes. Consequently, unadjusted secondary p-values carry less weight than a result specified as primary, and should be treated as exploratory evidence rather than definitive confirmation. The registry record also does not report whether particular secondary comparisons were prespecified or how missing observations were handled for each measure.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Pulmonary Function
• Exhaled Carbon Monoxide (CO)
• Urine Cotinine Concentration
• Blood Pressure
• Fagerstrom Test for Nicotine Dependence Mean Total Score
• Penn State Electronic Cigarette Dependence Index
• Cigarettes Per Day
• Number of Participants With Abstinence From Cigarettes and Other Tobacco (Not Including E-cigs)

9. Statistical Interpretation

10. Statistical Methods Explained

The primary result is reported as a continuous outcome: urinary NNAL concentration in ng/ml. For each group, the registry posts a mean, standard deviation, and analysis count. A mean describes the arithmetic average, while a standard deviation describes the spread of individual values around that average. These summaries are useful for understanding scale and variability, but they do not show the full distribution of the measurements.

The recalculated comparison is an unadjusted difference between the posted group means. The contrast is defined as the non-nicotine group minus the nicotine-containing group. Thus, a positive difference corresponds to a higher posted mean in the non-nicotine group. The recalculated estimate is 186.6 ng/ml. “Unadjusted” means that the comparison is calculated from the posted summary statistics without adjustment for other variables. The registry does not report adjusted estimates for this endpoint.

A confidence interval gives a range of values compatible with the data and the calculation’s assumptions. For the primary recalculation, the 95% interval runs from 63.34 to 309.9 ng/ml. Because the interval excludes zero, it is consistent with a nonzero difference under the unadjusted calculation. The interval does not mean that the true difference has a 95% probability of falling inside these particular limits. It describes the performance of the interval procedure across repeated studies under its assumptions.

The posted primary analysis used a Wilcoxon (Mann-Whitney) method. This is a rank-based method: it uses the ordering of observations rather than relying only on their numerical distances from the mean. It is often useful when an outcome’s distribution may be skewed or when extreme values could strongly affect arithmetic summaries. The registry labels the reported parameter “Mean Difference (Final Values),” but the listed test is rank-based. A rank-based test and a comparison of arithmetic means do not necessarily answer precisely the same statistical question. The recalculated mean difference and confidence interval should therefore be read as a simple summary-statistic comparison, not as a reproduction of the sponsor’s rank-based procedure.

Over-dispersion is a general concern when measurements vary widely relative to their average. In such settings, a small number of high observations can have a noticeable influence on the mean and standard deviation. Rank-based methods can be less sensitive to the magnitude of extreme values, although they have their own interpretation and assumptions. The posted primary standard deviations are 235.1 and 270.1, compared with group means of 151.6 and 338.2. These summaries indicate substantial variability, but do not reveal the detailed distribution or establish whether particular observations are extreme.

The p-value is a measure of how incompatible the observed data are with a specified null hypothesis, under the test’s assumptions. It is not the probability that the null hypothesis is true, and it does not measure the size or clinical importance of a difference. The recalculated primary p-value is unadjusted. The registry also reports the result of its own test; the test and its reported estimate should be distinguished from the recalculated comparison based on means.

For secondary outcomes, the registry posts additional comparisons using different measurement scales and, in some cases, different analysis counts. Their estimates should be interpreted on their own scales. A confidence interval spanning zero indicates that the corresponding unadjusted estimate is compatible with both directions of difference under the calculation, while an interval excluding zero indicates greater evidence against a zero contrast under those assumptions. Neither finding by itself establishes a clinical consequence.

These calculations are unadjusted and do not account for potential covariates, repeated testing across outcomes, or missing-data uncertainty. Whether the sponsor’s analysis used additional procedures is not reported in the registry record. The registry record also does not provide enough detail to verify distributional assumptions, the analysis population definition, or any prespecified analysis plan.

11. Confidence Intervals and Precision

The primary recalculated 95% confidence interval for the difference in mean NNAL concentration is 63.34 to 309.9 ng/ml, using the non-nicotine group minus the nicotine-containing group. The interval is entirely above zero, so the unadjusted summary-statistic comparison is compatible with a higher mean in the non-nicotine group. Its width also matters: it leaves room for a range of possible differences, rather than identifying one exact value.

Precision is affected by the amount of information and the variability in the outcome. The posted primary standard deviations are large, and four values were missing because samples could not be analyzed. The registry does not report whether those missing values differed by group or by participant characteristics. The interval calculation does not resolve that missing-data uncertainty.

A confidence interval is not a guarantee that the true effect lies within its limits, nor does it show the probability that a particular effect size is correct. It reflects uncertainty under the calculation’s assumptions. The recalculated interval is based on posted summary statistics, whereas the registry’s own analysis used a Wilcoxon (Mann-Whitney) method and did not post confidence limits for that analysis. The two forms of analysis can differ in their assumptions and interpretation.

The interval concerns a between-group mean difference at the 6-week visit. It does not quantify baseline change, effects beyond the visit, or clinical outcomes. For the secondary outcomes, the intervals vary in width and some include zero. Those differences in precision should be considered alongside the exploratory nature of multiple outcome comparisons.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size planning figures are participants per arm, not total events. They are illustrations based on the observed effect and are not the trial’s original design calculations. The planning assumptions listed in the registry-derived information are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. Each scenario asks what would be required to detect the stated fraction of the observed effect.

The 80% power column gives the number of participants per arm in a planning scenario designed to have an 80% chance of detecting the assumed effect if that effect were the true one, using the stated assumptions. The 90% power column represents a higher target chance of detection under the same kind of hypothetical planning calculation. These are prospective planning quantities, not descriptions of what happened in the completed trial.

Rows using 100%, 75%, and 50% of the observed effect illustrate how the assumed effect size changes the planning requirement. A smaller effect is harder to distinguish from random variation, so it requires more participants per arm to reach the same target power. The listed figures depend on the assumed effect and the planning approximation; they should not be read as guarantees that a study of that size will produce a particular result.

This is not observed power. It is a planning illustration computed from the observed effect, not a measure of the trial’s achieved ability to detect an effect. The trial’s own original sample-size assumptions and justification are not reported in the registry record. The planning figures should therefore be used to understand the relationship between assumed effect size and sample requirements, not to infer that the trial was designed around these calculations.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = 186.600; pooled standard deviation σ = 252.928.
2. n = 2 × (1.96 + 0.842)² × 252.928² / 186.600² = 28.8, rounded up to 29 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- 186.6                      # observed difference in means
sigma <- 252.928204569334                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(3625986)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 29 (80% power), 39 (90% power)
 75% of observed effect: n per arm = 52 (80% power), 69 (90% power)
 50% of observed effect: n per arm = 116 (80% power), 155 (90% power)
power.t.test: 29.8 
Simulated power at n = 29 per arm: 0.793 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Analyses Reported by the Sponsor

The registry reports a sponsor analysis using the Wilcoxon (Mann-Whitney) method, with the parameter described as a mean difference in final values. Its estimate is -186.68. The recalculation from the posted means gives a difference of 186.6 ng/ml when expressed as the non-nicotine group minus the nicotine-containing group. The opposite signs reflect the reversed direction of subtraction, not a disagreement about which group had the higher posted mean.

The sponsor’s analysis is rank-based, while the recalculation is an unadjusted comparison of arithmetic means from summary statistics. Those methods are related but not identical, and they can produce different estimates or p-values. The registry also posts a p-value for the sponsor analysis, but its exact value is not repeated because it exceeds the permitted decimal precision. The registry record does not flag a discrepancy, so there is no stated conflict requiring one result to supersede the other. The sponsor’s analysis remains the registry’s reported analysis; the recalculation is best understood as a transparent, simple comparison of the posted group summaries, not a replacement for the original test.

15. Multiplicity, Missing Data and Analysis Populations

The registry identifies urinary NNAL concentration as the primary outcome and posts results for multiple secondary outcomes. Testing several outcomes increases the chance that at least one result may appear statistically notable by chance. Whether the investigators controlled this family-wise or other multiplicity risk is not reported in the registry record. The registry also does not report a testing hierarchy, a prespecified distinction between confirmatory and exploratory secondary outcomes, or any adjustment of the posted secondary p-values. The secondary findings should therefore be interpreted cautiously and in their outcome-specific context.

For the primary endpoint, the registry states that four values were missing because samples could not be analyzed. The posted endpoint analysis counts are 33 in the nicotine-containing group and 32 in the non-nicotine group. The registry does not state how the missing values were distributed across groups, whether they were excluded from the analysis, or whether sensitivity analyses assessed the impact of missingness. It also does not report missing-data methods for the secondary outcomes.

The registry does not specify whether the primary analysis followed an intention-to-treat principle, whether participants were analyzed according to randomized assignment, or how protocol deviations were treated. The analysis population and handling of adherence are not reported in the registry record. These details matter because missing outcomes, nonadherence, and exclusions can affect how closely an analysis estimates the effect of assignment to a treatment group.

The randomized design is important for the intended comparison, but the registry entry alone does not provide enough detail to assess the complete analysis plan or all departures from it. The record does not report the trial’s original sample-size justification, interim analyses, or a prespecified statistical analysis plan. Absence of those details in the registry record should not be taken as evidence that a procedure was or was not used.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 3% reporting threshold.

Most frequent serious adverse event terms

Most frequent non-serious adverse event terms

Serious events with the highest arm-level rate: stroke (1.9%).

The safety figures list 52 participants at risk in each group. The registry reports one serious adverse event in the nicotine-containing group and none in the non-nicotine group, with zero deaths in both groups. Stroke is listed as the serious-event term at 1.9%. The registry also reports 11 participants with other adverse events in the nicotine-containing group and 8 in the non-nicotine group.

These are descriptive counts from the registry record. They do not establish that an event was caused by the assigned electronic cigarette, and the posted figures do not provide event timing, severity details beyond the serious-event classification, or a comparative statistical analysis. The registry’s definitions and analysis population for the safety figures are not reported in the registry record. The safety counts should therefore be read as a record of reported events, not as a causal estimate or a complete assessment of risk.

17. What the Result Does, and Does Not, Mean

The primary result describes a difference in mean urinary NNAL concentration between the two randomized groups at the 6-week post-treatment visit. The posted mean was lower in the nicotine-containing group than in the non-nicotine group, and the unadjusted recalculated interval excludes zero. This is evidence about one biomarker measurement at one follow-up visit, interpreted under the assumptions of the summary-statistic comparison.

It does not show how much NNAL changed within either group from baseline, because baseline values are not reported. It also does not establish a change in disease risk, a clinical benefit, or a long-term health outcome. NNAL is described in the registry as a carcinogen exposure biomarker, and a difference in a biomarker should not be treated as equivalent to a difference in clinical events.

The comparison also does not by itself establish what would happen outside the study conditions. The registry describes encouragement to switch completely to the assigned electronic cigarette for 6 weeks, but does not report adherence or the extent to which participants switched. The record does not provide details needed to assess how the results might vary with different populations, products, nicotine concentrations, or follow-up periods.

Secondary measures add context, but they are diverse and multiple. Their unadjusted results do not resolve the limitations of the primary comparison, and they should not be combined into a single claim about overall health. The findings are most appropriately read as trial-specific evidence about posted measurements, with uncertainty and missing-data limitations, rather than as a general conclusion about treatment effects.

18. Why This Trial Matters Statistically

This trial illustrates several statistical issues common in clinical research. First, the outcome is a biomarker rather than a direct clinical event. That distinction matters because statistical evidence for a difference in a measured exposure marker does not automatically establish a difference in health outcomes. The endpoint’s meaning depends on what it measures and how closely that measure relates to the clinical questions of interest.

Second, the result highlights the value of distinguishing the sponsor’s reported method from a recalculation using summary statistics. The sponsor used a rank-based Wilcoxon (Mann-Whitney) analysis, while the recalculated contrast is a difference between posted means. Both describe the same groups, but they are not identical analyses. Clear labeling of the estimand, contrast direction, and method helps readers understand why estimates can differ in sign or interpretation.

Third, precision and variability matter alongside a p-value. The primary standard deviations indicate substantial variation in observed NNAL concentrations, and the confidence interval describes a range of plausible differences under the recalculation. The four samples that could not be analyzed also raise a missing-data question that cannot be fully assessed from the registry entry.

Finally, the trial has multiple secondary outcomes and the registry does not report a multiplicity procedure. Separating a primary result from exploratory secondary findings helps prevent overinterpretation of isolated results. The record also makes clear that planning illustrations based on an observed effect are not the same as the original study design or its achieved power.

19. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record does not provide baseline NNAL values, so the posted primary comparison cannot quantify change from smoking baseline. The primary result is limited to a between-group comparison at the 6-week visit. No later follow-up results are included in the described endpoint.

Four primary-outcome values were missing because samples could not be analyzed. The record does not explain their distribution across groups, describe sensitivity analyses, or report how missing outcomes were handled. Analysis-population definitions, adherence, and protocol deviations are also not reported. These omissions limit assessment of whether the posted comparison reflects assignment, actual use, or a selected subset with available samples.

The posted group means and standard deviations do not show the full outcome distributions. The registry’s own analysis is rank-based, while the recalculation uses means; these analyses have different statistical interpretations. The record does not include the full statistical analysis plan, multiplicity procedures, interim analyses, or the original sample-size justification. The trial-specific calculations therefore cannot be fully audited from the registry entry alone.

The trial enrolled current smokers with tobacco dependence and compared two formulations of a standardized research electronic cigarette over 6 weeks. The registry does not provide enough information to assess how results might generalize to other populations, products, use patterns, or longer periods. Safety figures are descriptive, and additional detail about event definitions and the safety analysis population is not reported in the registry record.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

• ClinicalTrials.gov: Does Switching to Nicotine Containing Electronic Cigarettes Reduce Health Risk Markers — NCT03625986.
• Registry record last updated 2025-03-27; status last verified 2025-03.
• Results first posted 2025-03-27.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

23. Record Summary

The registry describes a completed randomized, quadruple-masked, parallel-group study of a nicotine-containing versus non-nicotine standardized research electronic cigarette. The primary outcome was creatinine-corrected urinary NNAL concentration at the 6-week post-treatment visit. The posted mean was 151.6 ng/ml in the nicotine-containing group and 338.2 ng/ml in the non-nicotine group. The unadjusted recalculated mean difference, defined as non-nicotine minus nicotine-containing, was 186.6 ng/ml, with a 95% confidence interval from 63.34 to 309.9.

The sponsor-reported analysis used a Wilcoxon (Mann-Whitney) method. Four primary values were missing because samples could not be analyzed. Secondary outcomes and descriptive safety figures are also posted, but the registry record does not report multiplicity adjustment, detailed missing-data methods, the analysis population definition, or the original sample-size justification. The results describe measurements at the reported follow-up and do not establish baseline change or clinical health effects.