This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for Low Nicotine Cigarettes Plus Electronic Cigarettes (NCT04058717), a randomized na study. The primary outcome is Urinary NNAL (4-(Methylnitrosamino)-1-(3-pyridyl)-1-butanol).
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT04058717 |
| Phase | NA |
| Condition | Tobacco Dependence |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | Double |
| Enrollment | 88 |
| Arms in the primary outcome | 4 |
| Lead sponsor | Milton S. Hershey Medical Center |
| Results posted | Yes |
2. Clinical Question
Population
Smokers with mental health conditions. The registry record identifies tobacco dependence as the condition. Further participant characteristics are not reported in the registry record.
Intervention
Very low nicotine content (VLNC) research cigarettes, containing 0.2 mg nicotine per cigarette, paired with either a high-nicotine e-cigarette or a zero-nicotine e-cigarette.
Comparator
Normal nicotine content (NNC) research cigarettes, containing 11.6 mg nicotine per cigarette, paired with either a high-nicotine e-cigarette or a zero-nicotine e-cigarette. The four combinations allow comparisons across cigarette and e-cigarette nicotine content.
Primary question
At Week 16, how did urinary NNAL, a measure of tobacco-specific nitrosamine, differ among the four combinations? The posted pairwise re-calculations compare each of three combinations with NNC cigarettes plus a high-nicotine e-cigarette. They estimate mean differences and confidence intervals, while the registry record also reports an overall Kruskal-Wallis analysis across all four groups.
3. Trial Design
The registry record describes a randomized, double-masked, parallel-group trial of four combinations of cigarette and e-cigarette nicotine content. The study is listed as completed, with enrollment of 88. It was sponsored by Milton S. Hershey Medical Center and lists tobacco dependence as the condition. The registry record gives a start date of 2021-06-01 and a primary completion date of 2024-07-15.
Participants were assigned to one of four combinations: NNC cigarettes plus a high-nicotine e-cigarette; NNC cigarettes plus a zero-nicotine e-cigarette; VLNC cigarettes plus a high-nicotine e-cigarette; or VLNC cigarettes plus a zero-nicotine e-cigarette. NNC cigarettes contained 11.6 mg nicotine per cigarette, and VLNC cigarettes contained 0.2 mg nicotine per cigarette. The registry record does not specify the nicotine concentration in the high-nicotine e-cigarette liquid.
Double masking is recorded, but who was masked and how masking was maintained are not reported in the registry record. The original allocation procedure, protocol details, and trial’s own sample-size justification are also not reported in the registry record. The primary endpoint was assessed at Week 16. The posted primary endpoint summaries include 17 participants in the NNC/high-nicotine group, 13 in the NNC/zero-nicotine group, 12 in the VLNC/high-nicotine group, and 13 in the VLNC/zero-nicotine group. These are endpoint-specific analysis counts rather than the overall enrollment.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: 88 participants consented and enrolled in the pre-randomization phase of the study. Eight participants were lost to follow-up or withdrawn from the study before being randomized to a study intervention. Therefore, 80 participants were randomized to a study intervention.
Randomization Period
| Stage | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette |
|---|---|---|---|---|
| STARTED | 20 | 20 | 20 | 20 |
| COMPLETED | 17 | 13 | 13 | 13 |
| NOT COMPLETED | 3 | 7 | 7 | 7 |
Reasons for not completing this period:
| Reason | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette |
|---|---|---|---|---|
| Lost to Follow-up | 2 | 3 | 4 | 4 |
| Withdrawal by Subject | 1 | 3 | 3 | 3 |
| PI decision | 0 | 1 | 0 | 0 |
Week 20 Follow up
| Stage | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette |
|---|---|---|---|---|
| STARTED | 17 | 13 | 13 | 13 |
| COMPLETED | 15 | 13 | 13 | 11 |
| NOT COMPLETED | 2 | 0 | 0 | 2 |
Analysis population for the primary outcome, as defined in the registry: The Overall Number of Participants Analyzed includes participants with non-missing values at Week 16. Therefore, in one of the groups, 20 people were randomized and 13 people completed the randomization phase, but only 12 of the samples could be analyzed.
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
| Characteristic | Unit | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette | Total |
|---|---|---|---|---|---|---|
| Participants analysed | 20 | 20 | 20 | 20 | 80 | |
| Age, Continuous | years | 43.9 (SD 9.2) | 51.2 (SD 10.4) | 44.6 (SD 9) | 47.8 (SD 12.1) | 46.9 (SD 10.5) |
| Sex: Female, Male: Female | Participants | 18 | 12 | 18 | 12 | 60 |
| Sex: Female, Male: Male | Participants | 2 | 8 | 2 | 8 | 20 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 0 | 1 | 0 | 0 | 1 |
| Race (NIH/OMB): Asian | Participants | 0 | 0 | 0 | 0 | 0 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 | 0 | 0 | 0 | 0 |
| Race (NIH/OMB): Black or African American | Participants | 1 | 2 | 2 | 2 | 7 |
| Race (NIH/OMB): White | Participants | 18 | 17 | 18 | 18 | 71 |
| Race (NIH/OMB): More than one race | Participants | 1 | 0 | 0 | 0 | 1 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 0 | 0 | 0 | 0 | 0 |
| Region of Enrollment: United States | participants | 20 | 20 | 20 | 20 | 80 |
| Cigarettes per day | cigarettes per day | 22 (SD 11.7) | 18.8 (SD 9.9) | 22.4 (SD 8.6) | 17.6 (SD 9.8) | 20.2 (SD 10.1) |
6. Primary Endpoint
The registry lists the primary outcome as Urinary NNAL (4-(Methylnitrosamino)-1-(3-pyridyl)-1-butanol), measured over Week 16.
| Element | Registry entry |
|---|---|
| Result type | Mean |
| Measure of spread | Standard Deviation |
| Unit | pg/mg creatinine |
| Endpoint classification | Continuous Endpoint |
The primary endpoint was urinary NNAL, reported as a mean with standard deviation in pg/mg creatinine at Week 16. The registry record describes NNAL as a measure of tobacco-specific nitrosamine. It therefore provides a laboratory measure related to exposure to this tobacco-specific substance, rather than a direct measure of cigarette consumption or a clinical health outcome.
The endpoint was summarized as a continuous measure: each group’s reported value is its mean, and the accompanying standard deviation describes variation among participants in that group. The registry record does not explain why Week 16 was selected, specify a clinically important difference, or describe whether any transformation of NNAL was planned. Those details are not reported in the registry record.
For the posted primary endpoint, the observed group means were 505.6 for NNC cigarettes plus a high-nicotine e-cigarette, 459.1 for NNC cigarettes plus a zero-nicotine e-cigarette, 373.3 for VLNC cigarettes plus a high-nicotine e-cigarette, and 180.5 for VLNC cigarettes plus a zero-nicotine e-cigarette. The group standard deviations were 432.7, 312.8, 553.1, and 218.2, respectively. These summaries describe the participants whose Week 16 samples could be analyzed; the registry record notes that not all randomized participants in one group had an analyzable sample.
7. Results Posted in the Registry
- NNC Cigarettes + Zero Nicotine Containing E-cigarette: mean 459.10 versus 505.60 on control; difference -46.50 (95% CI -313.37 to 220.37), standardized effect d = -0.12; z-test p = 0.733.
- VLNC Cigarettes + High Nicotine Containing E-cigarette: mean 373.30 versus 505.60 on control; difference -132.30 (95% CI -506.79 to 242.19), standardized effect d = -0.27; z-test p = 0.489.
- VLNC Cigarettes + Zero Nicotine Containing E-cigarette: mean 180.50 versus 505.60 on control; difference -325.10 (95% CI -562.54 to -87.66), standardized effect d = -0.91; z-test p = 0.007.
Mean difference: NNC Cigarettes + Zero Nicotine Containing E-cigarette
95% CI: -313.37 to 220.37
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Exhaled Carbon Monoxide | Continuous Endpoint | Mean difference | 1.60 (-7.79 to 10.99) | 854 per arm |
| Kessler-6 Score | Continuous Endpoint | Mean difference | -1.20 (-5.14 to 2.74) | 339 per arm |
| Penn State Cigarette Dependence Index | Continuous Endpoint | Mean difference | -0.40 (-2.53 to 1.73) | 859 per arm |
Exhaled Carbon Monoxide
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- NNC Cigarettes + Zero Nicotine Containing E-cigarette: mean 25.20 versus 23.60 on control; difference 1.60 (95% CI -7.79 to 10.99), standardized effect d = 0.14; z-test p = 0.738.
- VLNC Cigarettes + High Nicotine Containing E-cigarette: mean 21.90 versus 23.60 on control; difference -1.70 (95% CI -16.26 to 12.86), standardized effect d = -0.10; z-test p = 0.819.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NNC Cigarettes + Zero Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between VLNC Cigarettes + High Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
Kessler-6 Score
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- NNC Cigarettes + Zero Nicotine Containing E-cigarette: mean 5.70 versus 6.90 on control; difference -1.20 (95% CI -5.14 to 2.74), standardized effect d = -0.22; z-test p = 0.551.
- VLNC Cigarettes + High Nicotine Containing E-cigarette: mean 5.40 versus 6.90 on control; difference -1.50 (95% CI -5.60 to 2.60), standardized effect d = -0.25; z-test p = 0.473.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NNC Cigarettes + Zero Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between VLNC Cigarettes + High Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
Penn State Cigarette Dependence Index
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- NNC Cigarettes + Zero Nicotine Containing E-cigarette: mean 11.50 versus 11.90 on control; difference -0.40 (95% CI -2.53 to 1.73), standardized effect d = -0.14; z-test p = 0.712.
- VLNC Cigarettes + High Nicotine Containing E-cigarette: mean 11.80 versus 11.90 on control; difference -0.10 (95% CI -2.77 to 2.57), standardized effect d = -0.03; z-test p = 0.941.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between NNC Cigarettes + Zero Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between VLNC Cigarettes + High Nicotine Containing E-cigarette and the control. The interval gives the range of effects that remain compatible with the data.
The registry record lists five secondary outcomes: exhaled carbon monoxide, Kessler-6 score, Penn State Cigarette Dependence Index, Penn State Electronic Cigarette Dependence Index, and cigarette abstinence. Numerical Week 16 summaries and analyses are posted for exhaled carbon monoxide, Kessler-6 score, and the Penn State Cigarette Dependence Index. Numerical results for the Penn State Electronic Cigarette Dependence Index and cigarette abstinence are not reported in the registry record.
Across the three outcomes with numerical summaries, the overall pattern is not uniform. The group means for the psychological distress and cigarette dependence measures are relatively similar in the displayed summaries, while exhaled carbon monoxide shows more variation across combinations. The registry record’s omnibus tests also differ across these outcomes. This does not establish a consistent secondary-outcome pattern that independently confirms or overturns the primary NNAL finding.
Secondary outcomes can provide context about related measures, but they are not interchangeable with urinary NNAL. They capture different constructs and may have different measurement properties and missing-data patterns. The registry record does not report a hierarchy of secondary hypotheses or a multiplicity adjustment. When several outcomes and comparisons are considered without such adjustment, individual unadjusted p-values carry less weight because chance findings become more plausible across a collection of tests. The secondary results are therefore best read as exploratory context, not as a single confirmatory conclusion.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Exhaled Carbon Monoxide
- Kessler-6 Score
- Penn State Cigarette Dependence Index
- Penn State Electronic Cigarette Dependence Index
- Cigarette Abstinence
9. Statistical Interpretation
For urinary NNAL at Week 16, the posted mean was 505.6 pg/mg creatinine in the NNC-cigarette/high-nicotine e-cigarette group. The other group means were 459.1 for NNC cigarettes plus a zero-nicotine e-cigarette, 373.3 for VLNC cigarettes plus a high-nicotine e-cigarette, and 180.5 for VLNC cigarettes plus a zero-nicotine e-cigarette. These are descriptive group summaries, not by themselves estimates of a treatment effect.
The unadjusted re-calculated comparison of NNC cigarettes plus a zero-nicotine e-cigarette with the reference group estimated a mean difference of -46.5, with a 95% confidence interval from -313.4 to 220.4 and p = 0.733. The comparison for VLNC cigarettes plus a high-nicotine e-cigarette estimated -132.3, with a 95% confidence interval from -506.8 to 242.2 and p = 0.489. Both intervals include zero and span differences in either direction; the estimates are imprecise.
For VLNC cigarettes plus a zero-nicotine e-cigarette, the estimated mean difference versus the reference group was -325.1, with a 95% confidence interval from -562.5 to -87.66 and p = 0.007. This pairwise interval does not include zero. However, it is an unadjusted re-calculation, and the registry record’s reported analysis is an overall Kruskal-Wallis test across all four groups, with H-statistic 10.23 and p = 0.017. These analyses answer different questions: the pairwise estimate concerns one contrast, while the Kruskal-Wallis result assesses whether the distributions differ somewhere across the four groups.
The posted data therefore show a lower observed mean NNAL in the VLNC/zero-nicotine combination than in the reference group, alongside substantial uncertainty in the other pairwise comparisons. The discrepancy flag applies: the sponsor’s own analysis takes precedence over re-calculated comparisons. The registry record does not provide enough information to establish the reason for the discrepancy or to determine whether the sponsor’s analysis used a different model, adjustment, or analysis population.
The endpoint results are based on relatively small analyzed groups, and the standard deviations are large compared with several estimated differences. The estimates should be read as evidence about the recorded Week 16 measurements in these trial groups, not as a definitive account of all possible outcomes or populations. The registry record does not report a prespecified clinically important difference, so statistical separation alone does not establish the practical importance of the observed difference.
10. Statistical Methods Explained
The primary endpoint is continuous, so the posted group summaries use means and standard deviations. A mean gives the arithmetic average in a group. A standard deviation describes how spread out individual measurements are around that average. A large standard deviation relative to a mean difference signals substantial variation among participants and can contribute to imprecise comparisons.
What does the overall test assess?
The reported primary analysis is a Kruskal-Wallis test across the four groups. In general, this is a rank-based omnibus test: it evaluates whether the group distributions are consistent with coming from the same distribution, without assuming that the outcome follows a normal distribution. An omnibus result does not identify which particular groups differ. The registry record reports an H-statistic of 10.23 and p = 0.017. It does not report confidence limits for this statistic or describe a follow-up comparison procedure.
What do the pairwise estimates represent?
The re-calculated comparisons estimate unadjusted mean differences between the reference group, NNC cigarettes plus a high-nicotine e-cigarette, and each of the other three groups. A negative estimate means the compared group’s observed mean was lower than the reference group’s mean. For example, the estimate of -325.1 for VLNC cigarettes plus a zero-nicotine e-cigarette is the posted difference in means for that group relative to the reference group. These re-calculations are simple comparisons from the posted summaries; they are not a substitute for the sponsor’s analysis.
A confidence interval gives a range of values compatible with the data under the method used to calculate it. An interval that includes zero is compatible with no mean difference, as well as with differences in either direction. An interval that excludes zero indicates that zero is not within that particular interval. The registry record does not specify the method used for the re-calculated intervals, and the posted sponsor analysis does not provide pairwise confidence intervals.
Why might rank-based and mean comparisons differ?
The Kruskal-Wallis analysis uses ranks, whereas the re-calculated comparisons concern arithmetic means. They are not identical tests of the same quantity. In general, skewed distributions, outlying observations, or differences in spread can make rank-based and mean-based analyses behave differently. The primary group standard deviations vary, and the registry record does not report distributional diagnostics or individual-level data. It is therefore not possible to determine from the posted summaries whether distribution shape explains the discrepancy.
The re-calculated p-values are unadjusted. In general, when several comparisons are tested, the chance of at least one small p-value can increase unless a multiplicity strategy is used. No multiplicity adjustment for these comparisons is reported in the registry record. The reported overall test and the three unadjusted comparisons should consequently be kept distinct rather than treated as interchangeable confirmations.
What is not specified about the analysis?
The registry record does not report the original statistical analysis plan, a model specification, covariate adjustments, assumptions checks, interim analyses, or the planned handling of missing values beyond the primary endpoint’s non-missing sample description. It also does not identify the analysis population for each result. Those omissions limit assessment of whether the posted summaries follow a prespecified analysis plan or whether other planned analyses were conducted.
The same general principles apply to the continuous secondary outcomes: means and standard deviations describe the group values, while a confidence interval describes precision for a contrast. The reported secondary omnibus tests are Kruskal-Wallis analyses, with their own reported statistics and p-values. These findings do not remove the uncertainty caused by small endpoint-specific group counts, nor do they establish that every possible comparison or outcome was analyzed in the same way.
11. Confidence Intervals and Precision
The primary pairwise confidence intervals are broad. For NNC cigarettes plus a zero-nicotine e-cigarette versus the reference group, the 95% confidence interval for the mean difference extends from -313.4 to 220.4. For VLNC cigarettes plus a high-nicotine e-cigarette, it extends from -506.8 to 242.2. Each includes zero and allows for both a lower and a higher mean relative to the reference group. The corresponding estimates therefore do not pinpoint the direction or size of a difference precisely.
The interval for VLNC cigarettes plus a zero-nicotine e-cigarette versus the reference group runs from -562.5 to -87.66. It excludes zero under the re-calculated comparison, while still spanning a range of possible mean differences. Its interpretation remains subject to the unadjusted nature of the comparison and the discrepancy flag; the sponsor’s own analysis takes precedence.
In general, a confidence interval conveys the precision of an estimate: narrower intervals indicate greater precision, while wider intervals indicate less. Precision depends on factors including sample size and outcome variability. The primary analysis counts range from 12 to 17 participants per group, and the standard deviations range from 218.2 to 553.1. These features provide context for the broad intervals, without establishing a cause for the discrepancy between analyses.
The interval estimates concern mean differences in the analyzed Week 16 samples. The registry record does not report a clinically important threshold for NNAL, so the intervals cannot be classified as excluding or including a threshold for practical importance. Confidence intervals also do not describe the probability that a particular fixed effect is true; they indicate the range of estimates compatible with the data under the calculation method.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 1,082 | 1,448 |
| 75% of the observed effect | 1,923 | 2,574 |
| 50% of the observed effect | 4,325 | 5,790 |
The sample-size figures are planning illustrations expressed as participants per arm for continuous outcomes. They are not total-event counts. The calculations use a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. They use the observed effect as a reference and also show planning for 75% of the observed effect and planning for 50% of the observed effect. The trial’s own original design assumptions and sample-size justification are not reported in the registry record.
The 80% power column gives the number of participants per arm that the planning calculation estimates would be needed to detect the specified effect under those assumptions, with 80% power. The 90% power column gives the corresponding per-arm count for 90% power. In general, higher planned power requires more participants when the other assumptions are held constant. Likewise, smaller effects need more participants to detect than larger effects. The displayed curve uses the same planning framework to illustrate how the participant requirement changes with the assumed effect size.
For example, across the primary comparisons, the planning counts vary substantially depending on the contrast and the fraction of the observed effect used. That variation reflects that the calculations are tied to the observed effect and estimated variability, which may be unstable when based on small groups. These planning numbers are not observed power. They do not tell whether the completed trial had a particular chance of detecting the actual effect, and they are not evidence that an effect is present or absent.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.
Worked calculation for this trial
- Observed difference Δ = -46.500; pooled standard deviation σ = 385.903.
- n = 2 × (1.96 + 0.842)2 × 385.9032 / -46.5002 = 1,081.2, rounded up to 1,082 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
delta <- -46.5 # observed difference in means
sigma <- 385.903148471219 # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")
# Simulation: exact two-sample t-test using simulated means and variances
set.seed(4058717)
n <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n)) # difference in sample means
s2 <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1)) # pooled sample variance
tstat <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))
Output from running the code
100% of observed effect: n per arm = 1,082 (80% power), 1,448 (90% power)
75% of observed effect: n per arm = 1,923 (80% power), 2,574 (90% power)
50% of observed effect: n per arm = 4,325 (80% power), 5,790 (90% power)
power.t.test: 1082.1
Simulated power at n = 1,082 per arm: 0.799 (20000 replicates)
The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| Kruskal-Wallis | H-statistic 10.23 | not reported | 0.017 |
The registry record reports a Kruskal-Wallis analysis across all four primary endpoint groups, with H-statistic 10.23 and p = 0.017. This is the sponsor’s reported overall analysis. The re-calculated estimates compare the reference group separately with each of the other three groups using unadjusted mean differences and confidence intervals.
Because the discrepancy flag is true, the sponsor’s own analysis takes precedence. The overall rank-based test and the re-calculated pairwise mean comparisons are not the same analysis: one is an omnibus test across four groups, while the others estimate specific contrasts. The registry record does not explain the discrepancy, describe how the sponsor’s analysis was implemented, or report a sponsor-derived pairwise estimate for each comparison. It is not possible to infer from the posted record whether differences in analysis method, data handling, or analysis population account for the discrepancy.
The re-calculations can help readers understand the scale and precision of contrasts based on the posted group summaries, but should not be treated as a replacement for the reported sponsor analysis. The available record does not report a detailed analysis plan against which either approach can be checked.
15. Multiplicity, Missing Data and Analysis Populations
The registry record reports an overall Kruskal-Wallis analysis for the primary endpoint and also supports posted unadjusted pairwise re-calculations. It does not report whether a multiplicity procedure was planned or applied to comparisons among the four groups. No adjustment is reported for the multiple secondary outcomes or their comparisons. The reported p-values should therefore be interpreted as unadjusted unless the registry record explicitly identifies them otherwise; it does not.
The primary endpoint population is described as participants with non-missing Week 16 values. The record notes that, in one group, 20 people were randomized and 13 completed the randomization phase, but only 12 samples could be analyzed. The group is not identified in that description. The posted primary summaries include analyzed counts of 17, 13, 12, and 13 across the four groups. The record does not report imputation, sensitivity analyses for missing outcomes, or a detailed accounting of missing samples by reason.
Endpoint-specific counts also vary among secondary outcomes. The registry record does not explain why values are missing, how missingness was handled, or whether the same participants contributed to each outcome. It does not report a full analysis-population definition, such as whether an intention-to-treat population was used, or whether participants were analyzed according to assigned group in every analysis. These omissions make it difficult to assess how missing observations or population selection may affect the comparisons.
The original statistical analysis plan, interim analysis procedures, and prespecified handling of multiplicity are not reported in the registry record. Without those details, the distinction between confirmatory and exploratory analyses cannot be fully assessed from the posted record.
16. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| NNC Cigarettes + High Nicotine Containing E-cigarette | 4 | 20 | 20.0% |
| NNC Cigarettes + Zero Nicotine Containing E-cigarette | 4 | 20 | 20.0% |
| VLNC Cigarettes + High Nicotine Containing E-cigarette | 2 | 20 | 10.0% |
| VLNC Cigarettes + Zero Nicotine Containing E-cigarette | 1 | 20 | 5.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| NNC Cigarettes + High Nicotine Containing E-cigarette | 0 of 20 |
| NNC Cigarettes + Zero Nicotine Containing E-cigarette | 0 of 20 |
| VLNC Cigarettes + High Nicotine Containing E-cigarette | 0 of 20 |
| VLNC Cigarettes + Zero Nicotine Containing E-cigarette | 0 of 20 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 3% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| NNC Cigarettes + High Nicotine Containing E-cigarette | 7 of 20 |
| NNC Cigarettes + Zero Nicotine Containing E-cigarette | 11 of 20 |
| VLNC Cigarettes + High Nicotine Containing E-cigarette | 6 of 20 |
| VLNC Cigarettes + Zero Nicotine Containing E-cigarette | 8 of 20 |
Most frequent serious adverse event terms
| Event term | Organ system | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette |
|---|---|---|---|---|---|
| Depression | Psychiatric disorders | 1 of 20 | 0 of 20 | 0 of 20 | 1 of 20 |
| Upper respiratory infection | Infections and infestations | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Pregnancy loss | Pregnancy, puerperium and perinatal conditions | 1 of 20 | 0 of 20 | 0 of 20 | 0 of 20 |
| Lung infection | Infections and infestations | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Clostridium difficile infection | Infections and infestations | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Self-injurious ideation | Psychiatric disorders | 1 of 20 | 0 of 20 | 0 of 20 | 0 of 20 |
| Surgical and Medical Procedures, other back electrodes | Surgical and medical procedures | 0 of 20 | 0 of 20 | 1 of 20 | 0 of 20 |
| Alcoholism | Psychiatric disorders | 0 of 20 | 0 of 20 | 1 of 20 | 0 of 20 |
| Infections and infestations - Other, influenza | Infections and infestations | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Chest pain - cardiac | Cardiac disorders | 0 of 20 | 0 of 20 | 0 of 20 | 1 of 20 |
Most frequent non-serious adverse event terms
| Event term | Organ system | NNC Cigarettes + High Nicotine Containing E-cigarette | NNC Cigarettes + Zero Nicotine Containing E-cigarette | VLNC Cigarettes + High Nicotine Containing E-cigarette | VLNC Cigarettes + Zero Nicotine Containing E-cigarette |
|---|---|---|---|---|---|
| Suicidal ideation | Psychiatric disorders | 1 of 20 | 5 of 20 | 4 of 20 | 2 of 20 |
| Cough | Respiratory, thoracic and mediastinal disorders | 2 of 20 | 3 of 20 | 0 of 20 | 3 of 20 |
| Depression | Psychiatric disorders | 2 of 20 | 2 of 20 | 1 of 20 | 1 of 20 |
| Hypertension | Vascular disorders | 2 of 20 | 1 of 20 | 1 of 20 | 2 of 20 |
Serious events with the highest arm-level rate: upper respiratory infection (5.0%), pregnancy loss (5.0%), lung infection (5.0%), clostridium difficile infection (5.0%), self-injurious ideation (5.0%).
The registry record reports serious adverse events for groups with 20 participants at risk each. The serious-event counts were 4 for NNC cigarettes plus a high-nicotine e-cigarette, 4 for NNC cigarettes plus a zero-nicotine e-cigarette, 2 for VLNC cigarettes plus a high-nicotine e-cigarette, and 1 for VLNC cigarettes plus a zero-nicotine e-cigarette. No deaths were reported in any group.
The record also lists other, non-serious adverse events affecting 7, 11, 6, and 8 participants, respectively, across those same four groups. These counts describe events or affected participants as recorded; they do not establish whether an event was caused by an assigned product. The registry record does not provide event-level timing, duration, severity beyond the serious-event classification, or an assessment of relatedness.
Six serious-event terms are listed, each with a reported frequency of 5.0%: upper respiratory infection, pregnancy loss, lung infection, Clostridium difficile infection, self-injurious ideation, and surgical and medical procedures, other back electrodes. The small groups and limited event details mean these figures are descriptive. They should not be used alone to compare safety between combinations or to infer causation.
17. What the Result Does, and Does Not, Mean
The primary endpoint summaries show differences in observed Week 16 urinary NNAL means across the four combinations. The lowest posted mean was in the VLNC-cigarette/zero-nicotine e-cigarette group, and its unadjusted re-calculated contrast with the NNC-cigarette/high-nicotine e-cigarette group had a confidence interval that excluded zero. The sponsor’s reported overall analysis also indicates a difference among the four group distributions. Because the record flags a discrepancy, the sponsor’s own analysis takes precedence over the re-calculations.
These results do not establish that a particular combination changes clinical health outcomes. NNAL is described in the registry record as a measure of tobacco-specific nitrosamine; it is not itself a clinical outcome. The record does not provide a clinically important threshold or indicate what magnitude of NNAL difference would be meaningful to participants.
The other two re-calculated primary contrasts have broad confidence intervals that include zero. More generally, the small endpoint-specific group sizes and substantial variability limit precision. An interval that includes zero is not proof that the groups are identical, just as an interval that excludes zero does not by itself establish practical importance or prove causation.
The trial concerns smokers with mental health conditions, but further characteristics of the participants are not reported in the registry record. The findings should not be assumed to apply to populations or settings that the record does not describe. In general, randomization can support comparisons of assigned groups, but the available registry summaries do not provide enough detail about analysis populations, missing data, or the analysis plan to resolve all potential sources of uncertainty.
18. Why This Trial Matters Statistically
This trial illustrates why statistical interpretation requires more than comparing group means or reading a single p-value. It reports four combinations, a continuous primary measure, an overall rank-based test, and re-calculated pairwise mean differences. Each addresses a different aspect of the data: group summaries describe observed values, the omnibus test evaluates an overall distributional pattern, and pairwise estimates quantify particular contrasts.
It also demonstrates how precision affects interpretation. The primary endpoint group counts are small, outcome variability is substantial, and two of the three pairwise intervals span both negative and positive differences. A point estimate alone would hide that uncertainty. Reading the estimate together with its interval and the sponsor’s reported analysis gives a more complete account of what the registry record shows.
The discrepancy between the reported overall test and re-calculated comparisons is statistically important, not merely a technical detail. The tests use different approaches and answer different questions; the sponsor’s own analysis takes precedence. The record does not explain the discrepancy, emphasizing the value of a detailed statistical analysis plan and transparent reporting of methods, analysis populations, and missing-data procedures.
Finally, the listed secondary outcomes and adverse-event counts illustrate why multiplicity and descriptive context matter. Multiple outcomes can broaden understanding, but unadjusted findings warrant caution. Together, these issues make the trial useful for learning how design, analysis, precision, and reporting completeness shape interpretation of clinical-trial results.
19. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
Several limitations are specific to the information available for this trial. The primary endpoint summaries are based on 12 to 17 analyzed participants per group, despite overall enrollment of 88. The record notes that a group had 20 participants randomized, 13 completing the randomization phase, and 12 analyzable samples, but does not identify that group or explain the unavailable observations. This leaves uncertainty about the contribution of missing primary endpoint data.
The primary endpoint standard deviations are large, and the pairwise confidence intervals are broad for two comparisons. The sponsor’s reported overall analysis and the re-calculated pairwise results are not identical in approach, and the discrepancy is not explained in the registry record. The original statistical analysis plan, multiplicity strategy, model details, missing-data methods, and interim analyses are not reported.
Some secondary outcomes have posted numerical results, while numerical results for cigarette abstinence and the Penn State Electronic Cigarette Dependence Index are not reported in the registry record. Secondary endpoint counts vary, and the record does not describe the reasons or handling of missing values. The high-nicotine e-cigarette liquid concentration is also not specified. Further participant characteristics and details about who was masked are not reported.
The results concern Week 16 measurements, and the registry record does not describe additional follow-up for the reported primary endpoint. It also does not give a clinically important threshold for NNAL. These limitations constrain assessment of precision, practical importance, and how broadly the findings may apply.
20. Related Tutorials
21. Related Statistical Calculators
22. Sources
- ClinicalTrials.gov: Low Nicotine Cigarettes Plus Electronic Cigarettes — NCT04058717.
- Registry record last updated 2025-09-11; status last verified 2025-08.
- Results first posted 2025-09-11.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
23. Record Summary
The completed randomized, double-masked, parallel-group trial enrolled 88 smokers with mental health conditions and compared four combinations of NNC or VLNC cigarettes with high- or zero-nicotine e-cigarettes. The primary endpoint was mean urinary NNAL at Week 16, reported in pg/mg creatinine.
The posted group means ranged from 180.5 to 505.6. The registry record reports an overall Kruskal-Wallis H-statistic of 10.23 with p = 0.017. Re-calculated pairwise comparisons have varying estimates and precision; the discrepancy flag means the sponsor’s own analysis takes precedence. Several secondary outcomes and adverse-event counts are also recorded, while numerical findings for two listed secondary outcomes are not reported in the registry record.
Small analyzed groups, missing-sample information, unadjusted comparisons, and absent analysis-plan details limit interpretation of the estimates and their generalizability.