Improving Quitline Support Study: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Improving Quitline Support Study (NCT03538938), a randomized phase 4 study. The primary outcome is 7-Day Point-Prevalence Abstinence.

2. Clinical Question

3. Trial Design

The Improving Quitline Support Study was a completed, randomized, single-masked, Phase 4 factorial trial sponsored by the University of Wisconsin, Madison. The registry reports enrollment of 1316. The study began on 2018-06-07, and primary completion was recorded on 2023-08-15. Its stated aim was to assess approaches to improve follow-up support for socioeconomically disadvantaged callers who remained smokers after initial Wisconsin Tobacco Quit Line treatment.

The factorial design varied four support components: the number of quitline counseling calls, nicotine replacement intensity, access to SmokefreeTXT, and financial incentives for treatment engagement. Counseling was offered as 1-Call or 4-Call Quitline Counseling. Medication options were a nicotine patch or a nicotine patch combined with nicotine lozenges. SmokefreeTXT enrollment support and financial incentives were each either offered or not offered. The registry’s arm descriptions list combinations of these components.

The brief study description says the medication-intensity strategy applied among people medically cleared to use those medications. It describes the counseling and texting interventions as follow-up care for people reporting smoking four months after initial treatment. Details beyond the arm descriptions are not reported in the registry record; in particular, the full treatment descriptions are truncated in the record. The extent and implementation of masking are not reported in the registry record.

The primary outcome was a participant count for 7-day point-prevalence abstinence at 6-month follow-up. The analysis population is described as all randomized participants in intent-to-treat analyses. The registry record does not report the trial’s original sample-size justification or its planned statistical analysis approach. No sponsor-reported analysis is present in the registry record; the available comparisons are unadjusted recalculations from the posted arm summaries.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Only the first 6 of 16 groups are shown.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: All randomized participants were included in intent-to-treat analyses.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Only the first 6 of 17 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as 7-Day Point-Prevalence Abstinence, measured over 6-months.

The primary endpoint counts participants classified as abstinent at the 6-month assessment. To meet the definition, a participant had to report no smoking during the past 7 days and provide a saliva sample with cotinine below 4 ng/ml. The measure therefore combines a short-window self-report with a biochemical check. The registry specifies that participants who were lost to follow-up, reported any smoking in the past 7 days, had a cotinine result at or above 4 ng/ml, or did not return a usable saliva sample were classified as not abstinent.

A point-prevalence endpoint captures smoking status over a defined period immediately before assessment; it does not describe continuous abstinence throughout follow-up. A 7-day window can provide a consistent outcome definition across randomized groups, while the saliva test adds an objective check to self-report. These are general reasons such a combined endpoint may be used; they do not establish whether participants remained abstinent at other times.

The posted outcome is a count of participants meeting the definition, rather than a measure of time to quitting or duration of abstinence. The registry does not report additional primary endpoint details, such as a separate handling rule for unusable samples beyond classifying the participant as not abstinent. This endpoint definition makes follow-up and sample usability part of the observed outcome: participants without an evaluable assessment do not count as abstinent under the stated rule.

7. Results Posted in the Registry

• 1-Call, Patch, SmokefreeTXT, No Financial Incentive: 8.3% (7/84) versus 13.3% (11/83) on control; risk difference -4.9 percentage points (95% CI -14.3 to 4.5), risk ratio 0.63 (0.26–1.54); unadjusted two-proportion z-test p = 0.305.
• 1-Call, Patch, No SmokefreeTXT, Financial Incentive: 10.0% (8/80) versus 13.3% (11/83) on control; risk difference -3.3 percentage points (95% CI -13.1 to 6.6), risk ratio 0.75 (0.32–1.78); unadjusted two-proportion z-test p = 0.518.
• 1-Call, Patch, No SmokefreeTXT, No Financial Incentive: 11.8% (10/85) versus 13.3% (11/83) on control; risk difference -1.5 percentage points (95% CI -11.5 to 8.5), risk ratio 0.89 (0.40–1.98); unadjusted two-proportion z-test p = 0.771.
• 1-Call, Patch+Loz, SmokefreeTXT, Financial Incentive: 7.3% (6/82) versus 13.3% (11/83) on control; risk difference -5.9 percentage points (95% CI -15.2 to 3.3), risk ratio 0.55 (0.21–1.42); unadjusted two-proportion z-test p = 0.210.

8. Statistical Interpretation

9. Statistical Methods Explained

The reported endpoint is binary at the participant level: each person either meets the study’s abstinence definition or does not. The arm summaries give the number meeting the definition and the number in the corresponding arm. For a binary outcome, a risk is the proportion of participants with the event. A risk difference subtracts the reference-arm risk from the comparison-arm risk. This general interpretation explains the sign of the estimates; it does not turn the unadjusted contrasts into adjusted or factorial effects.

What does the risk difference represent?

The reference arm is 1-Call Quitline Counseling, patch, SmokefreeTXT, and financial incentives. The comparison estimates use the observed counts for that reference and for each other listed combination. A negative estimate indicates a lower observed abstinence proportion in the comparison combination than in the reference; a positive estimate indicates a higher observed proportion. For example, the risk difference for 1-Call, patch, SmokefreeTXT, and no financial incentives is -0.049. Its confidence interval runs from -0.143 to 0.045. The estimate is a simple unadjusted contrast, not an estimate from a reported regression model.

How should the confidence interval be understood?

A confidence interval describes the range of values reasonably compatible with the data and the calculation method, under repeated-sampling assumptions. In general, a narrower interval indicates greater precision, while a wider interval indicates greater uncertainty. If an interval for a risk difference includes zero, the results are compatible with no difference as well as with some differences in either direction. All listed intervals span zero. Their breadth indicates that the observed arm counts do not tightly pin down the size of a contrast.

The interval is not a guarantee that the true value lies within its endpoints, and it does not account for every source of uncertainty. In this trial, the posted contrasts are unadjusted and are drawn from a factorial study with multiple treatment combinations. The registry does not report a multiplicity adjustment or a model incorporating the factorial structure. The intervals should therefore be read as uncertainty descriptions for the individual recalculated comparisons, not as definitive evidence about the full set of intervention components.

What do the p-values add?

A p-value measures how unusual data at least as inconsistent with a specified null hypothesis would be, under that hypothesis and the assumptions of the statistical procedure. It is not the probability that the null hypothesis is true, and it is not a measure of clinical importance. The comparison p-values are labeled unadjusted. For instance, the 4-Call, patch, SmokefreeTXT, financial-incentive comparison has p = 0.068, with a risk-difference interval from -0.170 to 0.005. That result does not establish a difference; the interval still includes zero. No thresholding of these separate unadjusted comparisons can substitute for an analysis plan that addresses the factorial design and multiplicity.

Why does the factorial design matter?

Factorial studies vary multiple intervention components across treatment combinations. In general, this can allow investigators to assess components efficiently, but interpreting effects depends on how the analysis accounts for assignment structure and possible interactions. A component’s effect may differ depending on which other components accompany it. The registry record describes a factorial design, but does not report the sponsor’s planned factorial model, main-effect estimates, interaction estimates, or adjustment strategy. The posted comparisons against one combination change one or more components at a time; they do not by themselves isolate each component’s effect.

The comparison estimates are based on the posted summary counts and are unadjusted recalculations. Their purpose is descriptive and exploratory, especially in the absence of sponsor-reported analyses. They do not incorporate covariates or give evidence about whether treatment combinations differ beyond the uncertainty reflected in the individual intervals. If a reported analysis were available and disagreed with a simple recalculation, the sponsor’s own analysis would take precedence; no discrepancy is flagged for these results.

How is the analysis population defined?

The registry states that all randomized participants were included in intent-to-treat analyses. For the primary endpoint, however, the classification rule explicitly counts loss to follow-up, smoking in the past 7 days, an elevated cotinine result, or a missing usable saliva sample as not abstinent. This rule preserves a common assigned-group outcome definition, but it means that the abstinence count combines participants who were assessed and failed the criteria with participants who could not be classified as abstinent because of follow-up or sample issues. Separate missingness counts and sensitivity analyses are not reported in the registry record.

General statistical interpretation should keep the endpoint definition, estimand, and analysis method distinct. The endpoint definition says what qualifies as abstinent. The analysis population says whom the registry intended to include. The posted unadjusted contrasts describe differences in observed classified counts. A model-based analysis could answer a different or more adjusted question, but the registry record does not report one. Likewise, no claim about observed power follows from these recalculations or their planning illustrations.

10. Confidence Intervals and Precision

The confidence intervals for the risk differences are broad across the posted comparisons, and each spans zero. This means the data are compatible with no difference between a comparison combination and the reference, while also allowing differences in either direction. The intervals communicate uncertainty in the estimated contrasts; they do not demonstrate that the combinations are equivalent. Establishing equivalence would require a prespecified margin and an analysis designed for that question, neither of which is reported in the registry record.

For illustration, the comparison of 1-Call, patch, SmokefreeTXT, and no financial incentives with the reference has an estimate of -0.049 and an interval from -0.143 to 0.045. The estimate for 4-Call, patch, SmokefreeTXT, and financial incentives is -0.083, with an interval from -0.170 to 0.005. These intervals encompass appreciable uncertainty around the point estimates. Their endpoints should be interpreted on the risk-difference scale, not as counts of participants.

Precision is constrained by the size of the arm-level event counts and by the spread of results across arms. A confidence interval is a general statistical summary of sampling uncertainty under its method; it does not resolve possible bias from missing data, measurement, or an unreported analysis strategy. The registry’s endpoint rule classifies lost-to-follow-up participants and participants without a usable saliva sample as not abstinent, so the stated estimate depends in part on that rule.

The comparisons are also numerous, and they use a single reference combination rather than reported factorial main effects. The registry record does not report multiplicity handling. Consequently, individual interval coverage and p-values should not be treated as a complete answer to the study’s broader question about four support strategies. The posted evidence offers limited precision for individual combination contrasts and does not report a sponsor analysis that estimates the component effects.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total events. They are illustrations based on the observed effect in each comparison, rather than the trial’s own original design assumptions. The calculation assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is the stated share of the observed effect. The registry record does not report the trial’s original sample-size justification.

The 80% power column gives the illustrative number of participants per arm that would be needed, under those assumptions, for an 80% chance of detecting the specified effect if that effect were the true one. The 90% power column gives the corresponding per-arm number for a 90% chance. These are prospective planning concepts: they describe design sensitivity under assumed conditions, not the probability that a result from this completed trial is true.

The rows vary the assumed effect size. Planning for 75% of the observed effect or 50% of the observed effect asks about detecting a smaller difference than planning for 100% of the observed effect. Smaller effects generally require more participants to detect with the same target power. The figures can therefore rise substantially when the target effect is reduced. A curve based on these assumptions should be read as another view of that planning relationship, not as a reconstruction of the trial’s actual design process.

This is not observed power. It does not say how much power the completed trial had after results were known, and it does not establish whether any intervention difference exists. The illustrations are sensitive to the observed effect used as their starting point; an observed effect can be imprecise, particularly when event counts are limited. The registry’s trial-specific power assumptions, any allowance for attrition, and any sample-size adjustment for factorial comparisons are not reported in the registry record.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.133; treatment proportion p₁ = 0.083; average p̄ = 0.108.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.4388 = 0.8601.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.4374 = 0.3682.
4. n = (0.8601 + 0.3682)² / (-0.0492)² = 623.3, rounded up to 624 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.132530120481928                          # control proportion
p1 <- 0.0833333333333333                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3538938)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 624 (80% power), 834 (90% power)
 75% of observed effect: n per arm = 1,165 (80% power), 1,558 (90% power)
 50% of observed effect: n per arm = 2,743 (80% power), 3,672 (90% power)
power.prop.test: 623.3 
Simulated power at n = 624 per arm: 0.801 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry lists multiple treatment combinations and pairwise comparisons with one reference combination. Repeatedly examining contrasts creates a multiplicity issue: in general, the more comparisons made, the greater the chance that at least one result appears unusual by chance alone. The posted p-values are unadjusted, and the registry record does not report whether the trial’s original analysis plan specified a primary factorial contrast, a hierarchy of hypotheses, or a method to control multiplicity. The unadjusted comparisons should not be read as if they jointly provide a single confirmatory test of all four intervention strategies.

The study is described as factorial, but the registry does not report sponsor estimates of main effects for counseling intensity, medication intensity, SmokefreeTXT, or financial incentives. It also does not report interactions showing whether a component’s association with abstinence varied according to the other components. Pairwise comparisons against the reference combination can describe the listed arm contrasts, but they do not substitute for those component-level analyses.

The registry states that all randomized participants were included in intent-to-treat analyses. For the primary endpoint, loss to follow-up, smoking during the 7-day assessment window, an at-or-above-threshold cotinine result, or failure to return a usable saliva sample results in classification as not abstinent. This rule is explicit, but separate numbers for missing assessments and unusable samples are not reported in the registry record. The record also does not report sensitivity analyses using alternative assumptions about missing outcomes.

The registry record does not report the sponsor’s full statistical analysis plan, covariate adjustment, handling of any protocol deviations, or treatment of interactions. These omissions limit assessment of how the posted recalculations relate to the study’s original inferential objectives. The results can be described as unadjusted arm comparisons under the stated endpoint classification, but they cannot be presented as a complete account of the prespecified analysis strategy.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 3% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: infection prompting hospitalization (3.8%), gastrointestinal disease, pancreatitis, ulcer, or injury (e.g., hernia) (3.8%), cardiovascular diagnoses and events (3.7%), injury or poisoning (2.5%), cancer, biopsy, polyp, or cyst (2.5%).

The serious adverse event summaries give the number at risk and the number affected within each treatment combination. The reported affected counts differ across arms; for example, the 1-Call, patch, SmokefreeTXT, financial-incentive group lists 7 serious adverse events among 83 at risk, while the corresponding no-financial-incentive group lists 5 among 84 at risk. These counts are descriptive. They do not establish that an event was caused by a particular treatment component.

The registry reports zero other adverse events for every listed group. Deaths are reported as 0 in some groups, 1 in others, and 2 in one group. Serious adverse event terms appearing among the more frequent listed terms include infection prompting hospitalization (3.8%), gastrointestinal disease, pancreatitis, ulcer, or injury (e.g., hernia) (3.8%), cardiovascular diagnoses and events (3.7%), injury or poisoning (2.5%), cancer, biopsy, polyp, or cyst (2.5%), and suicidal ideation, bipolar, schizophrenia, anxiety, or alcohol use disorder symptoms (2.5%).

These registry summaries do not provide a causal comparison of safety risks, adjustment for exposure, or information about event timing and relatedness. The serious-event counts should be read as reported events in the listed groups, not as proof that an intervention caused or prevented them. Additional safety analysis details are not reported in the registry record.

15. What the Result Does, and Does Not, Mean

The results describe how many participants met the study’s specified abstinence definition in each posted treatment combination and how the corresponding unadjusted risk differences compare with one reference combination. The observed counts vary, and some comparisons have negative estimates while others have positive estimates. All listed confidence intervals span zero, so the contrasts remain uncertain in direction and size.

These observations do not establish that any one counseling, medication, texting, or incentive strategy caused a change in abstinence. The posted comparisons combine component differences in some cases, and no sponsor-reported factorial estimates separate main effects from possible interactions. The registry record does not report a multiplicity strategy, so the set of unadjusted comparisons should not be interpreted as a collection of definitive tests of the support strategies.

The endpoint represents self-reported absence of smoking during the past 7 days, verified by a saliva cotinine result below 4 ng/ml. It is not a measure of continuous abstinence across the entire follow-up period. Participants lost to follow-up or without a usable saliva sample are classified as not abstinent, so the reported count reflects that rule as well as reported smoking status and cotinine results.

The findings apply most directly to the population described by the study: low-education, uninsured or Medicaid-eligible smokers who continued to smoke four months after initial quitline treatment. They do not directly characterize all smokers or all quitline callers. The posted summaries offer an account of the observed arm counts, but broad intervals, missing sponsor analyses, and incomplete reporting about the analysis plan limit conclusions about which components may account for differences.

Accordingly, the registry results support a cautious descriptive reading rather than a claim that a treatment works or does not work. They show variation in the observed outcomes, with considerable uncertainty around the unadjusted contrasts. More detailed prespecified factorial analyses would be needed to address the separate contribution of each strategy and possible combinations of strategies; those analyses are not reported in the registry record.

16. Why This Trial Matters Statistically

This study is statistically informative because it evaluates multiple service strategies in a factorial randomized design rather than comparing only one package with usual care. The strategies span counseling intensity, nicotine replacement intensity, text-message support, and financial incentives. Designs of this kind can address several practical questions within a common study, but the interpretation depends on analyses that distinguish component effects and account for potential interactions.

The posted evidence illustrates the difference between an observed point estimate and a precise estimate. Arm counts and risk differences vary, yet the confidence intervals for all listed comparisons span zero and are broad. A numerical difference between observed arms is not, by itself, a precise or causal estimate of a component effect. Reading intervals alongside the point estimates helps keep the uncertainty visible.

The study also makes endpoint definition consequential. Abstinence requires both self-report and cotinine below 4 ng/ml, while loss to follow-up and unusable samples are classified as not abstinent. This yields a clear rule for the posted count, but it also means missing outcome information is incorporated into the outcome classification. Separate reporting of missingness and sensitivity analyses would help readers assess how this choice influences interpretation; those details are not reported in the registry record.

Finally, the contrast between planning illustrations and trial results is important. Planning numbers based on observed effects can show how assumed smaller effects require larger samples under stated assumptions, but they are not evidence of power achieved by the completed trial. The original design justification and sponsor-reported factorial analysis are not reported. The record therefore illustrates both the potential of a multi-component trial and the importance of complete reporting for interpreting its estimates.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The posted arm-by-arm estimates are unadjusted recalculations against one reference combination. The registry does not report sponsor estimates of factorial main effects or interactions, even though the study is described as factorial. Because listed comparisons can change more than one component, they do not isolate the independent contribution of counseling, medication, texting, or incentives.

The registry record does not report the original sample-size justification, the planned statistical analysis, any multiplicity strategy, or whether there were interim analyses. It also does not provide separate missing-outcome counts or sensitivity analyses. These omissions prevent a full assessment of how the posted comparisons relate to the original study objectives and how robust endpoint classification may be to missing follow-up.

The primary outcome is a point-prevalence measure at 6-month follow-up, based on the past 7 days and cotinine testing. It does not establish continuous abstinence through the follow-up period. The stated rule classifies lost-to-follow-up participants and those without a usable saliva sample as not abstinent; separate results under alternative missing-data assumptions are not reported in the registry record.

The population was selected from socioeconomically disadvantaged quitline callers who continued smoking four months after initial treatment. Results should not automatically be generalized to smokers outside that described population or to different service settings. The intervention descriptions are partly truncated in the registry record, and the extent of masking is not reported. Safety counts are descriptive and do not include reported causal attribution or exposure-adjusted comparisons. Together, these reporting limits argue for treating the results as uncertain descriptive evidence rather than definitive estimates of component effects.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Improving Quitline Support Study — NCT03538938.
• Registry record last updated 2024-07-12; status last verified 2024-07.
• Results first posted 2024-07-12.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

The registry describes a completed, randomized, single-masked, Phase 4 factorial study of follow-up support for low-education, uninsured or Medicaid-eligible smokers who continued smoking after initial Wisconsin Tobacco Quit Line treatment. Enrollment was 1316. The strategies varied counseling intensity, nicotine replacement, SmokefreeTXT, and financial incentives.

The primary outcome was the count meeting the 7-day point-prevalence abstinence definition at 6-month follow-up. The record states that all randomized participants were included in intent-to-treat analyses and classifies loss to follow-up and unusable saliva samples as not abstinent. Posted comparisons are unadjusted recalculations against the 1-Call, patch, SmokefreeTXT, financial-incentive combination. Their confidence intervals span zero. Sponsor-reported factorial analyses, the original sample-size justification, and the statistical analysis plan are not reported in the registry record.