reSET-O RCT (Randomized Controlled Trial): Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for reSET-O RCT (Randomized Controlled Trial) (NCT04129580), a randomized na study. The primary outcome is Retention in Treatment on MAT for 6 Months After Enrolling in the Study.

2. Clinical Question

3. Trial Design

The registry identifies NCT04129580 as a randomized controlled trial sponsored by Milton S. Hershey Medical Center. Its status is completed. The recorded enrollment is 100, and the study dates run from 2020-09-15 to primary completion on 2023-07-14. The condition listed is opioid-use disorder.

Participants were assigned to treatment as usual plus reSET-O or treatment as usual alone. The recorded allocation is randomized, and masking is listed as none. The intervention is described as an app owned by Pear Therapeutics, Inc.; the registry’s short description says the study evaluates treatment retention after medication-assisted treatment is initiated.

There is an inconsistency in the design descriptors: the record calls the study a randomized controlled trial and lists randomized allocation and two assigned arms, but its model field says single group. The registry record does not explain this inconsistency. The arm-level outcome counts are presented separately for the two groups, so the results can be described as a comparison of those posted groups, while keeping the design-label discrepancy in view.

The registry lists the primary outcome as retention in treatment on medication-assisted treatment for 6 months after study enrollment. It also lists secondary outcomes addressing substance use, depression, anxiety, and PTSD. The registry record does not report the original statistical analysis plan, how allocation was generated or concealed, or additional details of implementation and follow-up. Those details should not be inferred from the randomized label alone.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Retention in Treatment on MAT for 6 Months After Enrolling in the Study, measured over 6 months.

The primary endpoint is titled “Retention in Treatment on MAT for 6 Months After Enrolling in the Study.” It is measured in participants and posted as a count. The description says retention is assessed by the number of appointments attended, with electronic medical record data used to count appointments for study participants. The record does not state a specific appointment threshold for meeting the retention outcome, so the operational rule used to classify participants is not reported in the registry record.

Retention is relevant to this study’s stated question because the study concerns people initiating medication-assisted treatment and whether they remain in treatment during the stated follow-up period. Appointment attendance offers a record-based way to assess engagement with care. It is distinct from a direct measure of substance use or mental health symptoms, which are addressed separately among the listed secondary outcomes.

The posted primary counts are 34 of 48 in the treatment-as-usual-plus-reSET-O arm and 27 of 52 in the treatment-as-usual-only arm. These are the counts associated with the endpoint in the registry results. They should be interpreted according to the registry’s outcome definition and analysis population; details of the classification threshold and handling of incomplete records are not reported in the registry record.

7. Results Posted in the Registry

• TAU Only: 51.9% (27/52) versus 70.8% (34/48) on control; risk difference -18.9 percentage points (95% CI -37.6 to -0.2), risk ratio 0.73 (0.53–1.01); unadjusted two-proportion z-test p = 0.053.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Opioid and Other Substance Abuse as Assessed Through Biological Specimen

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • TAU Only: 63.5% (33/52) versus 66.7% (32/48) on control; risk difference -3.2 percentage points (95% CI -21.9 to 15.5), risk ratio 0.95 (0.71–1.27); unadjusted two-proportion z-test p = 0.737.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between TAU Only and the control. The interval gives the range of effects that remain compatible with the data.

Opioid and Other Substance Abuse as Assessed Through Participant Self-report

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • TAU Only: 63.0% (17/27) versus 67.6% (23/34) on control; risk difference -4.7 percentage points (95% CI -28.7 to 19.4), risk ratio 0.93 (0.64–1.35); unadjusted two-proportion z-test p = 0.702.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between TAU Only and the control. The interval gives the range of effects that remain compatible with the data.

The registry lists secondary outcomes covering substance use assessed through biological specimens and participant self-report, along with symptoms related to depression, anxiety, and PTSD. Numerical results are posted for the biological-specimen and self-report substance-use outcomes. Numerical results for the listed mental-health outcomes are not reported in the registry record.

For the two outcomes with posted comparisons, the estimates are close to zero on the risk-difference scale, and both confidence intervals include zero. Their intervals are broad, so the results do not precisely establish the direction or magnitude of a group difference. The two measurement approaches offer different perspectives: biological specimens and participant reports do not measure substance use in the same way. Neither should be treated as a direct substitute for the primary retention outcome.

These secondary findings do not change the interpretation of the primary endpoint. They address distinct questions and do not provide a direct explanation for the observed retention comparison. Their pattern is compatible with uncertainty about group differences, rather than a precise account of a consistent effect across outcomes. The registry does not report analyses that link the secondary outcomes to retention.

Secondary outcomes often involve multiple comparisons. General statistical principle: when many hypotheses are tested, the chance of at least one apparently unusual result can increase unless the analysis accounts for multiplicity. The registry does not report a multiplicity adjustment or a prespecified hierarchy for interpreting secondary outcomes. Therefore, unadjusted secondary results carry less weight as confirmatory evidence than results from a clearly specified primary analysis. In this record, the secondary outcomes are most appropriately viewed as descriptive and exploratory, while acknowledging that the details of the intended analysis plan are not reported.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Opioid and Other Substance Abuse as Assessed Through Biological Specimen
• Opioid and Other Substance Abuse as Assessed Through Participant Self-report
• Overall Mental Health Symptoms Related to Depression and Anxiety as Assessed by the K-10
• Mental Health Symptoms Specific to Depression as Assessed by the PHQ-9
• Mental Health Symptoms Related to PTSD as Assessed by the PCL-C

9. Statistical Interpretation

10. Statistical Methods Explained

The primary endpoint is binary in the posted comparison: each participant is counted according to whether the registry records them as meeting the retention outcome. A count of participants meeting an outcome is useful to describe how many people in each arm had the recorded event. To compare groups of different sizes, a risk or event proportion can also be reported; the registry provides these proportions and a risk difference.

What does the risk difference represent?

A risk difference subtracts the event proportion in one group from the event proportion in the other. The posted comparison is TAU Only relative to TAU plus reSET-O. Its estimate, -0.189, therefore represents the difference in the recorded outcome proportions in that direction. A negative value means the observed proportion was lower in the arm named as the comparison than in the reference arm. This is a summary of the observed data, not a guarantee that the same difference would occur in another study.

How should the confidence interval be read?

A confidence interval expresses the range of effect estimates reasonably compatible with the data under the method used. It conveys uncertainty about the estimated difference, rather than a range containing a fixed proportion of individual participants. For the primary risk difference, the reported interval runs from -0.376 to -0.002. Because both limits are negative and the upper limit is close to zero, the interval places the estimate near the boundary between a negative difference and no difference, while allowing a larger negative difference. The interval is broad enough that the size of the association is not precisely pinned down.

What does the p-value add?

The unadjusted p-value for the primary comparison is 0.053. In general, a p-value measures how unusual data at least as discrepant as those observed would be under a specified null model. It is not the probability that the null hypothesis is true, nor does it quantify the size or importance of an effect. Here it should be read alongside the estimated risk difference and its confidence interval. Treating a threshold as a dividing line would conceal how close the result is to that boundary and how uncertain the estimated magnitude remains.

The registry also supplies risk-ratio and odds-ratio summaries. These are alternative ways to describe a binary group comparison: a risk ratio compares event proportions, while an odds ratio compares event odds. Such measures can be less intuitive than a risk difference, and they answer the comparison on different scales. Their reported confidence intervals also express uncertainty. They should not be treated as independent confirmations of the primary result because they are calculated from the same underlying group data.

What does “unadjusted” mean?

An unadjusted comparison uses the posted group outcome summaries without incorporating other covariates into the effect estimate. Random assignment is intended to support a fair comparison, but the record does not provide a covariate-adjusted estimate or enough detail to evaluate the analysis population and missing data. No adjusted statistical analysis is reported in the registry record. The primary estimate is therefore the reported summary comparison, rather than an adjusted model result.

For a binary outcome, precision depends in part on the number of participants and the distribution of events across groups. When event counts are limited, estimates can be sensitive to relatively small changes in the observed data. Confidence intervals help show that uncertainty. General statistical principle: estimates based on a small study may be unstable, and a wide interval can remain compatible with materially different effect sizes. This does not mean the observed counts are invalid; it means that they provide limited precision for distinguishing among possible effects.

The registry’s endpoint description refers to counting attended appointments through electronic medical records, but it does not explain the threshold used to classify retention or how incomplete records were handled. A clear binary definition is important because changing the threshold can change which participants are counted as meeting the endpoint. The absence of that detail makes it harder to reproduce the analysis or judge whether the reported event corresponds to a clinically meaningful level of continued treatment.

The registry identifies randomized allocation but also records a single-group model. In general, randomization supports inference about group differences when assignment and follow-up are implemented as intended and outcomes are analyzed appropriately. The model discrepancy and lack of a reported analysis plan mean that those assumptions cannot be fully checked from the registry record. The comparison should therefore be interpreted as the posted result with these documentation limits in mind.

General statistical principle: a non-significant p-value does not demonstrate equivalence, and a nominally significant result would not by itself establish a large or important effect. For the posted primary endpoint, the estimate, interval, study context, outcome definition, and analysis documentation all matter together. The results describe the observed group contrast and its precision; they do not resolve every methodological uncertainty or establish broader effects.

11. Confidence Intervals and Precision

The primary risk-difference estimate is -0.189, with a reported 95% confidence interval from -0.376 to -0.002. The interval is wide relative to the estimate’s proximity to zero. It includes values close to no difference and values farther in the negative direction, so the direction of the observed contrast is clearer than its precise magnitude. The upper bound being near zero also means that modest changes in the data or analysis could affect how the comparison is characterized.

A confidence interval is a measure of statistical uncertainty under the method used. In general, a narrower interval indicates greater precision, while a wider interval indicates less precision. It does not show the range of outcomes for individual participants, and it does not assign a probability to a particular effect value after the data have been observed. The interval should be read as a range of effect estimates compatible with the data and method, rather than as proof that every value within it is equally plausible.

The registry also reports intervals for risk-ratio and odds-ratio measures. Their intervals span the null value for those scales, illustrating that the data do not pin down the relative comparison precisely. These measures use different scales from the risk difference and should not be interpreted as independent evidence, since they arise from the same group outcomes.

Precision is affected by the available outcome data and by how consistently the endpoint is defined and recorded. The primary group counts are 34 of 48 and 27 of 52. The record does not explain the retention classification threshold or report missing-data handling, so the displayed interval cannot address uncertainty arising from those operational choices. The statistical interval captures sampling uncertainty under its calculation, not every source of uncertainty in study conduct or reporting.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total events. They are illustrations based on the observed effect and assume a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. The assumed effect to detect is a stated fraction of the observed effect. The registry record does not report the original trial’s own sample-size assumptions or justification, so these planning illustrations should not be read as the design calculation used to enroll participants.

The 80% power column gives the estimated participants per arm under which a future study would have an 80% chance of detecting the specified effect, if the listed assumptions and statistical model hold. The 90% power column gives the corresponding per-arm estimate for a higher chance of detection. These are planning probabilities under assumptions, not guarantees about the result of a future study. They are not observed power.

The displayed scenarios plan for 100%, 75%, and 50% of the observed effect. A smaller effect is harder to distinguish from random variation, so the planning estimates require more participants per arm. This pattern is visible across the scenarios for the primary comparison and the posted secondary comparisons. The values are sensitive to the observed effect used as their starting point; an uncertain or unusually large observed effect can make a planning illustration a poor guide to the effect expected in a new study.

These estimates do not establish that the original trial was adequately or inadequately sized, because its own design assumptions are not reported in the registry record. They are best treated as conditional planning aids that show how required sample size changes with the target effect and desired power, rather than as a retrospective judgment on the trial.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.708; treatment proportion p₁ = 0.519; average p̄ = 0.614.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.6886 = 1.3495.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.6754 = 0.5685.
4. n = (1.3495 + 0.5685)² / (-0.1891)² = 102.9, rounded up to 103 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.708333333333333                          # control proportion
p1 <- 0.519230769230769                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(4129580)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 103 (80% power), 138 (90% power)
 75% of observed effect: n per arm = 180 (80% power), 240 (90% power)
 50% of observed effect: n per arm = 393 (80% power), 525 (90% power)
power.prop.test: 102.9 
Simulated power at n = 103 per arm: 0.809 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry identifies one primary outcome and lists secondary outcomes related to substance use and mental health. It reports no analyses in the reported-analyses list. No multiplicity procedure, hierarchy among secondary endpoints, or rule for controlling the overall false-positive rate is reported in the registry record. This matters because interpreting several outcome comparisons as though each were a standalone confirmatory test can overstate the strength of evidence. General statistical principle: multiplicity methods or a prespecified outcome hierarchy can help distinguish confirmatory questions from exploratory ones.

The record does not describe how missing outcome data were handled. In particular, the registry does not report whether all randomized participants were included in the primary analysis, how unavailable electronic medical record data were treated, or whether any imputation or sensitivity analyses were used. These details are important because missing outcomes can affect both the group counts and the estimated comparison, depending on their frequency and pattern.

Analysis populations are also not specified. The registry does not state whether the posted results follow an intention-to-treat approach, a per-protocol approach, or another population definition. It is therefore not possible to determine from the record how participants who did not follow their assigned care approach, if any, were treated analytically. The difference between arm-specific denominators across outcomes is visible in the posted summaries, but the reasons for those differences are not reported in the registry record.

The record identifies randomization and two assigned arms, but its model field says single group. It does not report the allocation sequence method, concealment procedures, or a statistical analysis plan that would clarify the intended analysis. These omissions do not negate the posted counts, but they limit independent assessment of whether the analysis matches the intended design and how robust the findings are to missingness, population definitions, or multiple testing.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: overdose (2.1%).

The serious adverse event summaries report one affected participant in the treatment-as-usual-plus-reSET-O arm among 48 at risk, and none in the treatment-as-usual-only arm among 52 at risk. No deaths are recorded in either group. The registry also lists overdose at 2.1% among the top serious adverse event terms.

These figures are counts reported for the recorded safety categories; they are not a complete account of the timing, duration, severity, or relationship of events to the intervention. The registry record does not provide further event narratives or a statistical comparison of safety between groups. With few reported serious events, the counts offer limited precision for comparing uncommon harms. A zero count in a group means no event was recorded in that group in this summary; it does not establish that the event cannot occur.

Safety counts should be interpreted separately from the primary retention result. The registry does not report a formal analysis connecting serious adverse events with treatment retention or app use. These summaries describe what was recorded, rather than establishing whether the intervention caused or prevented an event.

16. What the Result Does, and Does Not, Mean

The central result is a difference in the posted retention outcome counts between the randomized arms. More participants in the treatment-as-usual-plus-reSET-O arm were recorded as meeting the retention outcome in the posted data, and the reported risk-difference estimate is negative when expressed as TAU Only relative to TAU plus reSET-O. The confidence interval extends from -0.376 to -0.002, and the unadjusted p-value is 0.053. Together, these statistics indicate an uncertain estimate whose magnitude is not precisely determined.

The result does not establish that reSET-O causes improved retention. The registry record has a discrepancy between its randomized-trial description and single-group model field, and it does not report the analysis population, missing-data methods, or original analysis plan. The outcome definition also does not provide the precise appointment threshold used to classify retention. Those issues limit how fully the comparison can be evaluated.

The result also does not establish that the two approaches are equivalent or that there is no difference. A p-value above a conventional threshold is not proof of no effect. Conversely, the observed group contrast is not proof of a causal effect or a guarantee that a similar difference would occur in another setting. The posted confidence interval conveys substantial uncertainty about the size of the contrast.

The secondary outcomes address different domains and are not interchangeable with treatment retention. Numerical results are posted for two substance-use outcomes, while mental-health secondary outcome results are not reported in the registry record. The serious adverse event summary records few events and does not provide a formal comparative safety analysis. These data add context but do not settle the primary question.

The findings are most appropriately understood as a registry-reported comparison in this completed study, with uncertainty and reporting limitations. They do not provide medical advice or establish what an individual should do. Further conclusions would require fuller information about study conduct, endpoint definitions, missing data, and the planned analyses.

17. Why This Trial Matters Statistically

This trial raises a useful statistical question about how to interpret a binary retention outcome in a randomized comparison. The result is not adequately described by a single p-value: the risk difference gives the direction and scale of the observed contrast, while the confidence interval shows the uncertainty around that estimate. The unadjusted p-value of 0.053 is close to a conventional threshold, illustrating why a threshold alone is an incomplete summary.

The record also highlights the importance of transparent endpoint definitions. Retention is described through attended appointments counted from electronic medical records, but the classification threshold is not reported. A clearly stated outcome rule helps readers understand what the event represents and whether the result can be reproduced or compared with other studies.

Documentation of the analysis plan, missing-data methods, and analysis population is equally important. Without those details, readers cannot fully assess whether the posted results align with the randomized design or how incomplete records may have influenced the comparison. The single-group model descriptor alongside the randomized-trial description further demonstrates why registry fields should be read together rather than in isolation.

Finally, the secondary outcomes and planning illustrations show different uses of statistical evidence. Secondary outcomes can add context but need multiplicity-aware interpretation, while sample-size illustrations can inform future planning but do not establish what the original trial was designed to detect. Keeping these roles distinct helps avoid overstating what the registry results can support.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations specific to this record affect interpretation. The registry describes a randomized controlled trial and lists randomized allocation, but the model field says single group. No explanation for that discrepancy is reported. The record also does not provide allocation concealment details or a full statistical analysis plan, making it difficult to assess the intended analysis framework.

The primary outcome description says appointment attendance was used to gauge retention, but it does not specify the rule for classifying a participant as retained. Handling of missing or incomplete electronic medical record data is not reported. The analysis population and reasons for differences in outcome denominators are also not stated, so the relationship between enrollment and the posted analyses cannot be fully assessed.

Only summary counts and unadjusted comparisons are posted for the primary outcome. The registry’s reported-analyses list is empty, and there is no reported covariate-adjusted analysis, sensitivity analysis, or multiplicity procedure. Numerical results for the listed mental-health secondary outcomes are not reported in the registry record. These gaps constrain assessment of robustness and of secondary findings.

The primary estimate has a confidence interval extending close to zero and farther in the negative direction, indicating limited precision about the effect size. Serious adverse event counts are sparse, and the registry provides no detailed event narratives or formal between-group safety analysis. These limitations mean the record supports a descriptive reading of the posted outcomes, but not a complete evaluation of study conduct, outcome ascertainment, or causal interpretation.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: reSET-O RCT (Randomized Controlled Trial) — NCT04129580.
• Registry record last updated 2024-11-20; status last verified 2024-10.
• Results first posted 2024-05-23.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

NCT04129580 is a completed study sponsored by Milton S. Hershey Medical Center, with enrollment of 100 and a study period from 2020-09-15 to primary completion on 2023-07-14. The registry describes randomized assignment to treatment as usual plus reSET-O or treatment as usual alone, with no masking. Its model field is listed as single group, a discrepancy the record does not explain.

The primary outcome is retention in medication-assisted treatment over 6 months, assessed using appointment attendance recorded through electronic medical records. The posted risk difference for TAU Only relative to TAU plus reSET-O is -0.189, with a 95% confidence interval from -0.376 to -0.002 and an unadjusted p-value of 0.053. The registry also lists substance-use and mental-health secondary outcomes, but numerical results are posted for only two substance-use outcomes. Analysis-plan, missing-data, and analysis-population details are not reported in the registry record.