Nudges and Incentives to Enhance the Opioid Treatment Workforce: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Nudges and Incentives to Enhance the Opioid Treatment Workforce (NCT03834025), a randomized na study. The primary outcome is Number of Recruited Providers That Contact Staff for Information About Enrolling in the Study.

2. Clinical Question

3. Trial Design

This was a randomized, unmasked, factorial study sponsored by the University of North Carolina, Chapel Hill. The registry identifies the condition as opioid abuse (disorder), but the primary outcome concerns providers’ response to recruitment materials rather than a clinical outcome in patients. The recorded enrollment is 73. The study is marked completed, with a start date of 2019-02-12 and a primary completion date of 2020-08-28.

Phase 1 used four recruitment conditions. Recruitment as usual had neither prosocial messaging nor an additional mention of financial support. The other conditions added a compensation mention, prosocial messaging, or both. The primary comparisons assess each of those three conditions against recruitment as usual. The factorial label describes the combination of the two recruitment-message features; the registry record does not report further details of randomization procedures or allocation ratios.

The registry also describes three Phase 2 compensation conditions: no compensation, 30% compensation, and 100% compensation. These apply to participants who enroll, whereas the primary endpoint concerns providers’ initial contact about enrollment. The Phase 2 arms have no posted primary endpoint values, and the primary outcome description says it is not relevant for Phase 2. The registry record does not report further details of the Phase 2 allocation or the study’s original design assumptions.

The registered interventions are behavioral. No masking was used. The registry record does not report an analysis plan that would establish how the factorial features were modeled jointly, whether particular contrasts were prespecified, or how participants were assigned to analysis populations.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Once a participant was enrolled, then they were randomized for Phase 2. Phase 2 involved three levels of compensation: no compensation, 30% compensation, 100% compensation. Participants were compensated for each learning collaborative attended based on the Phase 2 randomization.

Only the first 6 of 7 groups are shown.

Sent Recruitment Letter (Phase 1)

Randomized to Payment Condition(Phase 2)

Analysis population for the primary outcome, as defined in the registry: This outcome is not relevant for Phase 2

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Investigators did not have access to baseline demographic information for study participants.

Only the first 6 of 8 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as Number of Recruited Providers That Contact Staff for Information About Enrolling in the Study, measured over From start of recruitment to end of data collection for last participant, up to 1.5 years.

The primary endpoint counts recruited providers who contacted study staff to ask for information about enrolling. The registry allows contact by email, phone, or another method. It is a count outcome, reported for the four Phase 1 recruitment conditions. The recorded Phase 1 denominators are the number of providers represented in each condition’s comparison; these are distinct from the registry’s overall enrollment figure of 73 participants.

This endpoint is used to assess an early step in recruitment: whether a provider reached out to learn about enrollment. It does not itself measure completion of enrollment, attendance at learning collaborative sessions, receipt of a DATA 2000 waiver, or whether a participant began prescribing medication-assisted treatment paid by Medicaid. Those outcomes appear separately among the secondary outcomes. A contact count can therefore describe the initial response to recruitment materials, but it cannot by itself establish whether providers subsequently enrolled or changed clinical practice.

The stated measurement period runs from the start of recruitment through the end of data collection for the last participant, up to 1.5 years. The registry record does not report additional endpoint ascertainment details, such as how duplicate contacts were handled or how a contact was attributed to a condition.

7. Results Posted in the Registry

• Phase 1 Condition 2 (no Prosocial Messaging, Compensation): 0.9% (36/3871) versus 0.9% (36/4150) on control; risk difference 0.1 percentage points (95% CI -0.4 to 0.5), risk ratio 1.07 (0.68–1.70); unadjusted two-proportion z-test p = 0.767.
• Phase 1 Condition 3 (Prosocial Messaging, no Compensation): 1.1% (44/4121) versus 0.9% (36/4150) on control; risk difference 0.2 percentage points (95% CI -0.2 to 0.6), risk ratio 1.23 (0.79–1.91); unadjusted two-proportion z-test p = 0.352.
• Phase 1 Condition 4 (Prosocial Messaging, Compensation): 1.2% (44/3693) versus 0.9% (36/4150) on control; risk difference 0.3 percentage points (95% CI -0.1 to 0.8), risk ratio 1.37 (0.89–2.13); unadjusted two-proportion z-test p = 0.154.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Number of Recruited Providers That Complete Enrollment in the Phase 2 Study

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • Phase 1 Condition 2 (no Prosocial Messaging, Compensation): 0.4% (17/3871) versus 0.4% (18/4150) on control; risk difference 0.0 percentage points (95% CI -0.3 to 0.3), risk ratio 1.01 (0.52–1.96); unadjusted two-proportion z-test p = 0.971.
  • Phase 1 Condition 3 (Prosocial Messaging, no Compensation): 0.5% (22/4121) versus 0.4% (18/4150) on control; risk difference 0.1 percentage points (95% CI -0.2 to 0.4), risk ratio 1.23 (0.66–2.29); unadjusted two-proportion z-test p = 0.512.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Phase 1 Condition 2 (no Prosocial Messaging, Compensation) and the control. The interval gives the range of effects that remain compatible with the data.
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Phase 1 Condition 3 (Prosocial Messaging, no Compensation) and the control. The interval gives the range of effects that remain compatible with the data.

The secondary outcomes concern later stages of participation and potential practice-related outcomes: completing enrollment in Phase 2, time spent attending a UNC ECHO clinic session, receiving a DATA 2000 waiver, and beginning Medicaid-paid medication-assisted treatment prescribing within one year of Phase 2 start. These outcomes are related to the study’s broader recruitment and workforce aims, but they are not interchangeable with the primary endpoint of contacting staff for enrollment information.

The posted secondary counts do not form a consistent pattern that would establish a clear advantage for one recruitment condition across the subsequent steps. They are descriptive counts, and the registry record does not report a complete set of inferential comparisons for the secondary outcomes. In particular, detailed results for attendance, waiver receipt, and prescribing are not reported in the registry record. The primary result therefore cannot be used as a substitute for evidence about enrollment completion or later provider activity.

Secondary outcomes generally carry less confirmatory weight when many outcomes or comparisons are considered without a reported multiplicity plan. Testing multiple outcomes increases the opportunity for chance findings, and unadjusted results do not account for that issue. The registry record does not report a multiplicity adjustment or a hierarchy that would distinguish confirmatory from exploratory secondary analyses. Accordingly, secondary patterns are best treated as context for the primary recruitment question, rather than as independent proof of an effect.

The different endpoints also measure different parts of a possible pathway. Initial contact, completed enrollment, attendance, waiver receipt, and prescribing are separate events. A difference at an early stage would not by itself demonstrate a difference at later stages, and absence of a clear primary contrast does not determine the results of those later outcomes.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Number of Recruited Providers That Complete Enrollment in the Phase 2 Study
• Total Amount of Time a Participant Attends a UNC ECHO Clinic Session
• Number of Participants That Receive a DATA 2000 Waiver
• Number of Participants That Begin Prescribing MAT Paid by Medicaid Within One Year of Phase 2 Start

9. Statistical Interpretation

10. Statistical Methods Explained

General: A randomized comparison assigns participants or units to conditions by chance, which can help make groups comparable at the start of a study. Here, the registry identifies the study as randomized and unmasked. It does not report the specific sequence-generation method, concealment procedures, or allocation ratio. The registered factorial structure concerns two features of Phase 1 recruitment: prosocial messaging and an additional mention of financial support.

What does a risk difference describe?

General: A risk difference is the event probability in one condition minus the event probability in a reference condition. It expresses an absolute difference, rather than a relative ratio. For this endpoint, the event is a provider contacting study staff about enrollment. A positive estimate means the recorded probability is higher in the named condition than in the reference condition; a negative estimate would mean it is lower. The posted estimates are expressed as decimals, and should be read on that probability-difference scale.

The registry also displays the underlying event counts and denominators. Those are useful because they make clear how many contacts contributed to each condition’s estimate and how many providers were represented in the comparison. Counts alone can be misleading when denominators differ, which is why the comparison uses rates and a risk difference rather than simply comparing the counts.

What do the intervals and p-values add?

General: An interval estimate describes a range of values compatible with the observed data under the statistical procedure used. A narrower interval indicates greater precision on that scale; an interval spanning zero does not rule out no difference. The registry record does not report the confidence level or the interval-construction method, so those details cannot be specified from the record.

A p-value measures how unusual data at least as extreme as those observed would be under a specified null model. It is not the probability that the null hypothesis is true, nor a measure of the size or practical importance of an effect. The posted p-values are unadjusted. The registry record does not report whether they were calculated under a particular factorial model, or how any dependence among the planned comparisons was handled.

Why does the factorial structure matter?

General: A factorial study can assess multiple intervention features in a shared design. Depending on the analysis, it may estimate effects for each feature, their combination, and whether one feature’s effect depends on the other. The posted Phase 1 results are pairwise comparisons against recruitment as usual. The registry record does not report a separate analysis of the main effects or interaction between prosocial messaging and the added compensation mention. The pairwise estimates should therefore not be presented as a complete factorial analysis.

The intervention wording also requires care. The Phase 1 compensation feature is an additional mention of financial support in recruitment materials. Phase 2 describes compensation levels for enrolled participants. These are different stages and should not be treated as the same intervention contrast.

What can the planning figures tell readers?

General: Sample-size planning translates a target effect and analysis assumptions into an approximate number of participants needed to detect that effect with a chosen probability. The registry’s planning illustration uses the observed effect or a stated fraction of it, a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. The figures are participants per arm because these are not time-to-event outcomes. The original trial’s sample-size justification and design assumptions are not reported in the registry record. The planning illustration is not a report of the study’s actual design target.

General: Smaller target effects generally require more participants to distinguish them from random variation, all else being equal. Planning values can be especially large when the observed event probabilities and differences are small. They should be read as conditional illustrations, not as evidence about what the completed study was powered to detect.

The registry record does not report the statistical handling of missing outcome data, interim analyses, or a complete protocol for the primary comparison. These omissions limit the level of detail with which the calculations and their relationship to the original design can be assessed.

11. Confidence Intervals and Precision

The point estimates are close to zero on the risk-difference scale, and every posted interval crosses zero. For the compensation mention without prosocial messaging, the estimate is 0.001 and the interval runs from -0.004 to 0.005. For prosocial messaging without the additional compensation mention, the estimate is 0.002 and the interval runs from -0.002 to 0.006. For the combined recruitment materials, the estimate is 0.003 and the interval runs from -0.001 to 0.008.

These intervals indicate uncertainty around each estimate. Because each includes zero, the posted data are compatible with no difference, but they also allow nonzero differences in either direction. The intervals are not concentrated on a single value, so the point estimates alone should not be treated as precise estimates of the recruitment effects. The registry record does not report the confidence level or the interval method.

Precision also depends on the number of observed events, not just on the denominator. The primary comparisons contain 36 contacts in the reference condition and between 36 and 44 contacts in the other conditions. With few events relative to the denominators, the estimates can remain uncertain even when the denominators are substantial. This is a general statistical consideration; it does not establish why any particular interval has its reported width.

The intervals describe uncertainty for the posted pairwise comparisons. They do not account for unreported choices about multiplicity, missing data, or factorial modeling. Thus they should not be interpreted as resolving every uncertainty about the study’s recruitment effects. No confidence interval for the secondary outcomes is reported in the registry record.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size figures are planning illustrations for two-arm comparisons and give participants per arm, not total events. The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, a normal approximation, and an effect to detect equal to the observed effect or a stated fraction of it. The original trial’s own sample-size justification and design assumptions are not reported in the registry record. These figures are not observed power.

The 80% power column describes the approximate number of participants per arm under those assumptions for an analysis designed to detect the chosen effect with 80% power. The 90% power column gives the corresponding planning target for 90% power. In general, higher planned power requires more participants, all else equal. Smaller effects also require more participants because they are harder to distinguish from random variation.

For example, the rows labeled as planning for 75% or 50% of the observed effect represent progressively smaller target effects than planning for the full observed effect. The participant requirements rise as the target effect becomes smaller. These are hypothetical planning scenarios based on the observed comparisons, not recommendations about an appropriate future study size. They also do not establish that the completed study was designed around these values or had a particular probability of detecting its results.

The curves should be read in the same conditional way: they relate an assumed effect and sample size to a planning quantity under the stated assumptions. They do not replace a design analysis that specifies the intended primary contrast, model, and analysis population.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.009; treatment proportion p₁ = 0.009; average p̄ = 0.009.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.1335 = 0.2616.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.1335 = 0.1123.
4. n = (0.2616 + 0.1123)² / (0.0006)² = 357,663.3, rounded up to 357,664 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.00867469879518072                          # control proportion
p1 <- 0.00929992250064583                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3834025)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 357,664 (80% power), 478,810 (90% power)
 75% of observed effect: n per arm = 630,368 (80% power), 843,883 (90% power)
 50% of observed effect: n per arm = 1,405,997 (80% power), 1,882,230 (90% power)
power.prop.test: 357663.3 
Simulated power at n = 357,664 per arm: 0.802 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The primary comparisons include three recruitment conditions contrasted with recruitment as usual. The registry record reports unadjusted p-values but does not report how multiplicity across those comparisons was handled. It also does not report whether the comparisons were prespecified as separate primary contrasts, whether the factorial features were evaluated jointly, or whether an interaction was tested. Without those details, the reported p-values should be read as unadjusted results rather than as a family-wise confirmatory analysis.

The registry lists multiple secondary outcomes, but does not report a complete inferential analysis for them. No hierarchy for primary and secondary hypotheses, correction for multiple testing, or exploratory-analysis designation is reported in the registry record. A pattern across multiple endpoints can be informative descriptively, but should not be treated as a set of independent confirmations when the analysis plan is unavailable.

Rules for missing data and the analysis population are also not reported in the registry record. It is not possible to establish from the record whether all randomized units were included in each comparison, how missing endpoint information was handled, or whether any exclusions occurred. The record does not report interim analyses or stopping rules for this outcome.

These omissions do not show that particular procedures were absent; they mean that the registry record does not describe them. They limit assessment of how the comparisons were selected, whether the uncertainty estimates reflect all relevant analysis choices, and how robust the conclusions would be to alternative handling of missingness or multiplicity.

15. What the Result Does, and Does Not, Mean

The posted results address whether recruitment materials were associated with differences in provider contact about study enrollment. They do not directly address whether medication-assisted treatment is effective, whether opioid-related health outcomes changed, or whether providers ultimately changed prescribing practice. The study’s primary endpoint is a recruitment process measure, not a patient health endpoint.

The primary estimates are positive, but their intervals include zero, and the unadjusted p-values do not establish a clear difference from recruitment as usual. This pattern is not proof that all recruitment conditions are equivalent. It means the posted estimates do not resolve whether the conditions produced no difference or a difference in either direction. The registry record does not report a prespecified equivalence margin or an equivalence analysis.

Contacting staff is an early step, separate from completing enrollment. Completing enrollment is in turn distinct from attending sessions, receiving a waiver, and beginning prescribing. The registry lists these outcomes separately. Without complete inferential results for those later endpoints, the primary comparisons cannot establish the downstream consequences of the recruitment approaches.

The registry describes Phase 2 compensation levels, but the primary endpoint is stated to be irrelevant for Phase 2. The primary comparisons should therefore not be interpreted as a test of those compensation levels. Similarly, the registry’s overall enrollment figure is not the primary endpoint denominator and does not itself indicate a recruitment effect.

The appropriate conclusion is limited to the posted Phase 1 comparisons: they show no clearly resolved difference in provider contact, while leaving uncertainty about the magnitude and direction of possible effects. They do not establish clinical benefit or lack of benefit, and they do not justify medical advice.

16. Why This Trial Matters Statistically

This trial illustrates a statistical distinction between improving an early recruitment measure and changing later behavior. Contact with study staff may be relevant to recruitment, but it is not the same outcome as enrollment completion, participation in sessions, or prescribing. Keeping those stages separate prevents a process measure from being mistaken for a clinical or workforce outcome.

It also shows why both counts and denominators matter. The conditions have different numbers of providers represented in the primary comparisons, and their contact counts alone do not express the probability of contacting staff. Risk differences place comparisons on an absolute probability scale, while intervals show that the point estimates remain uncertain.

The factorial structure raises a further analytic issue. A design that varies two recruitment features can potentially inform their separate and combined contributions, but pairwise comparisons against one reference condition do not, by themselves, report every factorial effect. The registry record does not report the joint factorial analysis, making it important to avoid attributing a result specifically to one feature beyond the comparisons actually posted.

Finally, this record demonstrates the value of transparent analysis details. Multiplicity handling, missing-data rules, analysis populations, and original sample-size assumptions affect how readers assess precision and evidential strength. Where these details are not reported, conclusions should remain tied to the estimates and uncertainty that are available, without treating an absence of clear evidence as proof of no effect.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record gives limited detail about the statistical plan. It does not report the original sample-size justification, a full analysis specification, the confidence level or construction method for the intervals, or how the factorial features were modeled. It also does not report multiplicity handling, missing-data rules, interim analyses, or the analysis population.

The primary endpoint measures contact about enrollment, not completed enrollment or a clinical outcome. It is therefore a narrow indicator of response to recruitment materials. The record does not describe how repeat contacts were counted, or how contacts made through different channels were verified and attributed.

The overall recorded enrollment is 73, whereas the primary comparison denominators represent much larger groups. Those quantities refer to different parts of the study record and should not be conflated. The registry does not provide enough information to reconstruct every step from recruitment exposure through contact and enrollment.

Results for the secondary endpoints are incomplete as inferential evidence. The registry lists later outcomes, but does not report a complete set of their analyses. The Phase 2 arms have no posted primary endpoint values, and the primary endpoint is designated as not relevant to Phase 2; this should not be read as a measured Phase 2 null result.

Finally, the adverse-event fields show zero participants at risk across the listed conditions. With no participants recorded as at risk in those fields, those entries cannot support a conclusion that no adverse events occurred.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Nudges and Incentives to Enhance the Opioid Treatment Workforce — NCT03834025.
• Registry record last updated 2021-08-20; status last verified 2021-06.
• Results first posted 2021-08-20.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT03834025, titled “Nudges and Incentives to Enhance the Opioid Treatment Workforce,” is a completed, randomized, unmasked factorial study sponsored by the University of North Carolina, Chapel Hill. It recorded enrollment of 73 and focused its primary outcome on the number of recruited providers who contacted study staff about enrollment.

Phase 1 compared recruitment as usual with materials adding a compensation mention, prosocial messaging, or both. All three posted risk-difference intervals include zero. The registry also describes Phase 2 compensation conditions and several secondary outcomes, but a complete inferential analysis of those later outcomes is not reported in the registry record. Original design assumptions, multiplicity handling, and missing-data procedures are not reported in the registry record.