This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for Mobile Medication Adherence Platform for Buprenorphine-Naloxone During Treatment of Opioid Use Disorder: Phase I Study (NCT04656899), a randomized na study. The primary outcome is Mean Medication Possession Ratio.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT04656899 |
| Phase | NA |
| Condition | Opioid-use Disorder |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | Double |
| Enrollment | 41 |
| Arms in the primary outcome | 2 |
| Lead sponsor | Washington State University |
| Results posted | Yes |
2. Clinical Question
Population
The study concerned patients with opioid-use disorder who had been prescribed buprenorphine-naloxone. The registry describes a phase I feasibility study and records an enrollment of 41 participants. It does not report further eligibility criteria or detailed characteristics of the enrolled participants.
Intervention
Participants assigned to the Pillsy arm received an active smart cap and a smartphone application. The cap was described as collecting information about bottle openings and missed doses, with text messages and automated calls among the reminder features. The registry describes the intervention as reminder messages intended to improve adherence.
Comparator
Participants assigned to service as usual received an inactive Pillsy cap that tracked bottle openings but did not provide reminders or other messaging. They also received a basic application that could deploy satisfaction surveys. The registry does not report additional detail about usual care beyond this description.
Primary question
The primary outcome was mean medication possession ratio over 12 weeks, based on pharmacy fills and days of medication supply. The study asked whether the mean differed between the active Pillsy arm and service as usual. The outcome is a pharmacy-based measure of medication availability; it does not directly establish whether medication was taken as prescribed.
3. Trial Design
The registry describes a randomized, double-masked, parallel-group study. Enrollment was 41, with 19 participants in the Pillsy arm and 22 in the service-as-usual arm for the primary outcome. The record is marked completed. The recorded start date is 2020-12-06, and the primary completion date is 2021-10-11. Washington State University is listed as the sponsor.
The study title and summary call the investigation phase I and describe it as a feasibility trial. The registry's phase field is recorded as NA. The stated focus was the feasibility of the Pillsy technology and associated smartphone application, with adherence and participant satisfaction among the measures of interest. The record does not report a formal phase classification beyond the NA entry.
In the active arm, the smart cap was described as collecting bottle-opening data and information about missed doses, while reminders could be sent by text message or automated phone call. The service-as-usual arm used an inactive cap that tracked openings without providing reminders or other messaging. Both arms were described as receiving buprenorphine-naloxone prescriptions. The registry does not explain who was masked, how masking was maintained, how the random sequence was generated, or how allocation was concealed.
The primary outcome was mean medication possession ratio over 12 weeks. Six secondary outcomes are listed, including engagement, satisfaction, and measures related to substance use. Numerical summaries are posted for some secondary outcomes, while other listed outcomes do not have numerical arm results in the record. Details of the original sample-size justification, the planned statistical analysis, and the analysis populations are not reported in the registry record.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: 84 individuals were assessed for eligibility, of these 41 were randomized
Overall Study
| Stage | Pillsy Arm | Service as Usual Arm |
|---|---|---|
| STARTED | 19 | 22 |
| COMPLETED | 16 | 13 |
| NOT COMPLETED | 3 | 9 |
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
| Characteristic | Unit | Pillsy Arm | Service as Usual Arm | Total |
|---|---|---|---|---|
| Participants analysed | 19 | 22 | 41 | |
| Age, Continuous | years | 36.1 (SD 8.1) | 37.7 (SD 11) | 36.9 (SD 9.7) |
| Sex: Female, Male: Female | Participants | 12 | 10 | 22 |
| Sex: Female, Male: Male | Participants | 7 | 12 | 19 |
| Race/Ethnicity, Customized: American Indian | Participants | 2 | 0 | 2 |
| Race/Ethnicity, Customized: White | Participants | 16 | 20 | 36 |
| Race/Ethnicity, Customized: Black | Participants | 0 | 2 | 2 |
| Race/Ethnicity, Customized: Prefer not to say | Participants | 1 | 0 | 1 |
| Employment: Unemployed | Participants | 11 | 13 | 24 |
| Employment: Employed | Participants | 8 | 9 | 17 |
| Marital Status: Single | Participants | 5 | 15 | 20 |
| Marital Status: Married | Participants | 9 | 3 | 12 |
| Marital Status: Divorced | Participants | 5 | 4 | 9 |
| Last Buprenorphine Initiation: Not Eligible | Participants | 0 | 1 | 1 |
| Last Buprenorphine Initiation: 0 to 30 days ago | Participants | 5 | 8 | 13 |
| Last Buprenorphine Initiation: 31 to 60 days ago | Participants | 2 | 1 | 3 |
| Last Buprenorphine Initiation: 60 to 90 days ago | Participants | 2 | 1 | 3 |
| Last Buprenorphine Initiation: 91 to 120 days ago | Participants | 9 | 11 | 20 |
| Last Buprenorphine Initiation: Missing | Participants | 1 | 0 | 1 |
| Baseline Opioid Use Disorder (OUD) Diagnostic and Statistical Manual (DSM) Diagnosis: Mild | Participants | 9 | 5 | 14 |
| Baseline Opioid Use Disorder (OUD) Diagnostic and Statistical Manual (DSM) Diagnosis: Moderate | Participants | 0 | 2 | 2 |
| Baseline Opioid Use Disorder (OUD) Diagnostic and Statistical Manual (DSM) Diagnosis: Severe | Participants | 10 | 15 | 25 |
6. Primary Endpoint
The registry lists the primary outcome as Mean Medication Possession Ratio, measured over 12 weeks.
| Element | Registry entry |
|---|---|
| Result type | Mean |
| Measure of spread | Standard Deviation |
| Unit | Medication supply / days in time period |
| Endpoint classification | Continuous Endpoint |
Mean medication possession ratio, or MPR, was the primary endpoint over 12 weeks. The registry defines it as the sum of the days' supply from pharmacy fills during a period divided by the number of days in that period. The applicable period could vary according to the prescription length set by the provider. The endpoint was based on pharmacy fills and days of supply, rather than a direct measurement of ingestion.
MPR is often expressed as a percentage, but this record reports it as the result of the fraction. A higher fraction represents more medication supply relative to the measured time period. As a general statistical point, a pharmacy-based possession measure can be useful for comparing medication availability between groups, but it does not by itself show whether doses were taken, whether the medication was used as directed, or why a participant might have had a gap in supply.
The posted mean was 0.839 in the Pillsy arm and 0.752 in service as usual. The posted standard deviations were 0.652 and 0.713, respectively. The unadjusted mean difference, calculated from the posted summaries, was 0.087, with a 95% confidence interval from -0.331 to 0.505. These results describe the observed pharmacy-fill measure in the enrolled groups; they are not a direct measure of clinical outcome or medication-taking behavior.
7. Results Posted in the Registry
- Pillsy Arm: mean 0.84 versus 0.75 on control; difference 0.09 (95% CI -0.33 to 0.50), standardized effect d = 0.13; z-test p = 0.683.
Mean difference: Pillsy Arm
95% CI: -0.33 to 0.50
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Participant Engagement | Continuous Endpoint | Mean difference | 18.70 (-1.88 to 39.28) | 53 per arm |
| Week 1, Number of Participants Satisfied With Pillsy Usability and Reminder Feedback System | Binary Endpoint | Risk difference | -7.2 pts (-32.0 to 17.6) | 455 per arm |
| Change in Use of Non-prescribed Opioids | Continuous Endpoint | Mean difference | 0.49 (-3.25 to 4.23) | 2,265 per arm |
Participant Engagement
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Pillsy Arm: mean 71.30 versus 52.60 on control; difference 18.70 (95% CI -1.88 to 39.28), standardized effect d = 0.54; z-test p = 0.075.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Pillsy Arm and the control. The interval gives the range of effects that remain compatible with the data.
Week 1, Number of Participants Satisfied With Pillsy Usability and Reminder Feedback System
- Endpoint type: Binary: the number of participants with the event, compared as proportions.
- Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
- Result:
- Pillsy Arm: 77.8% (14/18) versus 85.0% (17/20) on control; risk difference -7.2 percentage points (95% CI -32.0 to 17.6), risk ratio 0.92 (0.67–1.25); unadjusted two-proportion z-test p = 0.566.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Pillsy Arm and the control. The interval gives the range of effects that remain compatible with the data.
Change in Use of Non-prescribed Opioids
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Pillsy Arm: mean 2.17 versus 1.68 on control; difference 0.49 (95% CI -3.25 to 4.23), standardized effect d = 0.08; z-test p = 0.797.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Pillsy Arm and the control. The interval gives the range of effects that remain compatible with the data.
The secondary outcomes span several different concepts: engagement, satisfaction with the usability and reminder feedback system, and substance-use-related measures. The registry lists six secondary outcomes, but numerical arm summaries are present for only some of them. The secondary outcomes therefore do not form a single coherent scale or a direct replication of the primary MPR measure.
Across the numerical secondary results, estimates vary in direction and precision. Their intervals are broad, and the unadjusted comparisons do not provide a consistent, independently confirmatory pattern. The engagement and satisfaction measures relate to experience with or interaction with the platform, whereas MPR reflects medication supply from pharmacy fills. They may help describe feasibility, but they are not interchangeable with the primary endpoint. The substance-use-related measure addresses a different outcome again, so it should not be read as a direct explanation for the primary MPR result.
As general statistical context, examining multiple outcomes increases the chance that at least one estimate appears notable by chance, particularly when each is evaluated without adjustment. The registry does not report a multiplicity plan, a hierarchy among secondary outcomes, or adjusted secondary analyses. The posted p-values are unadjusted recalculations, and the registry lists no reported analyses for these outcomes. This makes the secondary results less suitable for confirmatory conclusions than a prespecified, fully reported analysis.
The primary result is itself imprecise, and the secondary data do not resolve that uncertainty. The differences between measures, outcome counts, and denominators also mean that secondary findings should be interpreted separately rather than combined into a claim about a single overall effect. Missing numerical results for listed outcomes further limit the breadth of what can be inferred from the registry record.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Participant Engagement
- Week 1, Number of Participants Satisfied With Pillsy Usability and Reminder Feedback System
- Number of Days Participant Self-reported Other Substance Use
- Change in Use of Non-prescribed Opioids
- Week 6, Number of Participants Satisfied With Pillsy Usability and Reminder Feedback System
- Week 12, Number of Participants Satisfied With Pillsy Usability and Reminder Feedback System
9. Statistical Interpretation
The posted primary outcome summaries show a mean MPR of 0.839 for the Pillsy arm and 0.752 for service as usual. The unadjusted difference between the means was 0.087 in favor of the Pillsy arm. Its 95% confidence interval extended from -0.331 to 0.505, spanning no difference and estimates in either direction. The interval is wide relative to the point estimate, so the observed difference is not precise.
The unadjusted p-value calculated from the posted summary statistics was 0.683. This is not a p-value from a reported sponsor analysis: the registry record lists no reported analyses for the primary outcome. The recalculation is a simple, unadjusted comparison of the arm summaries. It should therefore be read as a statistical description of the posted data, not as a substitute for an analysis that may have been planned or conducted by the study investigators. No discrepancy between the posted results and the recalculation is flagged.
A confidence interval that includes zero is consistent with a range of possible group differences, including no mean difference. It does not establish that the groups were equivalent, and it does not show that every value inside the interval is equally plausible. The broad interval also means that the data do not tightly distinguish a small difference from larger differences in either direction. With the recorded enrollment of 41 and outcome counts of 19 and 22, the comparison has limited precision.
The comparison is about MPR as defined from pharmacy fills and days' supply. It cannot on its own establish medication ingestion, explain the reasons for medication availability, or determine whether the platform changed a participant's clinical status. The registry describes a feasibility focus, and the primary outcome is an adherence-related measure. The data should be interpreted within that scope.
The record also lists engagement, satisfaction, and substance-use-related secondary outcomes. Some have posted numerical summaries, but no analyses are listed as reported analyses. Secondary findings are therefore descriptive and exploratory in this record. Multiple outcomes create opportunities for chance findings when each is assessed separately; without a reported multiplicity strategy, a secondary p-value should not be treated as confirmatory evidence. Overall, the posted primary summaries show a higher arm mean for MPR in the Pillsy group, but the unadjusted interval and lack of a reported registry analysis leave substantial uncertainty about the size and direction of the underlying difference.
10. Statistical Methods Explained
The primary endpoint is a continuous measure summarized by its mean and standard deviation in each arm. The posted mean describes the average MPR within an arm, while the standard deviation describes how dispersed participant values were around that arm's mean. These summaries do not show the individual participant values or the shape of their distribution.
What does a mean difference represent?
The mean difference compares the average outcome in one arm with the average in the other. For the primary outcome, the registry's comparison is Pillsy minus service as usual: 0.839 minus 0.752 as represented by the posted summaries, with a difference of 0.087. A positive value indicates a higher mean in the Pillsy arm for this direction of comparison. This is an unadjusted contrast and does not account for participant characteristics or other covariates.
The registry includes an unadjusted p-value and confidence interval derived from the posted summaries, but it does not report a sponsor analysis or specify the statistical test used in a planned analysis. The recalculated comparison should not be interpreted as proof that the trial's original analysis used a particular method. The study's own statistical plan, including any adjustments or model choices, is not reported in the registry record.
What does the standard deviation tell us?
A standard deviation describes variability among observed participant values on the outcome scale. It is not the uncertainty around the mean difference; that uncertainty is conveyed more directly by the confidence interval. The primary standard deviations were 0.652 for Pillsy and 0.713 for service as usual. The group summaries alone do not reveal whether the outcome distribution was symmetric, whether it contained unusually high or low values, or whether observations were missing.
In general, a mean-based comparison is easiest to interpret when the mean is a meaningful summary and the uncertainty calculation is appropriate for the data. Strong skew, outliers, or other departures from model assumptions can affect inferential results. The registry does not report diagnostic checks, transformations, robust methods, or participant-level data that would allow those issues to be assessed.
Why does uncertainty matter?
A confidence interval is a range of estimates compatible with the observed data under the method used to construct it. In repeated use of a valid procedure, intervals at a stated confidence level would cover the underlying parameter at the corresponding long-run rate. It is not a probability statement that the fixed true difference lies in this particular interval. The primary 95% interval, -0.331 to 0.505, is wide and includes zero. That means the posted summaries do not pinpoint the mean difference closely.
A p-value measures how incompatible the observed data are with a specified null model, assuming the analysis method and assumptions are appropriate. It does not measure the probability that the null hypothesis is true, and it does not describe the magnitude or clinical importance of a difference. The primary p-value of 0.683 is an unadjusted recalculation from summary data, not a reported sponsor result. The exact inferential approach used in any original analysis is not reported.
Why can over-dispersion matter?
As a general statistical point, over-dispersion means that the observed variability is greater than a model assumes. If a model understates variability, standard errors can be too small and confidence intervals too narrow; tests can then appear more decisive than warranted. This concern is especially relevant to count or binary outcomes when a model imposes a particular variance structure. It cannot be diagnosed from a mean and standard deviation alone, and the registry does not report a model-based variance assessment for this study. The primary outcome is reported as a continuous fraction with arm-specific standard deviations, so no claim about over-dispersion in these data can be made from the record.
The summary-based recalculation is unadjusted. It does not incorporate stratification, repeated measurements, baseline covariates, or other modeling choices. The registry does not report whether any such features were part of the original statistical plan, nor does it report an analysis population definition. The meaning of the posted comparison is consequently limited to the group summaries and the stated endpoint.
What can the secondary comparisons add?
Secondary outcomes use different measurement scales, including a continuous engagement measure, a participant count for satisfaction, and a continuous measure related to non-prescribed opioid use. A difference in means is used for continuous outcomes, while a risk difference is a difference between the proportions with an outcome for a binary measure. These are different estimands and should not be compared as if they measured the same quantity. The registry supplies no reported analysis details for these outcomes, and the posted comparisons are unadjusted. Their interpretation is also affected by the number of outcomes examined and by the outcome-specific denominators.
11. Confidence Intervals and Precision
The primary mean difference is 0.087, with a 95% confidence interval from -0.331 to 0.505. The interval includes zero and reaches into both negative and positive values. In practical terms, the posted results are compatible with a lower mean MPR in the Pillsy arm, little or no mean difference, or a higher mean MPR. The interval does not identify which of those possibilities best represents the underlying group difference.
The interval's width signals limited precision. This is consistent with the outcome counts of 19 in the Pillsy arm and 22 in service as usual, alongside substantial variation in the arm-level MPR values. The standard deviations were 0.652 and 0.713, respectively. The record does not provide a primary confidence interval from a sponsor-reported analysis, so the interval discussed is an unadjusted recalculation from the posted summaries.
Confidence intervals describe uncertainty in an estimate under the method used to calculate them; they do not cover uncertainty about whether MPR is the most relevant measure, whether missing outcomes affected the comparison, or whether the analysis assumptions were appropriate. Those questions require information not reported in the registry record. The interval also does not define a threshold for a meaningful difference, and no such threshold is reported.
The same caution applies to the other outcomes: their estimates and intervals reflect limited group sizes and are not substitutes for a prespecified analysis. More precision would require more information or data; the registry record does not establish that the estimates are stable across other populations or settings.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 975 | 1,305 |
| 75% of the observed effect | 1,733 | 2,320 |
| 50% of the observed effect | 3,899 | 5,220 |
The planning figures are illustrations based on the observed effect, not the trial's original design assumptions. They use a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. Each row asks what sample size per arm would be needed to detect a specified fraction of the observed effect under those assumptions. The displayed figures are participants per arm, not total participants across both arms and not total events.
The 80% power column gives the estimated participants per arm for a design with an 80% chance of detecting the chosen effect if that effect were the true one and the planning assumptions held. The 90% power column makes the same calculation for a design with a 90% chance. These are design-planning probabilities, not guarantees about a future result. The 100% row uses the full observed effect as the planning target; the 75% and 50% rows use smaller fractions of that observed effect.
Smaller effects are harder to distinguish from random variation, so the planning exercise requires more participants per arm as the target effect becomes smaller. For the primary outcome, the displayed requirements are 975 and 1,305 participants per arm for the full observed effect, 1,733 and 2,320 for planning at 75% of the observed effect, and 3,899 and 5,220 for planning at 50% of the observed effect, for 80% and 90% power respectively. These figures are not observed power and do not describe the chance that the completed study found a result.
A curve based on the same planning exercise can be read as a visual representation of how required sample size changes with the effect targeted. It inherits the same assumptions and should not be mistaken for a recruitment target chosen by the investigators. The registry does not report the study's original sample-size justification or original power assumptions.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.
Worked calculation for this trial
- Observed difference Δ = 0.087; pooled standard deviation σ = 0.686.
- n = 2 × (1.96 + 0.842)2 × 0.6862 / 0.0872 = 974.6, rounded up to 975 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
delta <- 0.087 # observed difference in means
sigma <- 0.685520969774084 # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")
# Simulation: exact two-sample t-test using simulated means and variances
set.seed(4656899)
n <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n)) # difference in sample means
s2 <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1)) # pooled sample variance
tstat <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))
Output from running the code
100% of observed effect: n per arm = 975 (80% power), 1,305 (90% power)
75% of observed effect: n per arm = 1,733 (80% power), 2,320 (90% power)
50% of observed effect: n per arm = 3,899 (80% power), 5,220 (90% power)
power.t.test: 975.6
Simulated power at n = 975 per arm: 0.799 (20000 replicates)
The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Multiplicity, Missing Data and Analysis Populations
The record names a primary outcome and six secondary outcomes, but it does not report how the analysis plan handled multiple comparisons. In general, testing multiple outcomes can raise the chance of a false-positive finding if each result is judged separately. A prespecified hierarchy, adjustment procedure, or clearly exploratory interpretation can help readers understand that risk; none is described in the registry record. The posted comparisons are unadjusted recalculations and should be treated accordingly.
The registry lists no reported analyses for the primary or secondary outcomes. It therefore does not report the sponsor's inferential method, any covariate adjustment, or whether analyses followed a prespecified plan. The outcome summaries and recalculated comparisons provide a descriptive view, but do not establish how the investigators handled the data in their own analysis.
Missing-data procedures are not reported in the registry record. Although arm-specific counts are available for the primary endpoint and several secondary measures, the record does not explain why counts differ across measures, whether any measurements were missing, how missing observations were handled, or whether imputation was used. It is not possible to determine from these summaries whether the available participants are representative of all randomized participants for each outcome.
The analysis population is also not reported. The record does not specify whether the estimates use all randomized participants, participants who received an intervention, participants with available outcome data, or another defined group. It does not state whether participants were analyzed according to assigned arm or whether exclusions occurred. These details matter because the interpretation of a randomized comparison depends in part on the population and analysis rules.
These omissions do not establish that a particular procedure was or was not used; they limit what can be concluded from the registry record. The study protocol, statistical analysis plan, and a full results report would be needed to assess these issues in greater detail.
15. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Pillsy Arm | 0 | 19 | 0.0% |
| Service as Usual Arm | 0 | 22 | 0.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Pillsy Arm | 0 of 19 |
| Service as Usual Arm | 0 of 22 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Pillsy Arm | 0 of 19 |
| Service as Usual Arm | 0 of 22 |
The registry records zero serious adverse events affecting participants in the Pillsy arm, among 19 at risk, and zero in the service-as-usual arm, among 22 at risk. It also records zero other adverse events affecting participants in either arm, with the same respective at-risk counts. Deaths are recorded as zero in both arms. These are the adverse-event figures posted for the study.
Zero reported events in a small study does not establish that an intervention has no safety risks. It means that the registry record lists no affected participants in these categories during the reported study record. The record does not provide a detailed account of how adverse events were collected, what time period applied to safety monitoring, how events were defined or adjudicated, or whether events were considered related to an intervention. The serious adverse event terms list is empty. Readers should therefore interpret the figures as limited event reporting, not as a comprehensive assessment of safety.
16. What the Result Does, and Does Not, Mean
The posted results describe a randomized feasibility study comparing an active Pillsy reminder platform with an inactive cap and service as usual. For the primary pharmacy-based measure, the mean was 0.839 in the Pillsy arm and 0.752 in service as usual. The unadjusted mean difference was 0.087, and the 95% confidence interval ranged from -0.331 to 0.505. This indicates that the estimate is uncertain and does not sharply distinguish among possible directions or sizes of the group difference.
The MPR is based on pharmacy fills and days' supply. It is a measure of medication availability over the defined period, not direct evidence that a participant took each dose. Nor does it measure clinical outcomes or explain what led to a participant's pattern of medication supply. The results therefore do not establish a clinical effect of the platform or identify the mechanism behind any difference in MPR.
The study is described as a phase I feasibility investigation, and its summary emphasizes tracking adherence and participant satisfaction. A completed study with 41 enrolled participants can provide descriptive information about the measured outcomes in this study, but its small group counts and wide primary interval limit precision. The registry record does not report the original statistical analysis plan, a sponsor-reported inferential analysis, or the analysis population. These gaps further constrain interpretation.
The secondary measures cover engagement, satisfaction, and substance-use-related outcomes. They may add context about the intervention, but they are not a replacement for the primary endpoint and are not reported with a multiplicity strategy. Their unadjusted comparisons should be regarded as exploratory. The absence of reported serious or other adverse events is also not proof that no risks exist, particularly without details about monitoring and ascertainment.
In short, the record reports a higher observed mean MPR in the Pillsy arm, but the estimate is imprecise and its interval includes differences in both directions. The evidence in the registry does not establish a definitive treatment effect, equivalence between groups, or a conclusion about clinical benefit. Those questions require evidence and analyses beyond the descriptive registry summaries.
17. Why This Trial Matters Statistically
This study is statistically interesting because it evaluates a technology intended to generate adherence reminders using a pharmacy-based outcome and measures of participant engagement and satisfaction. Those measures address related but distinct aspects of feasibility: a cap can record bottle openings, an application can deliver messages, and pharmacy fills can indicate medication availability, but none is identical to confirmed medication ingestion. Keeping those constructs separate is important when interpreting digital-intervention studies.
The randomized parallel-group design offers a direct comparison between an active reminder system and an inactive cap with service as usual. At the same time, the recorded group sizes are small, and the primary estimate has a wide interval. This illustrates why the point estimate alone should not be treated as a stable summary of the effect. The uncertainty around it is substantial, and the registry does not include a sponsor-reported analysis to clarify whether any planned adjustments or alternative methods were used.
The record also demonstrates why feasibility and effectiveness questions should not be conflated. Engagement and satisfaction can inform whether an intervention is usable or acceptable, while MPR concerns pharmacy supply and substance-use measures concern other outcomes. A pattern in one measure does not automatically establish a pattern in another. Multiple outcomes also make prespecification and transparent multiplicity handling important, yet these procedures are not reported.
Finally, the sample-size illustrations make clear that planning based on an observed effect can produce very different requirements depending on the effect targeted. Such illustrations can help readers understand precision needs, but they do not recover the study's original design rationale. The original sample-size justification is not reported. The main statistical lesson is to separate descriptive feasibility signals from precise, confirmatory evidence and to interpret each outcome according to what it actually measures.
18. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
The primary limitation apparent from the registry results is the small number of participants contributing to the arm summaries: 19 in the Pillsy arm and 22 in service as usual. The primary confidence interval is broad, so the estimated difference is not precise. The record does not report a threshold for a meaningful MPR difference, preventing interpretation against a stated target of practical importance.
The primary measure is based on pharmacy fills and days' supply, and the time period can vary according to prescription length. The record does not explain how prescription-length variation was handled in the analysis beyond its endpoint definition. MPR also cannot confirm ingestion. The registry does not report whether bottle-opening information was incorporated into the primary outcome; it defines the primary measure as based on pharmacy fills.
Although the study is described as randomized and double-masked, details of sequence generation, concealment, masking roles, and maintenance of masking are not reported. The registry does not describe participant characteristics in detail, recruitment procedures, or the representativeness of the enrolled group. It also does not report the original sample-size rationale, analysis populations, missing-data procedures, or the planned statistical methods.
Only some of the listed secondary outcomes have numerical arm summaries in the results, and no reported analyses are listed. There is no reported multiplicity strategy. The safety record contains zero affected participants in the posted categories, but adverse-event ascertainment and monitoring details are not reported. These limitations arise from the information in the registry record and do not establish that the unreported procedures were absent from the conduct of the study.
19. Related Tutorials
20. Related Statistical Calculators
21. Sources
- ClinicalTrials.gov: Mobile Medication Adherence Platform for Buprenorphine-Naloxone During Treatment of Opioid Use Disorder: Phase I Study — NCT04656899.
- Registry record last updated 2023-03-14; status last verified 2023-02.
- Results first posted 2023-02-10.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
22. Record Summary
NCT04656899 is a completed randomized, double-masked, parallel-group study sponsored by Washington State University. The title and study summary describe a phase I feasibility investigation of a connected pharmacy platform for patients prescribed buprenorphine-naloxone for opioid-use disorder. Enrollment was 41. The Pillsy arm used an active smart cap and reminder features; service as usual used an inactive cap that tracked openings without reminders.
The primary endpoint was mean MPR over 12 weeks, based on pharmacy fills and days' supply. The posted means were 0.839 for Pillsy and 0.752 for service as usual. The unadjusted mean difference was 0.087, with a 95% confidence interval from -0.331 to 0.505 and an unadjusted recalculated p-value of 0.683. The registry lists six secondary outcomes, but numerical summaries are posted for only some. No reported analyses are listed. The record does not report the original sample-size justification, statistical plan, analysis populations, or missing-data procedures.