Sustaining Patient-centered Alcohol-related Care: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Sustaining Patient-centered Alcohol-related Care (NCT02675777), a randomized na study. The primary outcome is Brief Alcohol Counseling Rate.

2. Clinical Question

3. Trial Design

The registry record identifies the study as randomized, unmasked, and crossover, with an enrollment of 19. The study summary, however, describes a pragmatic stepped-wedge quality-improvement trial. These are differing design descriptions, and the registry record does not explain how they relate or provide enough detail to reconstruct the allocation and timing of clinic transitions. The design should therefore be understood from the recorded descriptions with that uncertainty in mind, rather than by assuming a particular sequence or crossover structure.

The project aimed to integrate evidence-based alcohol-related care into primary care clinics. Its background refers to 25 primary clinics in the health system, while the objective describes implementation in 22 clinics. The summary also describes a pilot phase in 3 clinics. The registry record does not reconcile these clinic counts or specify which clinics contributed to the posted primary outcome analysis.

The intervention was a 4-month active implementation period supported by a practice facilitator. Usual care was described as including passive access to electronic health record tools and 2 months of preparation in each clinic before active implementation. The main comparison concerns rates before and after “time one,” the start of active implementation. The registry record does not report more detailed timing for each clinic or explain the statistical handling of any staggered implementation.

The study is marked completed. Its recorded start was 2015-01 and its primary completion was 2018-08. The sponsor is Kaiser Permanente. The phase is listed as NA. Although the summary discusses both preventive care and treatment for alcohol use disorders, the primary outcome analyzed is documented brief alcohol counseling. Alcohol screening rate and alcohol use disorder assessment rate are listed as secondary outcomes.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: In this stepped wedge study, 19 sites were randomly assigned to 7 waves, which determined the time period in which they would implement the Quality Improvement Intervention. This study used an open cohort design, in which patients were analyzed during the time periods in which they had visits to a particating site. Therefore, patients could visit the site in one or both periods (i.e., before and/or after the Quality Improvement Intervention began), and also could visit sites in multiple waves.

Only the first 6 of 7 groups are shown.

Usual Care

Quality Improvement Intervention

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Patients could have a visit in both periods (total number of patients with a visit = 333,596; number of patients with visits in both periods = 150,451).

6. Primary Endpoint

The registry lists the primary outcome as Brief Alcohol Counseling Rate, measured over Rates of documented brief alcohol counseling within 14 days after a positive alcohol screen will be compared before and after "time one" (T1: the start of the 4 months of active implementation) for the pragmatic stepped-wedge trial..

The primary endpoint is the brief alcohol counseling rate. Its unit is participants per 10,000 with visits, and the registry describes it as a count-type measure rather than a single comparison reducible to a simple individual-level estimate. The outcome description focuses on patients with at least one primary care visit who screen positive for unhealthy alcohol use and have brief alcohol counseling documented in the electronic health record.

The registry gives AUDIT-C thresholds of 3 or more points for women and 4 or more for men. It describes counseling documentation within 14 days after a positive screen or in the prior year. The timeframe states that rates of documented counseling within 14 days after a positive screen will be compared before and after “time one,” which is the start of the 4-month active implementation period. Because the outcome description and timeframe use both a 14-day window and a prior-year reference, the registry record does not make their relationship fully clear.

This endpoint is used to assess whether a recorded care process changed in relation to implementation. It captures documentation in the electronic health record, not every counseling conversation or its content. The registry record does not report a patient-level clinical outcome for this endpoint. No underlying rate, effect estimate, or confidence interval is reported with the primary analysis, so the magnitude and precision of any change cannot be assessed from the posted result.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The reported method is a mixed models analysis. General explanation: mixed models can represent outcomes measured in settings where observations are grouped, such as patients within clinics, and can account for dependencies that make observations within a group more alike than observations from different groups. This matters in clinic-level implementation research because the intervention is organized around clinics, not simply assigned to isolated patients. The registry record does not report the model formula, the random or fixed effects used, or the details of how clinic and time were represented.

Why account for clustering?

General explanation: when people receive care in the same clinic, their outcomes may share features of that clinic’s staffing, workflows, documentation, and implementation environment. Treating every patient record as fully independent can make the analysis appear more precise than it is. A model that accounts for grouping can reflect that shared context. The registry record’s note that an individual-level recalculation would be misleading is consistent with the need to respect the cluster-randomized design. The posted analysis is the sponsor’s mixed-model result, rather than a simple recalculation from summary statistics.

What does the p-value say?

The reported p-value is <.001. General explanation: a p-value describes how surprising data at least as incompatible with a specified null hypothesis would be if that hypothesis and the model assumptions were true. It is not the probability that the null hypothesis is true, and it does not measure the size of an effect. The registry record labels the analysis as superiority, but gives no numerical estimate or confidence interval. The p-value therefore signals statistical evidence under the reported model without showing the magnitude, direction, or precision of the difference.

Why is an effect estimate important?

General explanation: an effect estimate describes the amount and direction of a difference, while a confidence interval expresses the uncertainty around that estimate under the analysis. Together, they help distinguish a small but statistically detectable difference from a larger difference and show which values are compatible with the data. Here, the registry record gives neither an effect estimate nor a confidence interval for the primary outcome. The reported p-value cannot substitute for either one. The posted result also does not give the underlying rates needed to describe the observed level of counseling documentation.

What can be inferred from the outcome definition?

The outcome is documentation of brief alcohol counseling in the electronic health record among patients with a primary care visit who screen positive for unhealthy alcohol use. The timeframe says rates within 14 days after a positive screen are compared before and after the start of active implementation. The description also refers to counseling documentation in the prior year. The registry record does not explain how those references are combined in the analysis. The endpoint therefore represents recorded care as defined by the registry, not a direct measure of the content, quality, or effect of a counseling conversation.

General explanation: process outcomes are often useful in implementation research because they measure whether an intended part of care is being delivered or recorded. They do not, on their own, establish a change in health outcomes. In this study, the primary endpoint is a care-process measure, and the record does not report a clinical outcome tied to the primary analysis.

Why not recalculate the result?

The registry record states that a simple individual-level re-analysis would be misleading in this cluster-randomized setting and that only the sponsor’s own analysis is shown. General explanation: recalculation from unadjusted summary statistics may omit the design features and dependencies accounted for in a model. It can produce a result that does not represent the intended analysis, especially when assignment and implementation occur at the clinic level. There are no posted summary values for an independent estimate or confidence interval in any case.

The sample-size assumptions describe planning illustrations based on fractions of the observed effect, with a two-sided significance level of 0.05, equal allocation, and a normal approximation. These illustrations are not the trial’s original design justification and are not observed power. The registry record does not report the original sample-size justification. The illustration assumptions also do not supply a replacement for the missing effect estimate and confidence interval in the reported primary analysis.

10. Analyses Reported by the Sponsor

The sponsor’s own result is a mixed models analysis with a superiority designation and p <.001. The registry record gives no estimated rate difference, confidence interval, or numerical group rates alongside that result. It is therefore not possible to compare the sponsor’s estimate with an independent recalculation or describe how closely such estimates agree.

The record specifically cautions that a simple individual-level re-analysis would be misleading for the cluster-randomized design and says only the sponsor’s own analysis is shown. That is important because the intervention is organized at the clinic level, while the outcome concerns patients receiving care in those clinics. A calculation that ignored that structure would not be an equivalent check of the mixed-model result.

Accordingly, the sponsor’s p-value should be read as the reported model-based evidence, not as an estimate of the amount of change. No recalculated result is available to confirm or challenge it. Without the model specification, effect estimate, confidence interval, and fuller design details, readers cannot independently assess the result’s magnitude or precision. The registry record does not report enough information to reconstruct the sponsor’s analysis.

11. Multiplicity, Missing Data and Analysis Populations

The registry record lists 7 outcome measures and 2 analyses. It identifies brief alcohol counseling rate as the primary outcome and lists alcohol screening rate and alcohol use disorder assessment rate as secondary outcomes. A mixed-model result is reported for the primary outcome, but the record does not describe a multiplicity adjustment or explain how the multiple outcomes and analyses were handled when interpreting statistical evidence. The p-value should therefore be read as reported, without assuming that it was adjusted for multiple testing.

The record does not report how missing outcome data were defined or handled, whether electronic health record data were incomplete, or how patients with absent screening or counseling documentation were treated in the analysis. It does not describe an imputation method or a sensitivity analysis for missing data. The analysis population and the number of clinics and patients contributing to the primary analysis are not reported in the registry record.

The model details are also not reported: the record does not specify the covariates, time terms, clinic-level effects, correlation structure, or estimation method. Nor does it report whether the analysis followed an intention-to-treat principle or another defined population approach. These omissions make it difficult to evaluate the assumptions underlying the p-value or assess robustness to alternative analytic choices.

The primary outcome description and timeframe refer to both a 14-day post-screen period and counseling documented in the prior year. The record does not clarify how these windows relate to one another in the analysis. It also does not provide detailed clinic-level implementation timing, despite the summary’s stepped-wedge description. These are uncertainties in the outcome and design documentation, not evidence that a particular approach was or was not used.

12. What the Result Does, and Does Not, Mean

The posted result shows that the sponsor’s mixed-model comparison of brief alcohol counseling rates before and after the start of active implementation produced p <.001. This indicates statistical evidence against the modeled null hypothesis under the analysis used. It does not give the size or direction of the difference, because no effect estimate or group rates are reported. It also gives no confidence interval with which to judge precision.

The primary outcome concerns counseling documentation in the electronic health record among patients who had a primary care visit and screened positive for unhealthy alcohol use. It is a measure of recorded care. The result does not establish whether counseling occurred but was not documented, what was discussed during counseling, or whether patients experienced a health benefit. The registry record reports no primary clinical outcome in this analysis.

The study summary describes a clinic-level implementation effort involving screening, assessment, treatment, and follow-up for unhealthy alcohol use and alcohol use disorders. The primary analysis, however, concerns brief alcohol counseling documentation. The reported p-value should not be treated as evidence that every element of the broader care approach changed, since the registry does not report those results in detail in the primary analysis.

The registry record’s design descriptions are not fully aligned: its metadata says randomized and crossover, while its summary calls the project a pragmatic stepped-wedge trial. The record also refers to 25 clinics in the background, 22 in the objective, and a pilot in 3 clinics. It does not explain these differences. This limits interpretation of the implementation setting and the exact analysis population.

In general, statistical significance and practical importance are distinct. The result gives evidence under the reported model, but the absent estimate and confidence interval prevent assessment of the magnitude and precision of the change. It should not be interpreted as a quantified improvement or as proof of a clinical effect.

13. Why This Trial Matters Statistically

This study illustrates a statistical challenge common to implementation research: the intervention is organized around clinics, while the outcome is recorded for patients. Patients treated in the same clinic may have related outcomes because they share workflows, staff, documentation systems, and implementation support. Analyses that ignore this grouping can misrepresent uncertainty. The registry record acknowledges this issue by stating that a simple individual-level re-analysis would be misleading.

The primary outcome is a documented care process. Such outcomes can help assess whether an implementation effort is associated with a change in care delivery as recorded in routine systems. At the same time, electronic record documentation is not identical to the underlying conversation or experience of care. Interpreting process outcomes requires attention to how the outcome is defined and recorded.

The record also demonstrates why a p-value alone is an incomplete statistical report. The mixed-model analysis gives p <.001, but the absence of an effect estimate and confidence interval leaves the size, direction, and precision of the result unclear. A statistically small p-value does not replace those quantities.

Finally, the study descriptions raise practical questions about design transparency. The metadata calls the design randomized and crossover, while the summary calls it stepped-wedge; the record also gives different clinic counts for the broader setting, objective, and pilot. These details matter because they shape the units of implementation and analysis. Clear reporting of design, model, outcome windows, and analysis population is essential for readers to interpret results and understand how well an analysis reflects the study structure.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record has important reporting limitations for interpreting this result. It reports a p-value but no numerical effect estimate, confidence interval, or underlying rates for the primary outcome. As a result, the magnitude, direction, and precision of the modeled difference cannot be assessed. The record also does not provide the mixed-model specification or enough information to reproduce the analysis.

The design descriptions are not consistent across registry elements. Metadata identifies a randomized crossover design, whereas the study summary calls the project a pragmatic stepped-wedge quality-improvement trial. The record does not explain the relationship between these labels. It also refers to 25 clinics in the background, 22 clinics in the objective, and a pilot in 3 clinics, without clarifying how these sets relate to the analysis.

The primary outcome description refers to counseling documentation within 14 days after screening or in the prior year, while the timeframe emphasizes the 14-day period. The record does not explain how the time windows were operationalized. The number of patients and clinics contributing to the primary analysis, missing-data methods, analysis population, adjustment variables, and detailed implementation timing are not reported.

The posted analysis is the sponsor’s mixed-model result. The registry cautions against a simple individual-level recalculation, so there is no independent summary-statistic estimate available to check. The record lists zero serious adverse events, other adverse events, and deaths for both periods, but does not provide further safety analysis. These reporting gaps limit the conclusions that can be drawn from the registry result without implying that unreported methods or analyses were absent.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Sustaining Patient-centered Alcohol-related Care — NCT02675777.
• Registry record last updated 2024-12-24; status last verified 2024-11.
• Results first posted 2024-12-24.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

The registry record describes a completed study sponsored by Kaiser Permanente, with a recorded start of 2015-01 and primary completion in 2018-08. It concerns implementation of patient-centered alcohol-related care in primary care clinics. The intervention used a practice facilitator during a 4-month active implementation period; the usual-care description includes passive access to electronic health record tools and 2 months of preparation.

Brief alcohol counseling rate is the primary outcome. Alcohol screening rate and alcohol use disorder assessment rate are listed as secondary outcomes. The primary analysis is reported as a mixed models analysis with a superiority designation and p <.001. No effect estimate or confidence interval is reported. The record identifies a randomized crossover design in its metadata but calls the study stepped-wedge in its summary, and it does not reconcile the differing clinic counts. It also states that an individual-level recalculation would be misleading, leaving the sponsor’s analysis as the reported result.