Optimizing Behavioral Health Homes for Adults With Serious Mental Illness: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Optimizing Behavioral Health Homes for Adults With Serious Mental Illness (NCT02318797), a randomized na study. The primary outcome is Change in Patient Activation in Care (PAM, a 13-item Scale).

2. Clinical Question

3. Trial Design

The registry describes a randomized, parallel, single-masked trial with a cluster-randomized design. It lists an enrollment of 1229 and a completed status. The listed phase is NA. The sponsor is the University of Pittsburgh. The registry gives a start date of 2013-10 and a primary completion date of 2016-10.

Participants were assigned to Patient Self-Directed Care or Provider-Supported Integrated Care. The first approach emphasized patient self-management and peer support, including a web portal; the second involved nurse interactions during clinic visits. Both are categorized as behavioral interventions. The registry does not report the clusters’ identities, the unit of randomization in more detail, the allocation ratio, or the number of clusters. It also does not report which individuals were masked, or how masking was maintained.

The primary activation measure was scheduled at baseline and every 6 months over a 2 year active intervention period. The detailed measure description focuses on the Patient Activation Measure. The trial summary also identifies health status and engagement in primary or specialty care as primary patient-centered outcomes, but numerical results for those outcomes are not included in the posted results.

The registry lists 17 outcome measures and 32 analyses. It also lists secondary outcomes including hope, quality of life, diabetes medication adherence, functional status, emergent care use, laboratory monitoring, patient satisfaction, and antipsychotic medication adherence. The record does not report the full statistical analysis plan or provide enough detail to reconstruct the cluster-level design and analysis.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: Includes those who completed the measure at baseline and whose measure was able to be scored (ie. was not missing data which would have rendered it unusable in analysis)

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in Patient Activation in Care (PAM, a 13-item Scale), measured over Baseline and every 6 months over 2 year active intervention period.

The detailed primary measure is change in patient activation in care, assessed with the 13-item Patient Activation Measure (PAM). It is intended to assess patients’ knowledge, skills, and confidence for managing their health and health care. The registry describes progressively higher activation levels, from starting to take a role through maintaining health-related behaviors. Higher values are described as better outcomes.

The record describes both a raw score scale ranging from 13 to 52 and an activation scale ranging from 0 to 100. The measure was scheduled at baseline and every 6 months over a 2 year active intervention period. The analysis population is described as participants who completed the measure at baseline and had data that could be scored. The number of participants contributing to the primary analysis, and the handling of measurements missing at later times, are not reported in the registry record.

Activation is relevant to the trial’s stated focus on patient-centered outcomes because it captures capacities involved in managing health and care. The trial summary identifies patient activation, health status, and engagement in primary or specialty care as primary outcomes. However, the posted numerical results described in the registry concern the PAM measure; they do not establish the magnitude or direction of change in activation, or provide numerical results for the other two outcomes.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The registry identifies the primary outcome as continuous and reports its summary measure as a mean, with standard deviation as the dispersion measure. It describes the two posted analyses as mixed-model analyses. The numerical results include p-values but no reported estimates or confidence intervals. The following notes explain the statistical ideas in general; they do not add details about the trial’s unreported model specification.

What does a mixed model do?

In general, a mixed model can account for data that have a grouped or repeated-measures structure by representing some sources of variation with fixed effects and others with random effects. In a study with repeated measurements, measurements from the same participant may be related rather than independent. In a cluster-randomized study, participants in the same cluster may also share characteristics or care context. Accounting for relevant dependence can affect estimated uncertainty and statistical tests.

The registry’s description says that this trial was cluster randomized and that the sponsor used mixed models. The specific model terms, covariance structure, cluster adjustment, and assumptions are not reported in the registry record. It is therefore not possible to assess from the listed details exactly how the analysis represented clustering or repeated measurements.

What is a p-value?

In general, a p-value describes how unusual data at least as inconsistent with a null hypothesis as the observed data would be, if that null hypothesis and the statistical model were correct. A small p-value can be evidence against the null hypothesis. It does not give the probability that the null hypothesis is true, and it does not quantify the size or practical importance of an effect.

The two reported p-values are < 0.001 and 0.002. The registry labels the analyses as superiority analyses, but does not report which specific time points or contrasts each p-value represents. Without the estimated contrasts and their confidence intervals, statistical evidence cannot be translated into a direction, magnitude, or precision statement about the difference between the interventions.

Why does the cluster design matter?

In general, cluster randomization assigns groups rather than treating every participant as an independently randomized unit. Outcomes from participants in the same group can be correlated. If that correlation is ignored, a conventional individual-level analysis may understate or otherwise misrepresent uncertainty. The registry specifically cautions that a simple individual-level re-analysis would be misleading, so only the sponsor’s analysis is shown. Details about cluster identities, cluster counts, and the model’s handling of within-cluster correlation are not reported in the registry record.

What does a confidence interval add?

In general, a confidence interval gives a range of values compatible with the data and the assumptions of the estimation method, and communicates uncertainty around an estimated effect. A confidence interval is not reported for either listed mixed-model result. The absence of an interval, together with the absence of a model estimate, prevents assessment of the estimated contrast’s precision and plausible range from the posted results.

How should planning illustrations be read?

In general, sample-size planning calculations describe what a future study might require under stated assumptions; they are not a measure of how much power the completed study achieved. The stated planning assumptions use a two-sided significance level of 0.05, equal allocation between arms, a normal approximation, and an effect defined as a fraction of the observed effect. The record says that non-time-to-event illustrations express participants per arm, while time-to-event illustrations express total events. These are planning conventions, not observed trial outcomes. The values for those illustrations are not reported in the registry record, and no observed-power claim follows from them.

10. Analyses Reported by the Sponsor

The registry identifies the posted p-values as results from the sponsor’s mixed-model analyses. It also states that the study was cluster randomized and that a simple individual-level re-analysis would be misleading. For that reason, no unadjusted re-calculation is presented alongside the sponsor’s results. The sponsor’s analysis is therefore the only inferential analysis described in the posted results.

The registry reports p-values of < 0.001 and 0.002, but does not report the corresponding model estimates, confidence intervals, or the precise contrasts represented by each value. These omissions limit what can be said about effect size, direction, and precision. The sponsor’s analysis can be described as showing statistical evidence in its specified mixed-model tests, but the p-values alone do not establish whether either approach had higher activation or by how much.

The registry does not report enough model detail to independently reproduce the analysis or evaluate all its assumptions. This does not justify replacing the sponsor’s cluster-aware analysis with a simpler individual-level calculation. Instead, interpretation should retain the reported design constraint and distinguish the posted inferential evidence from effect estimates and uncertainty measures that are not reported.

11. Multiplicity, Missing Data and Analysis Populations

The registry lists 17 outcome measures and 32 analyses. The trial summary describes three primary patient-centered outcomes: patient activation, health status, and engagement in primary or specialty care. The detailed primary measure is change in PAM activation, while the reported results include two mixed-model p-values. The registry record does not state how the two p-values map to particular contrasts or time points, or how the 32 analyses relate to the listed outcomes.

When multiple outcomes, time points, or analyses are tested, the chance of observing at least one small p-value can increase. In general, a statistical plan may address this through a prespecified hierarchy, an adjustment for multiple comparisons, or a clear distinction between confirmatory and exploratory analyses. The multiplicity strategy, if any, is not reported in the registry record. The posted p-values should therefore be read as individual reported test results, not assumed to be adjusted for the full set of analyses.

The analysis population for the PAM measure is described as participants with a completed, scoreable baseline measure. The number included is not reported in the registry record. The record does not give the amount or pattern of missing data at follow-up, explain how missing measurements were handled in the mixed models, or describe any sensitivity analyses for missing data. It also does not state whether the analysis followed an intention-to-treat principle or another population definition beyond the baseline measure criterion.

These omissions affect interpretation, rather than proving that the analysis was inappropriate. Readers cannot determine from the registry description whether missingness differed between groups, how assumptions about missing outcomes were handled, or whether the analyses were prespecified. The listed p-values should be considered together with those limits and with the absence of outcome estimates and confidence intervals.

12. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: death - not associated with study activities (0.8%).

The registry reports serious adverse events affecting 3 participants in Patient Self-Directed Care and 6 participants in Provider-Supported Integrated Care. It reports 0 other adverse events in each group. Deaths are listed as 3 and 6, respectively. The registry also lists “Death - not associated with study activities” as a serious-event term with a reported value of 0.8%.

These figures describe the safety entries in the registry; they do not establish that an intervention caused an event. The listed deaths are described in the serious-event information, while the specific serious-event term says the deaths were not associated with study activities. The record does not provide event-level narratives, timing, adjudication details, or the denominator used for the 0.8% entry. It also does not report comparative safety estimates or uncertainty intervals.

Counts alone do not show whether groups differed in exposure time, follow-up completeness, or other factors relevant to interpreting adverse events. The reported at-risk counts are 516 for Patient Self-Directed Care and 713 for Provider-Supported Integrated Care, but the registry does not describe how those counts relate to event follow-up or analysis populations. These safety data should be read as registry-reported counts, not as proof of causation or as a complete evaluation of comparative safety.

13. What the Result Does, and Does Not, Mean

The posted results support a narrow conclusion: the registry reports two statistically significant mixed-model tests for the randomized comparison, with p-values of < 0.001 and 0.002. The outcome detail identifies patient activation measured with the PAM, and the trial summary frames activation as one of three patient-centered outcomes. These p-values indicate evidence against the null hypotheses tested in the sponsor’s models, subject to the models’ assumptions.

They do not show the amount of change in activation, which group had higher scores, or how precise the estimated differences were. Neither model estimates nor confidence intervals are reported. The p-values also do not establish that any difference was meaningful to participants. The registry does not map each p-value to a named time point or contrast, which further limits outcome-specific interpretation.

The results should not be taken as a complete report of all the outcomes named in the trial summary. Numerical results for health status and engagement in primary or specialty care are not reported in the results described. The record lists additional secondary outcomes, but their estimates are not included in the registry record for interpretation.

The result also does not establish a causal explanation for any difference beyond the trial’s randomized comparison and the sponsor’s analysis. The cluster-randomized design is important because a simple individual-level re-analysis could misrepresent uncertainty. The record does not report enough detail to independently reproduce the cluster-aware model.

Accordingly, the evidence is best described as reported statistical evidence from mixed-model analyses, without the effect estimates needed to judge direction, magnitude, or precision. It should not be summarized as proof that one approach works better, or that either approach does not work. Such conclusions would require information not reported in the registry record, including estimates, confidence intervals, outcome-specific results, and fuller analysis details.

14. Why This Trial Matters Statistically

This trial raises a central issue in clinical research: patient-centered outcomes can be central to a study’s purpose while still requiring careful statistical interpretation. The registry describes activation, health status, and engagement in care as outcomes of interest, and defines activation in terms of knowledge, skills, and confidence for managing health and health care. That makes the interpretation of the measurement and its changes important, not just the reported test results.

It also illustrates why trial design and analysis cannot be separated. The study is described as cluster randomized, and the registry warns that a simple individual-level re-analysis would be misleading. In general, clustering can create dependence among observations, so analysis must reflect the assignment structure when estimating uncertainty. The posted mixed-model p-values must be understood in that design context.

Another statistical lesson is the difference between evidence against a null hypothesis and a useful description of an effect. The reported p-values are small, but the registry does not report estimates or confidence intervals. Without those, readers cannot judge the direction, magnitude, or precision of the activation contrast. Statistical significance is not a substitute for an interpretable effect estimate.

Finally, the record lists many outcomes and analyses but does not report a multiplicity strategy or a full statistical plan. That does not determine whether the analyses were appropriate, but it leaves uncertainty about how the reported tests fit into the wider analysis program. This example shows why transparent reporting of design, estimands, model details, missing-data handling, and effect estimates is important for assessing clinical trial evidence.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry description leaves several trial-specific questions unanswered. It does not report which clusters were randomized, how many clusters there were, the allocation ratio, or how cluster membership was represented in the mixed models. The masking category is listed as single, but the identity of the masked individuals and the masking procedures are not reported in the registry record.

For the primary activation outcome, the record gives the measurement schedule and score descriptions but not the group-specific means, changes, model estimates, or confidence intervals. It reports two p-values without identifying the corresponding contrasts or time points. The results for health status and engagement in care, although named in the trial summary as primary outcomes, are not reported in the results described. These gaps prevent a full comparison of the interventions across the trial’s stated primary outcomes.

The record lists 17 outcome measures and 32 analyses but does not report a complete statistical analysis plan, a multiplicity approach, or which analyses were prespecified. Missing-data amounts and patterns, follow-up analysis populations, and sensitivity analyses are also not reported. The PAM analysis population is described using a baseline completion and scoreability criterion, but its size is not given.

The intervention descriptions are broad. The registry does not report detailed content, frequency, duration, fidelity, or participant exposure for either approach. The safety entries provide counts but not event narratives or the denominator for the listed 0.8% serious-event term. Finally, because the design is cluster randomized and the record cautions against simple individual-level re-analysis, the sponsor’s mixed-model results cannot be independently checked using the information reported.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Optimizing Behavioral Health Homes for Adults With Serious Mental Illness — NCT02318797.
• Registry record last updated 2018-02-22; status last verified 2018-02.
• Results first posted 2018-02-22.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

NCT02318797, “Optimizing Behavioral Health Homes for Adults With Serious Mental Illness,” is listed as a completed, randomized, parallel, single-masked, cluster-randomized study sponsored by the University of Pittsburgh. The registry lists enrollment of 1229 and phase NA.

The study compared Patient Self-Directed Care, which involved self-management strategies and peer support, with Provider-Supported Integrated Care, which involved nurse interactions during clinic visits. The detailed primary outcome was change in patient activation measured with the 13-item PAM, scheduled at baseline and every 6 months over a 2 year active intervention period.

The registry reports two mixed-model p-values, < 0.001 and 0.002, but does not report the associated estimates or confidence intervals. It also lists serious adverse events affecting 3 and 6 participants in the respective groups, and 0 other adverse events in each. The trial’s effect direction, magnitude, and precision cannot be determined from the reported p-values alone.