Evaluation of Peer Specialists on VA PACTs: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Evaluation of Peer Specialists on VA PACTs (NCT02732600), a randomized na study. The primary outcome is Patient Activation Measure Change.

2. Clinical Question

The project evaluates support for implementing a new team role, rather than comparing two clinical treatments. In the registry description, Peer Specialists are Veterans recovered from mental illness who are trained to support other Veterans with mental illness. The project grew from an Executive Action mandating that 25 VA medical centers place Peer Specialists on Primary Care Teams. Its stated premise is that outside aid and facilitation may improve the success of adding staff to existing teams.

The comparison is between implementation approaches: standard guidance and limited consultation versus a year of structured, ongoing support. The primary outcome asks whether PAM scores differ between those conditions. PAM measures an individual’s perceived ability to manage illness and health behaviors and act as an effective patient. The registry record reports three mean-difference results, but does not explicitly link each result to a particular assessment time or state the subtraction order for the coded groups.

3. Trial Design

This is a randomized, parallel-group project recorded as completed. The registry identifies the design as cluster-randomized, with teams or sites as the allocation units rather than individual patients. It reports enrollment of 5616. The registry record does not state how many clusters were allocated to each condition, the number of participants in each arm, or the participant flow through follow-up.

The two conditions are Standard Implementation and Facilitated Implementation. Standard Implementation sites received written guidance and limited consultation from the investigators’ team. Facilitated Implementation sites received one year of support based on the i-PARIHS implementation model, including training, implementation planning, ongoing external facilitation, feedback and consultation. The registry lists the intervention as an implementation intervention, not a drug or device.

The project is described as a quality improvement evaluation. Its brief summary says the evaluation concerns whether expanded support for half of the Primary Care Teams leads to better outcomes than teams without extra support. The stated enrollment is 5616. The record lists a start date of 2016-01-01, primary completion of 2019-03-31 and status as completed. It lists the sponsor as the VA Office of Research and Development.

The primary outcome is Patient Activation Measure Change, described as a comparison across the two conditions on PAM Total Score. Its stated assessment timeframe is baseline, 6 months and 1 year. The registry also lists nine outcome measures and 49 analyses. Secondary outcomes include measures of team development, peer fidelity, satisfaction, Veterans seen, visits, services and time to first service. The registry record does not provide a detailed allocation procedure, cluster count, or fuller account of the analysis populations.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Comparison across two conditions on PAM Total Score - PAM Total Score (0-100)

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: baseline characteristics only completed for Veterans completing surveys

6. Primary Endpoint

The registry lists the primary outcome as Patient Activation Measure Change, measured over Baseline, 6 months, 1 year.

The primary endpoint is the Patient Activation Measure (PAM), a 13-item survey of a person’s perceived ability to manage illness and health behaviors and to act as an effective patient. The registry describes PAM as reliable, valid and sensitive to change, and reports that it correlates with measures of improved self-management. Scores range from Zero to One Hundred, with higher values indicating higher patient activation, which the registry describes as a better outcome.

The endpoint is relevant to this implementation evaluation because the intervention is intended to support Peer Specialists working within primary care teams. PAM captures patients’ perceptions of their capacity to manage health, rather than a clinical event or a count of services. A score can therefore be used to examine whether patient activation differs across implementation conditions. That role does not, by itself, establish that a difference in score was caused by the implementation strategy.

The stated timeframe is baseline, 6 months and 1 year. The posted results are described as mean differences in final values, while the primary outcome title refers to change. The record does not explain how the final-value comparisons relate to the titled change outcome, whether baseline values were incorporated into the analysis, or which reported result corresponds to each assessment time. Those details matter when interpreting the estimate as a change over time rather than a comparison of final scores.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What is being compared? The posted primary analyses compare mean final PAM values between coded groups OG000 and OG001. The registry describes the method as a two-sided t-test and the parameter as a mean difference in final values. The two-sided test evaluates departures in either direction rather than only a prespecified increase or decrease. The record does not identify the subtraction order for the coded groups, so the signs of the mean differences cannot be translated into a claim that one named implementation condition had higher scores.

General: A mean difference expresses the gap between group averages on the outcome’s scale. A confidence interval gives a range of values that are compatible with the estimate under the statistical method and its assumptions. A confidence interval crossing zero means that a zero difference is among the values compatible with that interval; it does not prove that the true difference is zero. A p-value describes how unusual data at least as discrepant as those observed would be under a specified null model. It is not the probability that the null hypothesis is true, and it does not measure the importance of a difference.

The posted confidence intervals are 95% intervals. Each includes zero, and the corresponding p-values are 0.39, 0.35 and 0.21. These values should be read alongside the intervals rather than as a stand-alone declaration. In particular, a result that does not cross a conventional significance threshold is not the same as evidence that two implementation conditions are equivalent. Equivalence would require a defined margin and an analysis designed to assess it; such a margin or analysis is not reported in the registry record.

Why does clustering matter? The project is cluster-randomized. In a cluster design, units such as teams or sites are allocated together, and participants within the same unit may have more similar outcomes than participants from different units. An analysis that treats every participant as independently randomized can understate uncertainty if it fails to account for that similarity. The registry’s note says a simple individual-level re-analysis would be misleading and that only the sponsor’s own analysis is shown. The record does not describe the specific adjustment or model used to account for clustering, so that aspect of the method cannot be independently assessed from the reported details.

General: When outcomes within a cluster are correlated, the effective information in the data may be less than an analysis treating all observations as independent would suggest. Accounting for clustering is therefore important for standard errors, confidence intervals and tests. The size of this effect depends on features such as the within-cluster correlation and cluster sizes, which are not reported in the registry record. No new individual-level calculation should be substituted for the sponsor’s analysis in this setting.

The registry lists baseline, 6 months and 1 year as the primary outcome timeframe, but the three posted analyses are all labeled as mean differences in final values. It does not map the comparisons to time points or describe whether baseline adjustment was used. The record also does not report whether the analysis used all available measurements, a particular repeated-measures model, or another approach to account for measurements over time. Consequently, the displayed estimates should not be interpreted as a specific longitudinal model or as baseline-adjusted changes without further documentation.

What about planning calculations? A planning illustration is described as using the observed effect, a two-sided significance level of 0.05, equal allocation between arms and a normal approximation. It is an illustration of participants per arm for a stated fraction of the observed effect, not an account of the trial’s original design rationale. Such calculations are not observed power and do not establish the probability that the study detected an effect. The registry record does not report the trial’s sample-size justification, and the planning illustration should not be treated as evidence about how the original enrollment target was selected.

General: Sample-size planning depends on the effect a future study aims to detect, the variability of the outcome, the significance level and the desired power, among other design features. Planning based on an observed effect can be unstable because that estimate is itself uncertain. An illustration can help show how assumptions influence a future study’s size, but it does not repair uncertainty in the existing result or change the interpretation of its confidence interval.

The record lists nine outcome measures and 49 analyses. It does not report a statistical analysis plan, whether the primary comparison was adjusted for multiple testing, or how secondary outcomes were incorporated into any inferential framework. Multiple analyses can make isolated nominal p-values harder to interpret as confirmatory evidence, particularly when the testing strategy is not described. Since the posted primary p-values are not below 0.05, no claim of a positive primary finding follows from them; still, the absence of multiplicity details limits a fuller appraisal of the analysis strategy.

10. Analyses Reported by the Sponsor

The registry identifies the posted primary comparisons as the sponsor’s own analyses: two-sided t-tests of mean differences in final values. It also states that the trial is cluster-randomized and that a simple individual-level re-analysis would be misleading. Accordingly, no individual-level recalculation is presented as a substitute for the sponsor’s results. The reported estimates, confidence intervals and p-values should be read as the sponsor’s analysis, with the limitations in reporting noted separately.

The three estimates are 0.51, 0.65 and 1.01, with 95% confidence intervals of -0.66 to 1.68, -0.75 to 2.06, and -0.57 to 2.60, respectively. Their p-values are 0.39, 0.35 and 0.21. All intervals include zero. The registry record does not say which coded group is subtracted from the other, or associate each comparison with a particular assessment time. It also does not explain how the reported final-value comparisons correspond to the outcome title, Patient Activation Measure Change.

The record does not provide enough detail to reconstruct or verify the sponsor’s cluster-aware analysis. Its model specification, handling of missing observations and analysis populations are not reported in the registry record. Those omissions are reasons for caution in evaluating the analysis method, not a basis for replacing its reported results with a calculation that ignores the randomized clusters.

11. Multiplicity, Missing Data and Analysis Populations

The registry lists nine outcome measures and 49 analyses, but it does not report a multiplicity strategy for the primary or secondary analyses. It is therefore unclear whether any adjustment for multiple comparisons was planned or performed, whether the primary comparison was designated within a broader testing hierarchy, or how the many analyses relate to confirmatory conclusions. When many outcomes or analyses are considered, isolated p-values can be more difficult to interpret as evidence unless the testing strategy is specified. The posted primary p-values should be understood in that context, without assuming an adjustment that the record does not describe.

Missing-data methods are not reported in the registry record. The record does not state how many PAM observations were available at baseline, 6 months or 1 year, how missing follow-up values were handled, or whether participants with incomplete measurements were included in the analysis. It also does not report whether an imputation method, a complete-case approach or a repeated-measures method was used. Missingness can affect both the estimates and their uncertainty, so these omissions limit assessment of the results.

Analysis populations are also not described in enough detail to determine which participants contributed to the posted comparisons. The registry reports enrollment of 5616, but does not give arm-specific participant counts or explain exclusions from the analyses. Because allocation was by cluster, the record would also need to describe which clusters and participants were included and how clustering was represented in the analysis to support a fuller appraisal.

The primary outcome timeframe includes baseline, 6 months and 1 year, but the record does not link the three reported comparisons to those specific assessments. It does not state whether the analysis compared follow-up values, changes from baseline, or another summary. Nor does it report an interim-analysis plan or a complete statistical analysis plan. These details are not reported in the registry record, so they should not be inferred from the available estimates.

12. What the Result Does, and Does Not, Mean

The posted estimates describe comparisons of mean final PAM values between two coded groups under the sponsor’s reported analysis. Their confidence intervals include zero, and their p-values are 0.39, 0.35 and 0.21. Thus, the reported analyses do not establish a statistically clear difference in final PAM values between the groups. The intervals retain uncertainty about the direction and magnitude of a possible difference.

This result is not proof that the implementation approaches have identical effects. A non-significant test does not demonstrate equivalence, and the registry does not report a threshold for a meaningful PAM difference or an equivalence analysis. The estimates cannot be classified as clinically important or unimportant using a criterion that is not reported. Their precision should be judged from the confidence intervals, which include both zero and values on either side of it.

The outcome measures patient activation as perceived ability to manage illness and health behaviors. It is not a direct measure of whether Peer Specialists were successfully integrated, whether teams followed implementation procedures, or whether all aspects of patient care changed. The registry lists secondary measures related to team development, peer fidelity, satisfaction, service use and timing, but the registry entries available for these results do not include their estimates. Those outcomes therefore cannot be used to expand the conclusion about PAM.

The project is cluster-randomized, and the registry cautions against a simple participant-level re-analysis. Interpretation rests on the sponsor’s reported comparisons, while the record leaves important methods unclear, including the handling of clustering, missing data and repeated assessments. The reported endpoint title refers to change, but the analysis labels refer to final values. Without clarification of that distinction and the time-point mapping, the findings should be described narrowly as the posted final-value comparisons, not as definitive evidence about change over time or the overall success of implementation.

13. Why This Trial Matters Statistically

This evaluation illustrates a central statistical feature of implementation research: the intervention is delivered to teams or sites, while outcomes may be measured for individual patients. That cluster-randomized structure affects how uncertainty should be estimated. Treating participants as if each were independently assigned could give misleading results, a concern explicitly noted in the registry. A suitable interpretation therefore depends on the sponsor’s cluster-aware analysis rather than a simple calculation from participant-level summaries.

The trial also connects an implementation strategy to a patient-reported measure. Facilitated Implementation included training, planning, ongoing facilitation, feedback and consultation, while Standard Implementation involved written guidance and limited consultation. PAM addresses patients’ perceived ability to manage health. That makes the question broader than whether the support package was delivered: the proposed comparison concerns whether implementation conditions correspond to differences in patient activation.

The results demonstrate why point estimates, confidence intervals and design details need to be read together. The mean differences alone do not show the range of uncertainty; the confidence intervals include zero and extend in both directions. At the same time, the registry leaves the time-point mapping, group-code direction and relationship between the change endpoint title and final-value analysis unclear. Those reporting gaps constrain what readers can conclude even when estimates and tests are available.

Finally, the registry records a substantial evaluation, with enrollment of 5616 and 49 analyses listed, but does not report a full statistical analysis plan, multiplicity handling, or missing-data methods. This is a reminder that a completed registry record can document outcomes and results without giving enough information for every aspect of statistical appraisal. The central lesson is to keep conclusions aligned with the defined endpoint, the cluster allocation and the precision of the posted estimates.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to the information reported for this evaluation. The project is cluster-randomized, but the registry does not state the number of teams or sites allocated to either condition, their sizes, or the method used to account for clustering in the analysis. Those details are important for judging the precision of the sponsor’s estimates. The record itself cautions that a simple individual-level re-analysis would be misleading, and no such calculation should be treated as an alternative result.

The primary outcome is titled Patient Activation Measure Change, yet the reported analyses are labeled mean differences in final values. The record does not explain whether these are change scores, follow-up comparisons adjusted for baseline, or unadjusted final-value comparisons. It also does not identify which of the three results corresponds to baseline, 6 months or 1 year. The direction of subtraction between the coded groups is not stated in the analysis details.

Other relevant information is not reported in the registry record: arm-specific participant counts, participant flow, missing-data handling, analysis populations, a multiplicity plan, and a full statistical analysis plan. Although nine outcome measures and 49 analyses are listed, the results available for this evaluation include only the primary comparisons. Findings for the listed secondary outcomes cannot be inferred from their names.

The registry does not report a threshold for a clinically meaningful PAM difference, so the practical importance of the estimates cannot be judged against a stated criterion. It also does not report the original sample-size justification. The separate planning illustration is based on the observed effect and is not a description of the original design. These limitations call for restrained interpretation rather than assumptions about unreported design or analysis choices.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Evaluation of Peer Specialists on VA PACTs — NCT02732600.
• Registry record last updated 2023-07-27; status last verified 2023-07.
• Results first posted 2020-06-12.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT02732600, titled Evaluation of Peer Specialists on VA PACTs, is a completed, randomized, parallel-group, cluster-randomized quality improvement project sponsored by the VA Office of Research and Development. The registry reports enrollment of 5616 and compares Standard Implementation with Facilitated Implementation. The latter involved one year of support based on the i-PARIHS implementation model.

The primary outcome is Patient Activation Measure Change, with a stated timeframe of baseline, 6 months and 1 year. The three posted two-sided t-test comparisons of mean final values have estimates of 0.51, 0.65 and 1.01; all three 95% confidence intervals include zero. Their p-values are 0.39, 0.35 and 0.21. The registry does not map these comparisons to time points or specify group subtraction order.

The record lists nine outcome measures and 49 analyses. It does not report key details including multiplicity handling, missing-data methods, analysis populations or the full approach to clustering. Interpretation is therefore limited to the sponsor’s posted comparisons and the uncertainty reflected in their intervals.