Coordination Toolkit and Coaching Project: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Coordination Toolkit and Coaching Project (NCT03063294), a randomized na study. The primary outcome is Health Care System Hassles Scale.

2. Clinical Question

The project aimed to disseminate strategies for coordinating care across outpatient settings and to evaluate the added coaching strategy. The Hassles Scale asks about problems patients encounter with their general healthcare, rather than their experience with one specific visit or provider. Its scores are intended to reflect reported hassles related to care coordination. The registry record describes a randomized, parallel project and says participating clinics were randomly assigned to the toolkit strategy or the toolkit-plus-coaching strategy. It does not report a single numerical estimate that answers the comparative question across the stated survey timepoints. Consequently, the record supports describing the intended comparison and outcome, but not a conclusion about the direction or size of a difference between strategies.

3. Trial Design

The Coordination Toolkit and Coaching Project, also titled Improving PACT Coordination Across Settings and Services, is listed as a completed, randomized, parallel project. The registry record lists the sponsor as the VA Office of Research and Development, enrollment as 12, and the condition as coordination, administrative. The project start date is 2017-03-01, and primary completion is listed as 2020-01-31. Phase is listed as NA.

The project description identifies the participating clinics as the units assigned to the strategies: toolkit access alone or toolkit access with distance-based coaching. The registry classifies allocation as randomized and masking as none. It does not report further details about how the random assignment was generated, whether assignments were concealed, or how many clinics were assigned to each strategy. Those details are not reported in the registry record.

The online toolkit was intended to support care coordination for vulnerable patients visiting primary care. The additional coaching strategy paired toolkit access with quality improvement support from a distance-based coach. The project description says the quality improvement approach used locally initiated plan-do-study-act cycles. These features describe the intervention strategies; they do not establish how consistently clinics used the tools or coaching, which is not reported in the registry record.

The primary outcome is the Health Care System Hassles Scale, assessed at baseline and 12 months' follow-up. The record describes surveys at separate timepoints as independent cross-sectional samples. Thus, the outcome description does not characterize the observations as repeated measurements on the same individuals. The record does not report additional design details about recruitment, survey response, or how the clinic-level assignment was accounted for in the outcome analysis. Although the project is marked completed, completion status alone does not supply the missing comparative estimates or explain how the analysis was conducted.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Patient survey data were collected cross-sectionally at baseline and 12-months' follow-up to measure clinic outcomes. For the primary outcome (Health Care System Hassles Scale), both baseline and follow-up data are presented in the "outcomes" section; the primary measure of effectiveness depends on the change in outcome from baseline to follow-up.

Overall Study

Analysis population for the primary outcome, as defined in the registry: These data derive from two cross-sectional participant surveys at separate (i.e., baseline and follow-up) timepoints. The numbers of participants shown here exceed the numbers presented in the Baseline module because respondents from the surveys are two separate (i.e., independent) samples of participants. Sample sizes for each arm are smaller than reported in the participant flow module because not all survey responses received had valid primary outcome data for analysis.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Baseline participant data attributed to analyzed units (i.e., the 12 primary care clinics) come from 2484 clinic participants who responded to a cross-sectional baseline survey at the 12 clinics. Of these individuals, 40 reported receiving no VA care (the clinics being analyzed are part of the VA) in the past 12 months, so they were excluded from further analysis. Data on the remaining 2444 respondents are presented here.

6. Primary Endpoint

The registry lists the primary outcome as Health Care System Hassles Scale, measured over Baseline and 12 months' follow-up.

The primary endpoint is the mean Health Care System Hassles Scale score, expressed in units on a scale. It is described as a 16-item measure of problems patients may encounter with their general healthcare. The questions concern care experiences across the healthcare system rather than a single visit or provider. An example in the record is lack of information about why a patient has been referred to a specialist.

Each situation is rated on a 5-point scale ranging from 0 -4. Ratings are then dichotomized: 0 means “Not a problem at all,” while 1 represents any indicated level of problem. The dichotomized item responses are summed to produce a hassles count ranging from 0 to 16. Higher scores indicate more hassles. The registry specifies the mean as the outcome parameter and a 95% confidence interval as the dispersion measure.

This endpoint is used because the project focuses on coordination of care and the measure captures reported problems with general healthcare. A score can describe the burden of reported hassles, while comparing scores between strategies could address whether the groups' survey responses differ. Such a comparison requires reported outcome estimates and an analysis that reflects the study design. The registry record does not report arm-specific means or confidence intervals for a single comparison. It describes measurement at baseline and 12 months' follow-up, with separate independent survey samples at the two timepoints.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What does the mean represent?

For a continuous outcome, a mean describes the arithmetic average of the recorded scores in a group. In this study, each respondent's score is a hassles count formed by summing the dichotomized responses across 16 items. The outcome is therefore expressed as units on a scale, and a higher mean would correspond to more reported hassles on average. The mean does not show how scores vary between respondents, whether a small number of responses are especially high, or whether the groups have similar score distributions. Those features require additional descriptive information, which is not reported in the registry record.

A group mean is not itself a comparison between strategies. To assess the randomized comparison, an analysis would ordinarily report a contrast between the toolkit-plus-coaching strategy and toolkit access alone, with uncertainty around that contrast. The registry record does not report a single contrast or arm-specific means for the primary outcome. No direction or size of a treatment-strategy difference can therefore be inferred from the outcome description alone.

What does a confidence interval communicate?

General statistical explanation: a confidence interval gives a range of estimates compatible with the observed data under the assumptions of the analysis. A narrower interval indicates greater precision than a wider interval, while an interval that spans values on both sides of a no-difference value may leave the direction of the comparison uncertain. Its interpretation depends on the quantity being estimated and on the analysis assumptions. Here, the registry specifies a 95% confidence interval as the dispersion measure for the mean, but does not report the interval endpoints or a single between-strategy confidence interval.

A confidence interval is not a guarantee that the true value lies within the reported limits, and it does not resolve biases or design limitations. It also does not by itself indicate whether a difference is important to patients. In this record, the absence of reported endpoints means that neither the precision of group means nor the precision of a comparative estimate can be evaluated.

Why does the assignment level matter?

The project description says clinics were randomly assigned to the strategies, while the outcome is measured through participant surveys. General statistical explanation: when assignment occurs at a group level, responses from people treated within the same assigned group may be more alike than responses from people in different groups. An analysis that treats every response as fully independent can understate uncertainty if within-clinic similarity is present. Methods that account for clustering can adjust the uncertainty to reflect the assignment structure.

The registry record does not report the number of clinics assigned, the number of respondents within each clinic, or whether the analysis accounted for clustering. It is therefore not possible to assess from the registry description whether the reported analysis, if any, reflected the clinic-level randomization. This is a design and analysis consideration, not a claim that the analysis was conducted incorrectly.

What is the role of the survey timepoints?

The outcome is listed at baseline and 12 months' follow-up. The record describes two cross-sectional survey samples at separate timepoints and explicitly characterizes them as independent. General statistical explanation: paired analysis is appropriate when the same units are observed at multiple times and the analysis uses their within-unit relationship. Independent-sample approaches instead treat observations from different samples as unpaired. Because the registry describes separate samples, an individual change score should not be assumed.

A baseline measure can describe the groups before follow-up, and a later survey can describe responses at follow-up. Whether and how baseline information was used in the analysis is not reported in the registry record. Nor are the timing details, respondent counts, or methods for handling differences between the samples reported. These details are relevant when interpreting any comparison over time.

What can be said about the analysis plan?

The registry record reports one outcome measure and one analysis, but does not describe the statistical model, adjustment variables, hypothesis test, handling of clustering, or analysis population. It does not report whether there were interim analyses or how the sample size was justified. Those details are not reported in the registry record and should not be inferred from the randomized design or the stated outcome.

The record describes planning illustrations based on an observed effect, with a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. Such calculations are planning illustrations rather than observed power. They do not replace the actual outcome estimate, and their interpretation depends on the effect assumption. Since the registry record does not report a single observed comparative effect for the primary outcome, no specific planning illustration can be interpreted as evidence about the result. The listed assumptions do not establish that these were the trial's own design assumptions.

For this record, the central statistical limitation is not that a particular result is imprecise or statistically inconclusive; it is that a single comparative result is not reported. A p-value, confidence interval, or effect direction should not be reconstructed without the relevant underlying outcome statistics and analysis details.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists one primary outcome measure, the Health Care System Hassles Scale, and one analysis. However, the outcome description includes baseline and 12 months' follow-up, and the record does not reduce those timepoints to a single comparison. It therefore does not specify a single primary contrast to interpret. The registry record does not report whether multiple timepoints, subgroups, or other analyses were considered in the statistical plan, or whether any adjustment for multiple comparisons was used. The prespecified approach to multiplicity is not reported in the registry record.

Missing-data methods are also not described. The record does not report survey response rates, missing item counts, exclusions, imputation, or sensitivity analyses. These details matter because incomplete survey responses can affect both the estimated mean and its uncertainty, especially if missingness differs between strategies or survey timepoints. No pattern or extent of missing data can be inferred from the registry description.

The outcome description states that baseline and follow-up respondents came from separate, independent cross-sectional samples. It does not report the respondent counts by strategy and timepoint or how each sample was defined for analysis. The analysis populations are not reported in the registry record. In particular, the record does not explain whether analyses included all eligible survey responses, only complete questionnaires, or another defined population.

Although clinics were randomly assigned, the statistical approach to the clinic-level assignment is not described. The record does not say whether the analysis adjusted for clustering or how any clinic-level variation was handled. Taken together, the absence of a single reported comparison, details on multiplicity, missing-data methods, analysis populations, and clustering leaves substantial uncertainty about how the primary outcome would be analyzed and interpreted.

11. What the Result Does, and Does Not, Mean

The registry record describes a project intended to improve coordination of care across outpatient settings and to evaluate whether distance-based coaching adds value to an online toolkit strategy. It identifies a patient-reported measure of healthcare hassles and explains how higher scores correspond to more reported problems. These features make the intended research question understandable, but they do not establish the result of the comparison.

The record does not report arm-specific means, a between-strategy estimate, confidence interval endpoints, or a p-value for a single primary comparison. It is therefore not possible to say whether the scores differed, which strategy had the higher mean, or how precisely any difference was estimated. Absence of a reported result is not evidence that the strategies were equivalent, and it does not show that either strategy did or did not change care coordination.

The outcome is a survey-based hassles count. It represents the number of listed problems that respondents indicated as present at any level, rather than the severity of each problem. The record describes separate respondent samples at baseline and follow-up, so it does not support interpreting the outcome as within-person change. The results, if available, would also need to be understood in light of survey participation, the clinic-level assignment, and the analysis methods, which are not fully described in the registry record.

The zero adverse-event entries are recorded with zero at risk in both groups. They should not be interpreted as evidence that an intervention has no risks. The record's safety fields do not provide an evaluable risk comparison.

Overall, the registry supports a description of the project's design, intervention strategies, and intended endpoint, but not a conclusion about comparative effectiveness or the size or importance of any outcome difference. A clinical interpretation would require reported outcome estimates and sufficient analysis details.

12. Why This Trial Matters Statistically

This project illustrates why the unit of randomization and the unit of measurement need to be considered together. Clinics were assigned to strategies, while the primary outcome came from participant surveys. General statistical explanation: outcomes from people within the same assigned clinic may be related, so the analysis should reflect the grouping if that dependence is present. The registry record does not report whether the analysis accounted for this structure, making it difficult for readers to evaluate the precision of a comparative estimate.

The record also distinguishes repeated survey timepoints from repeated observations on the same people. Baseline and follow-up surveys are described as separate independent samples. That distinction affects which kinds of change statements are appropriate: differences between cross-sectional samples do not directly describe how individual respondents changed over time. Clear reporting of respondent counts and timepoint-specific estimates would help readers understand what population and contrast each result represents.

The endpoint's scoring method is another important statistical feature. The original item ratings are dichotomized and summed into a count, with higher scores indicating more hassles. Interpretation therefore concerns the average number of listed problems identified, rather than an average intensity rating on the original response scale. Reporting distributions alongside means could help contextualize the score, but such distributions are not reported in the registry record.

Finally, a completed status and a clear outcome definition are not substitutes for comparative results. The registry describes the intended question but does not report a single primary-outcome estimate. This distinction is important for clinical research communication: readers need both a precise account of how an outcome is constructed and transparent estimates, uncertainty, and analysis details to judge what the evidence shows.

13. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

A key limitation for interpretation is that the registry record does not report a single comparative estimate for the primary outcome. Arm-specific means, a between-strategy contrast, and confidence interval endpoints are not reported for one comparison. The primary outcome is described at baseline and 12 months' follow-up, but the record does not provide a unified result across those timepoints. This prevents assessment of effect direction, magnitude, and precision.

The outcome description identifies separate independent cross-sectional survey samples at baseline and follow-up. As a result, the registry description does not support individual-level change interpretations. It also does not report the number of survey respondents by strategy and timepoint, response rates, or how the samples were selected. The enrollment figure is listed as 12, but the record does not connect that figure to outcome-specific respondent counts in the described analysis.

Clinics were randomly assigned, but the registry does not report how many clinics were assigned to each strategy, the assignment procedure, or whether the analysis accounted for clinic-level clustering. It also does not report the statistical model, adjustment variables, missing-data methods, or analysis populations. These omissions limit assessment of whether uncertainty and potential differences in the survey samples were handled appropriately.

The record describes a toolkit and distance-based coaching strategy but does not report intervention uptake, implementation fidelity, or the amount of coaching received. Thus, the description of the intended strategies cannot establish how they were delivered in practice. The adverse-event fields list zero at risk in both groups, so the event entries cannot support a meaningful safety comparison. These limitations concern what can be learned from the registry record; they do not establish that the underlying project lacked additional information.

14. Related Tutorials

15. Related Statistical Calculators

16. Sources

• ClinicalTrials.gov: Coordination Toolkit and Coaching Project — NCT03063294.
• Registry record last updated 2023-07-27; status last verified 2023-07.
• Results first posted 2020-12-24.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

17. Record Summary

The Coordination Toolkit and Coaching Project is a completed, randomized, parallel project sponsored by the VA Office of Research and Development. The registry lists enrollment as 12, allocation as randomized, and masking as none. The project compared online toolkit access alone with toolkit access plus quality improvement support from a distance-based coach. Clinics were randomly assigned to the strategies, according to the project description.

The primary outcome was the mean Health Care System Hassles Scale score, measured at baseline and 12 months' follow-up. The scale summarizes reported problems with general healthcare; higher scores indicate more hassles. The record describes separate independent survey samples at the two timepoints.

The registry record does not report arm-specific means or a single comparative estimate for the primary outcome. It also does not report the analysis details needed to assess clustering, missing data, or the analysis populations. As a result, the intended comparison and endpoint can be described, but the comparative result and its precision cannot be established from the registry record.