Reverse Innovation and Patient Engagement to Improve Quality of Care and Patient Outcomes: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Reverse Innovation and Patient Engagement to Improve Quality of Care and Patient Outcomes (NCT02222909), a randomized na study. The primary outcome is Change in Emergency Department Visits + Hospital Days.

2. Clinical Question

In plain terms, the question is about the change over time in a combined measure of emergency department visits and hospital days, comparing the intervention and control CBOs. It is not simply a comparison of post-intervention values: the stated endpoint uses both pre-intervention and post-intervention periods. The result is reported as a difference in differences, with randomization inference as the analysis method.

The record identifies a cluster-randomized design: organizations, rather than individual patients, were allocated to groups. That distinction matters when interpreting what was randomized and the level at which outcomes were collected. The listed enrollment is 6767, while the primary outcome description specifically says data were collected at the CBO level. The record does not clarify how the enrollment figure relates to the primary outcome analysis population.

3. Trial Design

Project CONNECT, whose expanded name is Community-based Organizations Neighborhood Network: Enhancing Capacity Together, was a randomized controlled trial involving 22 community-based organizations in Baltimore, Maryland. According to the registry summary, half of the organizations were randomly assigned to the intervention group using constrained cluster randomization; the remaining 11 were control CBOs. Thus, the organization was the unit of assignment described in the record, rather than an individual patient.

The intervention was a co-developed set of information-technology tools. The arm description characterizes the tools as primarily web based and describes intended support for connections among CBOs, clients, and Johns Hopkins Medicine, along with referrals and volunteer services. The control CBOs were followed for data collection only during the stated one-year period. The record does not describe further activities for control organizations.

The registry lists the study as parallel, unmasked, and randomized. It lists phase as not applicable. The listed enrollment is 6767. That registry enrollment should not be assumed to equal the number of CBOs or the number of observations in the primary outcome analysis: the primary outcome description says data were collected at the community-based organization level, and the record does not explain the relationship between enrollment and the analyzed data.

The study start is listed as 2014-04, and primary completion as 2015-12. The record describes a pre-intervention period from 09/01/2012 to 03/09/2014 and a post-intervention period from 03/10/2014 to 09/30/2015 for the primary outcome. The study status is completed.

As a cluster-randomized study, its allocation and analysis concern organizations and their associated outcome data. The registry reports randomization inference for the primary analysis. A simple individual-level re-analysis is not presented because it could misrepresent the cluster-randomized design.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Data collected at the community-based organization level.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: The baseline characteristics describe demographic information about the J-CHiP patients who were each assigned to a 'proximal' CBO from the twenty participating. The average utilization outcomes and changes among the patients assigned to each of the CBOs constituted the CBO-level outcomes for the primary analysis.

6. Primary Endpoint

The registry lists the primary outcome as Change in Emergency Department Visits + Hospital Days, measured over Pre-Intervention (09/01/2012 to 03/09/2014); Post-Intervention (03/10/2014-09/30/2015.

The primary endpoint is titled “Change in Emergency Department Visits + Hospital Days.” It measures the change in the average number of monthly emergency department visits and days spent in the hospital from the pre-intervention period to the post-intervention period. Its stated unit is the difference in monthly hospital days plus emergency department visits. The outcome is described as a mean, with standard deviation as its dispersion measure.

The registry says the outcome is for J-CHiP patients assigned to each CBO and is adjusted for baseline prognostic variables. It also states that data were collected at the community-based organization level. The record does not explain the exact construction of this combined measure, the specific adjustment variables, or the method used to aggregate patient-level information into CBO-level outcome data.

General explanation: a change-over-time endpoint can account for the fact that groups may have different outcome levels before an intervention period begins. A difference-in-differences comparison then contrasts the changes in the groups rather than treating a single follow-up measurement as the full comparison. This describes the general statistical rationale, not additional details about how the endpoint was constructed in this trial.

The endpoint is relevant to the trial’s stated aim of improving connections between CBOs, clients, and health-care facilities, because it evaluates a utilization-related measure over time. It does not, by itself, measure every aspect of connection, referral activity, volunteer service, care quality, or patient engagement. The registry does not report additional primary outcome definitions or explain whether components of the combined measure were analyzed separately.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The registry identifies the primary analysis as randomization inference and the reported parameter as a difference in differences. The outcome is a mean of a continuous measure: the change in average monthly emergency department visits and hospital days. The description also says the estimate is adjusted for baseline prognostic variables. The exact adjustment variables and further implementation details are not reported in the registry record.

What does difference in differences represent?

General explanation: a difference-in-differences analysis compares the change over time in one group with the change over time in another. It focuses on whether the groups’ trajectories differ, rather than interpreting only their values at a single time point. In this trial, the record names pre-intervention and post-intervention periods and describes a comparison of the changes for the CBO groups. The reported value of 0.018 is the analysis estimate for that contrast.

The meaning of the sign depends on the order used to subtract the groups. Because the registry does not state the ordering of the two groups for the reported contrast, the sign alone cannot identify which group had the larger change. The outcome’s unit is the difference in monthly hospital days plus emergency department visits. The estimate should not be recast as a percentage, a patient-level count, or a number of visits attributable to the intervention.

Why use randomization inference?

General explanation: randomization inference evaluates an observed result against results that could arise under the random assignment process, under a specified null hypothesis. In a cluster-randomized study, the assignment process applies to clusters, so an analysis that respects the cluster allocation is important. The registry identifies constrained cluster randomization and reports randomization inference. It does not report the exact set of assignments used for inference, the test statistic implementation, or other computational details.

Randomization inference is distinct from a method that assumes independent assignment of every individual. Here the organizations were assigned to groups, and the record says the outcome data were collected at the CBO level. Treating all patient-level observations as if they had been individually randomized could fail to reflect the allocation structure. The registry’s exclusion of a simple individual-level re-analysis is consistent with keeping the analysis aligned with the cluster-randomized design.

How should the confidence interval be read?

The reported confidence interval is 95%, with limits -0.12 and 0.11. General explanation: a confidence interval expresses the uncertainty associated with an estimate under the method and assumptions used. A wider interval indicates less precision than a narrower interval on the same scale. Because this interval includes zero and also extends to values on either side, it does not isolate a single direction of difference. It should not be interpreted as proof that the true contrast is zero, or as a range that assigns a probability to each possible value.

The registry does not explain how the interval was constructed for the randomization-inference analysis. It is therefore appropriate to report the stated interval without attributing an unreported calculation method to it. Its limits should also be interpreted on the endpoint’s stated scale rather than as a relative change.

What does the p-value add?

The reported p-value is 0.63. General explanation: a p-value assesses how unusual the observed statistic, or a result at least as extreme under a specified measure of extremeness, would be under a null model. It is not the probability that the null hypothesis is true, and it does not measure the size or practical importance of an effect. The registry does not specify further details of the p-value calculation beyond naming randomization inference as the method.

Read alongside the confidence interval, the p-value indicates that the analysis does not distinguish the observed contrast from zero with strong statistical evidence. That is not equivalent to demonstrating no difference. Precision, design, endpoint definition, and completeness of the reported analysis all matter to interpretation. No claim about equivalence or a prespecified margin is reported in the registry record.

What does adjustment mean in this record?

The outcome description says the estimate is adjusted for baseline prognostic variables. General explanation: adjustment incorporates measured baseline information into an analysis to account for its association with the outcome. The record does not name the variables, explain their coding, or state how adjustment was integrated with the randomization-inference procedure. No more specific account of the adjustment can therefore be made from the registry description alone.

The record lists a standard deviation as the endpoint’s dispersion measure, but the reported analysis is a difference-in-differences estimate with an interval and p-value. It does not provide group-specific means or standard deviations in the registry record for this explanation. Such values should not be inferred from the contrast. In general, dispersion describes variation in a continuous measure, while the confidence interval describes uncertainty around an estimated contrast; those are related but different statistical ideas.

10. Confidence Intervals for Each Arm

No two-arm comparison is shown for this trial, but each arm's mean can be given with a t-based 95% confidence interval computed from the posted mean, spread and number of participants.

R code and output

arm <- c("Control CBOs", "Intervention CBOs")
n   <- c(10, 10)
m   <- c(-0.0069, 0.0113)
sp  <- c(0.513, 0.525)
sd  <- sp      # posted standard deviations
for (i in seq_along(n)) {
  se <- sd[i] / sqrt(n[i]); h <- qt(0.975, n[i] - 1) * se
  cat(sprintf("CARM|%s|%d|%.3f|%.3f|%.3f|%.3f\n", arm[i], n[i], m[i], sd[i], m[i] - h, m[i] + h))
}

CARM  Control CBOs  10  -0.007  0.513  -0.374  0.360
CARM  Intervention CBOs  10  0.011  0.525  -0.364  0.387

11. Analyses Reported by the Sponsor

The registry reports the sponsor’s analysis as randomization inference for a difference-in-differences parameter: 0.018, with a 95% confidence interval from -0.12 to 0.11 and a p-value of 0.63. It describes the estimate as adjusted for baseline prognostic variables. No independent recalculated estimate is reported for this trial. Because the design is cluster randomized and the primary outcome is described at the CBO level, a simple individual-level re-analysis would not provide a suitable substitute for the sponsor’s design-aligned analysis.

The reported result should therefore be read as the registry’s own analysis, not compared with an independently recalculated result. The record does not provide enough information about the adjustment variables or detailed randomization-inference procedure to reconstruct the analysis from the description alone.

The confidence interval includes zero, and the p-value is 0.63. These figures describe uncertainty and statistical evidence under the reported analysis; they do not establish equality of the groups’ changes. The positive estimate also cannot be assigned to intervention or control direction because the group ordering for the contrast is not stated. The sponsor’s reported analysis is the relevant result available in the registry record.

12. Multiplicity, Missing Data and Analysis Populations

The registry lists one outcome measure and one reported analysis. It does not report whether additional outcomes, time points, subgroup analyses, or alternative models were examined outside the listed primary result. A single listed analysis does not establish that no other statistical tests were conducted.

Information about multiplicity adjustment is not reported in the registry record. General explanation: when multiple hypotheses are tested, the chance of obtaining at least one apparently unusual result can increase, depending on the testing strategy. Whether or how that issue applies cannot be determined from the record, which does not describe a broader testing plan or any adjustment procedure.

Details about missing outcome data, exclusions, imputation, and the handling of incomplete observations are not reported in the registry record. The record also does not define the analysis population beyond describing the outcome as applying to J-CHiP patients assigned to each CBO and stating that data were collected at the CBO level. It does not say whether all randomized CBOs contributed to the analysis or how any unavailable data were handled.

Likewise, the analysis plan is not reported in enough detail to establish whether the reported analysis followed a prespecified plan. The exact baseline prognostic variables used for adjustment, the construction of the confidence interval, and the randomization-inference implementation are not reported in the registry record. These omissions limit assessment of multiplicity, missing-data assumptions, and analysis-population choices; they do not justify assuming that a particular procedure was or was not used.

The registry’s adverse-event fields list zero for at-risk and affected counts in both groups, and no serious-event terms are listed. Because the at-risk counts are also recorded as zero, these entries do not provide a meaningful basis for interpreting the frequency of adverse events among participants. The record does not clarify the status or completeness of safety reporting.

13. What the Result Does, and Does Not, Mean

The reported estimate describes a difference in changes over time in a combined monthly measure of emergency department visits and hospital days. It is not a direct estimate of change within the intervention group alone, and it is not a simple post-intervention comparison. The unit is the difference in monthly hospital days plus emergency department visits.

The 95% confidence interval, -0.12 to 0.11, includes zero, and the reported p-value is 0.63. In statistical terms, the registry result does not provide strong evidence against a null difference under the reported analysis. It does not prove that the groups changed identically, establish equivalence, or rule out effects in either direction. The record does not state a threshold for a clinically important difference.

The positive estimate of 0.018 should not be read as favoring either arm, because the registry does not say which group was subtracted from which in the reported contrast. Nor should the estimate be converted to a percentage or interpreted as a number of patients, visits, or hospital days without further details about the scale and aggregation.

The endpoint covers utilization as a combined measure. It does not, on its own, show whether the intervention changed the connections among CBOs, clients, and health-care facilities, or whether referrals and volunteer services changed. The registry does not report separate results for emergency department visits and hospital days, or measures of the other intended intervention functions.

General explanation: a statistical result is one component of evidence. Its interpretation depends on the design, measurement, analysis, and completeness of reporting. Here, assignment occurred at the CBO level, and the registry reports a cluster-aware analysis. Missing-data procedures, analysis populations, and further statistical-plan details are not reported in the registry record. Those limits call for a narrow reading of the result: it is the reported primary contrast for the specified combined outcome, not a comprehensive evaluation of every possible effect or mechanism.

14. Why This Trial Matters Statistically

This trial illustrates why the unit of randomization matters to statistical analysis. Project CONNECT assigned community-based organizations, not individual patients, to intervention or control. The primary outcome description concerns patients assigned to those organizations, while data collection is described at the CBO level. The statistical comparison therefore needs to reflect the cluster structure rather than treating every patient observation as independently randomized.

The registry identifies constrained cluster randomization and randomization inference. These details are important because the allocation process shapes which comparisons are appropriate. The record explicitly excludes a simple individual-level re-analysis as potentially misleading. This is a useful example of a general principle: the analysis should correspond to the design and assignment unit, especially when organizational clusters are randomized.

The endpoint also demonstrates the distinction between a change within a group and a difference in changes between groups. The primary parameter is a difference in differences, comparing pre-intervention and post-intervention change across the CBO groups. That is a different question from whether a group’s outcome rose or fell in isolation. Keeping that distinction clear prevents the estimate from being overinterpreted.

Finally, the registry makes visible the limits of a concise statistical record. It reports an estimate, confidence interval, p-value, and named analysis method, but does not report several implementation details, including the adjustment variables and missing-data handling. Statistical interpretation depends not only on the headline result but also on knowing what was measured, how assignment occurred, and what analysis details are available. In this case, those considerations support a cautious reading of the reported contrast and discourage claims beyond the specified outcome.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry does not report the exact baseline prognostic variables used for adjustment, the detailed implementation of randomization inference, or the ordering of groups in the reported contrast. As a result, the direction represented by the positive estimate cannot be assigned to intervention or control, and the analysis cannot be fully reconstructed from the record’s description.

The primary outcome is a combined measure of monthly emergency department visits and hospital days. The record does not give separate component results or explain the aggregation procedure. It also does not report group-specific outcome summaries in the registry record, so the overall contrast cannot be used to infer each group’s underlying level or trajectory.

The record states that the outcome was collected at the CBO level and describes it as applying to J-CHiP patients assigned to each CBO, but it does not clarify how the enrollment figure of 6767 relates to the primary analysis. Analysis-population definitions, missing-data handling, exclusions, and completeness of outcome data are not reported in the registry record.

Multiplicity procedures and the broader statistical analysis plan are also not reported. The registry lists one outcome measure and one analysis, but that alone does not establish that other analyses were not conducted. The record does not state whether the reported analysis was prespecified or whether sensitivity analyses were performed.

Finally, the safety fields list zero at-risk and affected counts in both groups, without clarifying the meaning or completeness of those entries. No serious-event terms are listed. These fields do not support conclusions about event frequency. Taken together, the limitations call for interpreting the primary estimate as the reported registry analysis of a particular combined outcome, rather than as a full account of all trial outcomes or analyses.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Reverse Innovation and Patient Engagement to Improve Quality of Care and Patient Outcomes — NCT02222909.
• Registry record last updated 2024-02-09; status last verified 2023-06.
• Results first posted 2024-02-09.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

Project CONNECT was a completed, randomized, parallel, unmasked trial involving 22 community-based organizations in Baltimore. The registry describes constrained cluster randomization, with 11 intervention CBOs and 11 control CBOs. The intervention was a co-developed set of primarily web-based tools intended to support connections among organizations, clients, and Johns Hopkins Medicine, as well as referrals and volunteer services. The listed enrollment is 6767.

The primary endpoint was change in average monthly emergency department visits and hospital days from the pre-intervention period to the post-intervention period, adjusted for baseline prognostic variables. Data were collected at the CBO level. The reported analysis used randomization inference for a difference-in-differences estimate of 0.018, with a 95% confidence interval from -0.12 to 0.11 and p = 0.63.

The confidence interval includes zero. The group ordering for the contrast, detailed adjustment variables, missing-data procedures, and broader analysis plan are not reported in the registry record. The reported result should be interpreted as the registry’s analysis of the specified combined outcome, with those reporting limits in mind.