Collaborating to Implement Cross-System Interventions in Child Welfare and Substance Use: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Collaborating to Implement Cross-System Interventions in Child Welfare and Substance Use (NCT03931005), a na na study. The primary outcome is Perceived Acceptability.

2. Clinical Question

3. Trial Design

This is a completed, single-group study of an implementation decision-support guide, rather than a randomized clinical treatment trial. The registry lists allocation as NA and masking as NONE. The listed phase is NA. Enrollment is 9. Those design details describe the registered study but do not indicate how many participants completed the primary measure or were included in an analysis; those numbers are not reported in the registry record.

The intervention is described as an electronic copy of a collaboration decision-support guide. The brief summary places the work in the context of implementing the Sobriety Treatment and Recovery Team (START) model across child welfare and substance-use systems. It describes identifying collaborative strategies, developing a guide to help agency leaders select strategies, and assessing feasibility, acceptability, and appropriateness. The record does not describe a control intervention or a usual-care group.

The registry lists the study start as 2024-04-25, primary completion as 2024-05-16, and status as COMPLETED. The primary measure is a post-test assessment 8 weeks after presentation of the toolkit. The record does not report participant-level assessment dates, recruitment details, the settings where the guide was used, or the exact procedures for presenting it.

The registry lists 3 outcome measures and 0 analyses. The record does not report a statistical analysis plan, the analysis population, or a primary endpoint result. The design can describe perceptions after exposure to the guide, but without a comparator or a reported baseline assessment it cannot estimate a between-group effect or change from baseline.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Perceived Acceptability, measured over One time point (Post-test only), 8 weeks after presenting the toolkit.

The primary endpoint is perceived acceptability, measured with the Acceptability of Intervention Measure (AIM). It consists of 4 survey items asking participants to rate agreement about the degree to which they perceive the intervention to be acceptable. Each item uses a 5 point scale, from 1= completely disagree to 5= completely agree. The four item scores are to be averaged, and higher averages indicate greater perceived acceptability.

The endpoint is collected at one post-test time point, 8 weeks after presenting the toolkit. It measures participants’ perceptions of the guide, not whether child welfare or substance-use services changed, whether the START model was implemented successfully, or whether any participant or family experienced a clinical outcome. The study summary also identifies feasibility and appropriateness as areas of interest, but the primary endpoint named in the registry is acceptability.

As a general measurement point, an average score combines responses across several items into one continuous summary. This can make an overall perception easier to describe, while concealing differences in how people answered individual items. The registry does not report item-level responses, the observed average score, or its standard deviation. Thus the measure’s definition and interpretation are available, but the observed level of acceptability cannot be determined from the registry record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What does the primary summary represent?

The registry defines the primary outcome as a continuous measure reported as a mean and standard deviation. A mean is an average of observed scores. For this measure, the score is intended to be the average of responses to the four AIM items. The direction is defined: higher scores correspond to greater perceived acceptability. The registry does not report the observed mean or standard deviation, so those quantities cannot be interpreted for this study.

As a general statistical explanation, the standard deviation describes variability among individual scores around their mean. A small standard deviation indicates that scores are more tightly clustered; a larger one indicates greater spread. It is not the same as the standard error, which concerns the precision of an estimated mean. No standard error or confidence interval for the primary endpoint is reported in the registry record.

What can a single-group post-test show?

The registered model is single-group, with no comparator described. A post-test score can describe responses at the measurement time, but it cannot by itself establish how responses would have differed without the guide. The primary endpoint has one post-test time point, 8 weeks after presentation of the toolkit; no baseline value is specified. As a general design principle, change requires measurements that can be compared across time, and a causal comparison usually requires an appropriate counterfactual. Neither a baseline comparison nor a control-group contrast is described for this primary question.

The record lists masking as NONE. For a perception survey, responses represent participants’ reported views, not an independently observed measure of service delivery. General sources of variation in such measures can include how respondents understand items, the context in which they answer, and differences among respondents. These are considerations for interpretation, not findings reported for this study. The registry does not describe steps used to standardize survey administration or reduce response bias.

How should precision and statistical evidence be read?

A confidence interval is a general way to express the uncertainty associated with an estimated quantity under a specified statistical method. Its width depends on factors including the amount of information and the variability in the data. A confidence interval does not establish that an effect is important, and it is not a range guaranteed to contain the true value. No confidence interval, hypothesis test, or p-value for the primary endpoint is reported in the registry record. The registry lists 0 analyses, so no analysis procedure or inferential result can be identified from the record.

For an averaged multi-item scale, it is generally useful to know whether all item responses were present, how incomplete responses were handled, and whether the planned score could be calculated for each respondent. These details affect the interpretation of a mean and its denominator. The registry describes the intended average of four items but does not report missing-item rules, observed item responses, or the number of valid endpoint scores. It also does not report whether score distributions were checked or whether any alternative summaries were planned.

What do the planning assumptions mean?

The registry’s planning illustration describes a two-sided significance level of 0.05, equal allocation between two arms, a normal approximation, and an effect specified as a fraction of the observed effect. It says that non-time-to-event planning quantities are expressed as participants per arm; time-to-event quantities are expressed as total events. These are planning assumptions, not a description of the observed single-group study design. A planning sample size is not observed power, and it does not demonstrate that an analysis had adequate precision. The registry record does not report an outcome effect estimate or the numerical planning results needed to interpret such an illustration.

More generally, sample-size planning depends on the target comparison, variability, chosen error rate, desired power, and assumptions about allocation and effect size. When an effect is defined relative to an observed effect, it is a planning scenario rather than independent confirmation of that effect. Those general principles do not fill in missing study results: no calculation of a new effect, precision estimate, or sample size is made.

What analysis details remain unknown?

The registry does not report the statistical test or estimation method for the primary endpoint, whether the primary outcome was analyzed as planned, how missing observations were treated, or which participants were included in the analysis. It also does not report adjustment for covariates, subgroup analyses, or sensitivity analyses. Without those details and the underlying outcome summaries, readers cannot assess analytic robustness. The most defensible statistical reading is therefore descriptive of the registered measurement and design, not inferential about the guide’s impact.

10. Confidence Intervals for Each Arm

No two-arm comparison is shown for this trial, but each arm's mean can be given with a t-based 95% confidence interval computed from the posted mean, spread and number of participants.

R code and output

arm <- c("CASPI")
n   <- c(9)
m   <- c(4.3)
sp  <- c(0.65)
sd  <- sp      # posted standard deviations
for (i in seq_along(n)) {
  se <- sd[i] / sqrt(n[i]); h <- qt(0.975, n[i] - 1) * se
  cat(sprintf("CARM|%s|%d|%.3f|%.3f|%.3f|%.3f\n", arm[i], n[i], m[i], sd[i], m[i] - h, m[i] + h))
}

CARM  CASPI  9  4.300  0.650  3.800  4.800

11. Multiplicity, Missing Data and Analysis Populations

The registry lists 3 outcome measures, while the primary endpoint is perceived acceptability. It lists 0 analyses. The record does not report how the other outcome measures were analyzed, whether any were designated as secondary, or whether results were planned to be interpreted together. It also does not report a multiplicity adjustment or a strategy for controlling the risk of false-positive findings across multiple outcomes.

As a general statistical concept, multiplicity arises when several outcomes, comparisons, time points, or analyses are considered. Testing many questions can increase the chance of obtaining at least one apparently notable result by chance. Methods to address this depend on the analysis plan and the intended claims. Since no analyses or adjustment approach are reported, the role of multiplicity cannot be assessed for this study.

Missing-data handling is not reported in the registry record. The measure is described as an average of 4 items, but the record does not say what happens if a participant omits an item, misses the post-test, or otherwise has an incomplete score. It also does not report the number of completed primary assessments. Enrollment is 9, but the number contributing analyzable endpoint data is not reported; enrollment should not be used as a substitute for that denominator.

The analysis population is also not reported. The record does not specify inclusion rules, exclusions, or whether any participants were omitted from an endpoint summary. These omissions limit assessment of possible selection effects and the representativeness of any estimate. Without a reported analysis population, missingness procedures, or outcome results, neither the magnitude nor the direction of any missing-data bias can be evaluated.

12. What the Result Does, and Does Not, Mean

The registry record defines a question about perceived acceptability of an electronic collaboration decision-support guide. It explains how the primary measure is scored and when it is collected, but does not report the resulting score. Therefore, the record does not establish the level of acceptability observed among participants or how consistently participants responded.

Even if an observed score were available, its meaning would be limited to reported perceptions on the specified measure and at the stated post-test time point. Acceptability is one implementation-related outcome. It is not by itself evidence that the guide was feasible to use, appropriate for a particular setting, adopted in practice, or associated with improved implementation or service outcomes. Those are distinct questions in the study summary, and their results are not reported in the registry record.

The study is single-group and has no described randomized comparator or baseline primary assessment. Its design therefore does not provide a comparative estimate of what would have happened without the guide, nor a pre-to-post change in acceptability. It cannot establish that the guide caused a change in perceptions or services. The registry’s planning assumptions do not change this interpretation: planning illustrations are not observations and are not observed power.

The record also does not provide a standard deviation, confidence interval, or endpoint analysis population. The precision and completeness of any primary endpoint estimate therefore cannot be judged. The reported enrollment of 9 is not a reported count of completed surveys or analyzed scores. The most accurate conclusion is that the registry documents an intended acceptability assessment, while the result and its uncertainty are not reported in the registry record.

13. Why This Trial Matters Statistically

This study raises a useful statistical distinction between evaluating an implementation tool and evaluating downstream services or clinical outcomes. The stated primary endpoint is participants’ perceived acceptability of a guide. That is a relevant, measurable construct for an implementation study, but it answers a narrower question than whether a cross-system model was implemented effectively or whether service outcomes changed.

The design also illustrates why a clearly defined measure is not enough to support an effect claim. The record explains the four-item scale, its response range, scoring direction, and post-test timing. Yet the observed mean, variation, and analysis denominator are not reported. Without these, readers cannot quantify the responses or their precision. The distinction between a measurement plan and an observed result is central to sound interpretation.

Another statistical issue is the difference between descriptive evidence and comparative evidence. A single-group post-test assessment can describe reported views at that assessment, but it does not offer a control comparison or a baseline measure for the primary endpoint. Those design features constrain the questions that can be answered, regardless of the scale’s detail. The listed planning assumptions for a two-arm comparison are not the same as the registered single-group design and do not show observed power.

Finally, the registry identifies several broad aims and 3 outcome measures, but reports 0 analyses. This makes transparent reporting of endpoint results, missing-data procedures, and analysis populations especially important for interpretation. These details would help distinguish a favorable average from a precise, complete, and broadly applicable finding. As recorded, the study’s statistical value lies in its stated measurement and design questions; the record does not provide the results needed to determine what the assessment found.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The principal limitation for interpreting this record is the absence of reported outcome values. The registry describes the primary measure but does not report its mean, standard deviation, confidence interval, or analysis result. The registry also lists 0 analyses. Consequently, the level and precision of perceived acceptability cannot be assessed.

The design is single-group, with allocation listed as NA and masking as NONE. No comparator is described, and the primary endpoint is assessed at one post-test time point rather than with a reported baseline measure. These features limit causal interpretation and prevent a direct comparison of post-test perceptions with a pre-intervention value. The registered design does not establish whether any response pattern is attributable to the guide.

Enrollment is 9, but the registry does not report how many people completed the survey or contributed valid scores. Participant roles and characteristics are not reported, so the reach of any findings to agency leaders or other intended users cannot be assessed. The record also does not describe the settings, survey administration procedures, or the handling of incomplete item responses.

The registry lists 3 outcome measures, but does not give analysis details, missing-data methods, analysis populations, or multiplicity procedures. The brief summary identifies wider aims related to feasibility, appropriateness, collaboration, implementation, and service outcomes, but results for those aims are not reported in the registry record. These limitations are specific to the information and design recorded; they do not establish what findings may exist outside the registry record.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Collaborating to Implement Cross-System Interventions in Child Welfare and Substance Use — NCT03931005.
• Registry record last updated 2026-04-07; status last verified 2026-03.
• Results first posted 2026-04-07.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

The registry identifies a completed, single-group study of an electronic collaboration decision-support guide in the context of child welfare and substance use. Enrollment is 9; allocation is listed as NA, masking as NONE, and phase as NA. The registered dates are 2024-04-25 for study start and 2024-05-16 for primary completion.

The primary outcome is perceived acceptability, measured with the four-item AIM and summarized as an average score, with higher scores indicating greater acceptability. It is assessed at one post-test time point 8 weeks after presenting the toolkit. The registry lists 3 outcome measures and 0 analyses, but does not report the observed primary score, its variability, the analysis population, or missing-data procedures. No comparator is described. The record therefore documents the study question and measurement approach, but does not provide the primary endpoint result or evidence of a comparative effect.