Extension for Community Healthcare Outcomes Autism Replication Evaluation: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Extension for Community Healthcare Outcomes Autism Replication Evaluation (NCT03677089), a na na study. The primary outcome is Longitudinal Pattern of ASD Screening in PCP Charts.

2. Clinical Question

3. Trial Design

The registry identifies this study as the ECHO Autism Replication Evaluation and gives its official title as ECHO Autism Replication: Step Wedge Randomized Trial. It records the intervention as behavioral and describes one ECHO Cohort receiving 12 ECHO Autism telehealth clinics. Sites were to begin the intervention in clusters of two sites, with 3 months between the start of each cluster.

That descriptive rollout does not, by itself, establish how sites were assigned to start times. The registry records allocation as NA, masking as none, and the model as single group. It also flags that this is not a randomized comparison and that the arms are not a like-for-like control comparison. The registry record does not report a control-group plan or the method used to determine intervention timing.

Enrollment is recorded as 148. The primary outcome was planned at four points: baseline (Month 0), mid-intervention (Month 3), post-intervention (Month 6), and end of follow-up (Month 9). It was based on reviews of subsets of charts from each PCP’s practice rather than a measure of all children in the enrolled group. The eligible chart population was narrower still: Canadian-site participants were excluded from this outcome, and inclusion required children with a well-child visit at 18 or 24 months.

The registry status is completed. It lists a study start date of 2016-09-01 and primary completion date of 2018-11-27. These record details describe the planned and registered design; they do not supply evidence about the outcome of the intervention. The registry does not report how many participants contributed data at each assessment point, how many charts were reviewed, or how the timing of site rollout was incorporated into the analysis.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: The 10 sites were randomly assigned to cohort using SAS. Each site was assigned a random number (from a uniform distribution), and the two lowest numbers were assigned to period 1, etc. through the two highest numbers assigned period 5. No blocking or stratification was used in the randomization.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Participants were not included if they were at the Canadian site because Canadian screening practices are different from the AAP recommendations used in the study. In addition, participants can be included in this outcome measure only if they have children with a well-child visit at 18 or 24 months in the 30 days prior to the chart review.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Longitudinal Pattern of ASD Screening in PCP Charts, measured over Baseline (Month 0), mid-intervention (Month 3), post-intervention (Month 6), and end of follow-up (Month 9).

The primary endpoint is the longitudinal pattern of ASD screening in PCP charts, expressed as the percentage of children screened. The registry describes chart review for appropriate ASD screening during well-child visits, with results summarized for each PCP. Measurements were planned at baseline (Month 0), mid-intervention (Month 3), post-intervention (Month 6), and end of follow-up (Month 9).

This outcome is intended to describe clinical practice or behavior, rather than provider knowledge alone. Chart documentation can indicate whether screening was recorded in eligible visits, while the repeated assessment points are intended to show how that measure changes over time. The registry specifies a mean as the summary parameter and standard deviation as the dispersion measure. General statistical interpretation: a mean describes the average of the measured percentages, while a standard deviation describes how dispersed those values are. Neither alone indicates the precision of an estimated change.

The target population for this endpoint is restricted. The registry excludes the Canadian site because its screening practices differed from the AAP recommendations used in the study. It also limits inclusion to participants with children who had a well-child visit at 18 or 24 months. The registry record does not report the number of eligible charts, the observed percentages at any time point, or the method used to handle repeated observations from the same PCP. Consequently, the outcome’s definition is available, but its observed pattern is not reported in the registry record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The registry describes the primary endpoint as a mean of a continuous measure, with standard deviation as the dispersion statistic. The measured quantity is a percentage for each PCP, based on chart reviews of appropriate ASD screening during eligible well-child visits. The assessment schedule comprises baseline (Month 0), mid-intervention (Month 3), post-intervention (Month 6), and end of follow-up (Month 9).

What do a mean and standard deviation describe?

General explanation: a mean summarizes the central level of a set of measurements. A standard deviation describes how much those measurements vary around their mean. It is a description of spread, not a measure of how precisely the mean is estimated. Precision depends on the amount and structure of the data, among other factors. The registry record does not give the observed means or standard deviations for this outcome.

Why does the repeated-measures structure matter?

General explanation: measurements taken repeatedly from the same provider or practice may be related rather than independent. A statistical model can account for such within-provider dependence; treating correlated measurements as independent can misstate uncertainty. The registry record does not report the statistical model used, whether provider-level correlation was addressed, or how the chart subsets were formed across assessment points. It is therefore not possible to evaluate those choices from the posted record.

The rollout description also identifies sites as the units beginning the intervention in clusters of two, with 3 months between the start of each cluster. General explanation: when intervention timing is organized by site or cluster, outcomes within a site may share features. An analysis may need to represent both the timing of intervention and dependence among observations from the same site. The registry record does not report whether or how site clustering or rollout timing was incorporated into an analysis.

How should uncertainty be read?

General explanation: a confidence interval conveys uncertainty around an estimate under the assumptions of the analysis. A narrow interval generally indicates greater precision than a wide interval, but neither width nor statistical significance alone establishes practical importance. The registry record does not report a confidence interval or other uncertainty measure for the primary endpoint. It also does not report a p-value or a model-based estimate of change over time.

The registry records no analyses. That means the record does not identify a fitted model, covariates, a contrast between assessment points, or a procedure for accounting for the restricted chart population. It does not report an analysis population for the primary outcome or the number of providers and charts represented at each time point. These omissions prevent readers from checking whether the reported endpoint definition was carried through consistently into an analysis.

What do the sample-size assumptions represent?

The registry describes a planning illustration based on an observed effect, with a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. It says the effect to detect is a stated fraction of the observed effect. These are assumptions for an illustration, not evidence of the trial’s own design or an observed-power result. The illustration’s equal-allocation, two-arm premise does not match the registry’s single-group model and its warning that there is no like-for-like control comparison. The registry record does not report the trial’s own sample-size justification. Nor does it report a primary-outcome effect from which the illustration’s effect basis could be evaluated. Any planning figure should therefore be understood as an assumption-based illustration, not as a finding about the study’s statistical power.

General explanation: over-dispersion occurs when variability is greater than a model assumes. If a model assumes less variability than the data contain, its uncertainty estimates can be too small. The registry does not report a model or diagnostics that would allow assessment of over-dispersion for this outcome. More broadly, the method used to summarize percentages, account for provider or site structure, and manage repeated observations is not reported in the registry record.

10. Confidence Intervals for Each Arm

No two-arm comparison is shown for this trial, but each arm's mean can be given with a t-based 95% confidence interval computed from the posted mean, spread and number of participants.

R code and output

arm <- c("Baseline", "Mid-Intervention", "Post-Intervention", "End of Follow-Up")
n   <- c(86, 86, 87, 86)
m   <- c(61, 64, 66, 64)
sp  <- c(41, 41, 40, 39)
sd  <- sp      # posted standard deviations
for (i in seq_along(n)) {
  se <- sd[i] / sqrt(n[i]); h <- qt(0.975, n[i] - 1) * se
  cat(sprintf("CARM|%s|%d|%.3f|%.3f|%.3f|%.3f\n", arm[i], n[i], m[i], sd[i], m[i] - h, m[i] + h))
}

CARM  Baseline  86  61.000  41.000  52.210  69.790
CARM  Mid-Intervention  86  64.000  41.000  55.210  72.790
CARM  Post-Intervention  87  66.000  40.000  57.475  74.525
CARM  End of Follow-Up  86  64.000  39.000  55.638  72.362

11. Multiplicity, Missing Data and Analysis Populations

The registry lists the primary outcome of longitudinal ASD screening in PCP charts and secondary outcomes covering provider ASD knowledge, provider self-efficacy, perceived barriers to care, participant satisfaction, program attendance, and the number of co-morbidities in children with ASD. It records 8 outcome measures in total. Considering multiple outcomes or multiple time-point comparisons can raise multiplicity questions: when many statistical tests are performed, the chance of at least one apparently notable result may increase. General statistical practice may address that issue by prespecifying a primary contrast or using an adjustment strategy. The registry record does not report a multiplicity plan, a hierarchy for interpreting secondary outcomes, or an adjustment method.

The record lists 0 analyses and does not report outcome estimates, p-values, or confidence intervals. It does not state which time-point comparisons, if any, were tested. As a result, there is no reported analysis from which to assess how multiplicity was handled.

The primary outcome has specific eligibility restrictions: Canadian-site participants were excluded, and participants could be included only if children had a well-child visit at 18 or 24 months. The registry does not report the number excluded or the number with eligible charts at each assessment point. It also does not report missing-data reasons, the extent of missing chart information, or a method for handling missing observations.

General statistical explanation: missingness can affect interpretation when the probability of having an observed measurement is related to the outcome or to participant characteristics. Repeated assessments also require clarity about whether the same providers and chart-review rules contributed over time. The record does not report an analysis population definition, an intention-to-treat approach, or a sensitivity analysis for missing data. These omissions do not show that any particular bias occurred; they mean the registry record does not provide enough information to assess it.

12. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry reports safety figures for the ECHO Cohort, with 132 participants at risk in the serious-adverse-event listing and 132 at risk in the other-adverse-event listing. It records 0 serious adverse events, 0 other adverse events, and 0 deaths. No serious adverse-event terms are listed.

These figures describe the entries in the registry record. They do not provide a detailed account of event definitions, monitoring procedures, follow-up duration for safety, or how events were assessed. The record does not explain the difference between the enrollment count of 148 and the 132 participants listed as at risk for these safety summaries. The safety denominators should not be assumed to represent the primary chart-review population.

General statistical explanation: a recorded zero is a report of no events in the stated category and population in the registry entry; it does not establish that risk is impossible or that all possible events were captured. Because there is no like-for-like control comparison recorded, these figures do not support a comparative estimate of adverse-event risk. They should be read as descriptive registry counts, not as evidence for a causal safety conclusion.

13. What the Result Does, and Does Not, Mean

The registry record establishes the purpose and planned structure of the primary outcome: review PCP charts for appropriate ASD screening and summarize the percentage of children screened at four assessment points. It also describes a 12-session telehealth training program intended to assess provider knowledge, clinical behavior, and self-efficacy. These are descriptions of study aims and measurement, not outcome findings.

No numerical result for the primary screening outcome is reported in the registry record. There is no reported estimate of change over time, no uncertainty interval, and no analysis result. The record therefore does not establish whether screening percentages increased, decreased, or remained similar, nor how precisely any observed pattern might have been estimated. A completed status does not supply missing results.

The design record should also temper interpretation. The title uses the term step-wedge randomized trial, but allocation is recorded as NA, the model is single group, and the record says this is not a randomized comparison with like-for-like control arms. Staggered starts across site clusters are described, but the method for assigning start times and the analysis of those timings are not reported. The registry record therefore does not establish a controlled causal contrast.

The endpoint applies to a selected set of charts. The Canadian site was excluded from this outcome, and inclusion required children with a well-child visit at 18 or 24 months. The number of eligible charts, the number with usable data at each assessment, and the consistency of chart sampling over time are not reported. Any interpretation of the endpoint would need to consider those limits as well as the missing statistical analysis.

General statistical explanation: a change in a measured outcome over time is not automatically attributable to an intervention. Interpretation depends on design, measurement, analysis, and uncertainty. In this case, the registry record specifies the intended outcome and timing but does not report findings needed to estimate a pattern or separate it from other influences. It supports a description of what was planned and registered, not a conclusion about the effect of the training program.

14. Why This Trial Matters Statistically

This trial record is statistically notable because its title describes a step-wedge randomized trial while the recorded design fields identify a single-group model and allocation as NA. The registry itself cautions that the study is not a randomized comparison and that the arms are not a like-for-like control comparison. Reading those details together helps distinguish a study title and rollout description from the information needed to establish a randomized contrast.

The primary outcome also illustrates how an apparently straightforward percentage can depend on a defined measurement population. It comes from chart reviews at PCP practices, excludes the Canadian site for a stated difference in screening practices, and depends on children having a well-child visit at 18 or 24 months. The outcome is planned repeatedly, while the registry does not report the number of eligible charts or the analysis method for repeated observations. Such features matter when judging both the meaning of a percentage and the precision of a trend.

The record’s lack of outcome analyses is equally important. It lists 0 analyses, so readers cannot examine estimates, uncertainty, or how the analysis addressed the site rollout, repeated measurements, or missing observations. The planning illustration is described with equal allocation between two arms, although the registry records a single-group model. Recognizing that distinction prevents a planning assumption from being mistaken for a feature of the trial or a result.

General statistical lesson: clear registration of outcomes and assessment times helps define what a study intends to measure, but interpretation also requires reported results and methods. In this record, the outcome definition and design limitations are available, while the numerical evidence needed to evaluate the primary question is not reported in the registry record.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to the registered outcome and design. First, despite the randomized-trial language in the title, allocation is recorded as NA and the model as single group. The registry explicitly says this is not a randomized comparison and that the arms are not like-for-like control groups. The record describes a staged site rollout but does not report how start times were assigned or how they were analyzed.

Second, the primary endpoint is based on chart subsets from PCP practices, not every child or every visit. Canadian-site participants are excluded because their screening practices differed from the recommendations used in the study. Inclusion also depends on children having a well-child visit at 18 or 24 months. The record does not report how many charts met these conditions or whether chart sampling was consistent at each assessment.

Third, the endpoint is repeated across baseline (Month 0), Month 3, Month 6, and Month 9, but the registry does not report how observations were linked across providers or sites. It also does not report the statistical model, the number of observations at each time point, missing-data handling, or sensitivity analyses. These gaps prevent assessment of the analysis assumptions and the precision of any trend.

Finally, no primary-outcome results are reported and the record lists 0 analyses. The safety entries report counts for 132 participants at risk, while enrollment is 148; the registry does not explain that difference. These reporting limitations do not demonstrate that the study was conducted improperly, but they restrict what readers can infer from the registry record.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Extension for Community Healthcare Outcomes Autism Replication Evaluation — NCT03677089.
• Registry record last updated 2020-04-30; status last verified 2020-04.
• Results first posted 2020-04-30.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

The registry record describes a completed evaluation of 12 ECHO Autism telehealth clinics intended to assess PCP knowledge, clinical behavior, and self-efficacy related to ASD screening and care. Enrollment is 148. Sites were described as starting the intervention in clusters of two, with 3 months between cluster starts. The primary outcome was the longitudinal pattern of ASD screening in PCP charts, measured at baseline (Month 0), Month 3, Month 6, and Month 9.

The primary outcome excluded the Canadian site and required children with a well-child visit at 18 or 24 months. The record lists 8 outcome measures and 0 analyses. It reports no primary-outcome estimates or uncertainty measures. Safety entries list 132 participants at risk, 0 serious adverse events, 0 other adverse events, and 0 deaths. Allocation is recorded as NA, the model as single group, and the registry cautions that there is no like-for-like control comparison. The numerical primary results and analysis methods are not reported in the registry record.