SEEK: Dissemination and Implementation: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for SEEK: Dissemination and Implementation (NCT03642327), a randomized na study. The primary outcome is Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level).

2. Clinical Question

The central statistical question is about implementation rather than a clinical treatment effect. The trial compares two ways of training practitioners and examines whether SEEK is adopted and put into practice. Its primary measure records startup at the practice level; it does not directly measure whether children experienced a change in maltreatment or another health outcome. The registry also lists secondary outcomes concerning possible abuse and neglect, adherence to medical care, immunization, and practitioner responses to psychosocial needs. Those outcomes add context about implementation and related practice measures, but they do not change the meaning of the primary endpoint.

3. Trial Design

The registry describes a randomized, parallel-group study with no masking. It lists an enrollment of 360 and identifies the sponsor as the University of Maryland, Baltimore. The study status is completed. The recorded start date is 2019-03-15 and the primary completion date is 2024-03-31. The phase is listed as not applicable.

The two training approaches are Independent Online Training (IND) and Maintenance of Certification (MOC). IND involves practitioners independently doing online training. MOC is described as a guided learning experience approved for Maintenance of Certification credits and involving a Quality Improvement project. The registry description of that project is cut off, so further protocol details are not reported in the registry record. The brief study summary also mentions examining software versus traditional paper-and-pencil approaches to implementation; how those approaches relate to the two training groups is not reported in the registry record.

The primary outcome is a practice-level count of practices that started implementing SEEK, with a timeframe of up to 30 months. The posted arm counts are 148 for IND and 212 for MOC, and the reported startup counts are 10 and 9, respectively. The registry does not establish that those arm counts represent the denominator of practices eligible for the startup measure, so they should not be treated as denominators for a rate without further clarification. The total enrollment is 360, but the unit represented by enrollment is not further described in the registry record.

The registry lists 23 outcome measures and 20 posted secondary outcomes. It does not report a statistical analysis plan, allocation details beyond randomized assignment, or procedures for handling missing outcome data. The results record contains no reported analyses. Those omissions limit what can be concluded about how the groups were compared and how the posted results should be interpreted as prespecified trial analyses.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Of the 44 practices that expressed initial interest, 29 agreed to participation. Four withdrew prior to randomization, so 25 practices were randomized to IND (13) or MOC (12), stratified by practice type (family medicine/pediatrics) and resident training site (yes/no). 6 additional practices withdrew early on. Reasons included feeling already overwhelmed, compounded by the pandemic; the sole family medicine practice cited plans for universal screening within their healthcare system.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Baseline demographic data was only collected on the PCPs.

6. Primary Endpoint

The registry lists the primary outcome as Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level), measured over up to 30 months.

The primary endpoint is Stages of SEEK Implementation Completion (SIC) - Startup, assessed at the practice level over up to 30 months. The SIC assessment has 8 stages, extending from Engagement to achievement of Competency, and maps those stages onto 3 implementation phases: Pre-Implementation, Implementation, and Sustainment. Startup captures whether a practice implemented SEEK: yes or no. The registry says practices that did start are reported.

This endpoint is useful for an implementation study because it captures whether the intervention reached the initial stage of use in a practice. It is not a measure of implementation quality, sustained use, practitioner competence, or a child health outcome. Although the larger SIC framework encompasses those broader stages and phases, the posted primary result concerns startup only. The outcome is reported as a count, not as a rate or a model-based comparison. The registry record does not report a confidence interval or a statistical test for the primary outcome, and it does not provide a denominator that can be confidently linked to the reported startup counts. Consequently, the count can describe the posted startup results, but cannot by itself establish the size or precision of a difference between training approaches.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The posted results include a count endpoint and continuous questionnaire outcomes. These require different statistical interpretations. The primary result is a count of practices that started SEEK. The secondary comparisons use group means and standard deviations for questionnaire subscales. The registry does not report a formal analysis for the primary count outcome or sponsor-reported analyses for the posted continuous outcomes. The comparison estimates described for the continuous endpoints are unadjusted recalculations from the posted summary statistics.

What does a mean difference represent?

A mean difference compares the average score in one group with the average score in the other. For these outcomes, the contrast is MOC relative to IND. A negative estimate means the posted average is lower in MOC; a positive estimate means it is higher. This direction describes the arithmetic of the scale, not whether one approach is better. The registry does not provide a threshold for a meaningful score difference or enough scale interpretation to attach a clinical meaning to a particular change.

The group means are summaries, not values that every practitioner attained. Standard deviations describe how spread out individual scores are around each group mean. A small difference between averages can coexist with substantial overlap in individual scores. Conversely, an average difference alone does not explain the distribution of individual scores. The posted sample sizes for these secondary measures are 70 in IND and 95 in MOC, which are smaller than the arm counts recorded for the primary results. Reasons for the different counts are not reported in the registry record.

How should confidence intervals be read?

A confidence interval expresses the uncertainty around an estimated group difference under the assumptions of the analysis. A narrower interval indicates greater precision than a wider interval, while an interval spanning zero is compatible with no mean difference as well as differences in either direction at the interval’s stated confidence level. The intervals for all three posted continuous comparisons include zero. That does not establish that the groups are identical; it means the estimates do not precisely distinguish a difference from zero under the unadjusted calculations. The registry does not report adjusted confidence intervals or a prespecified threshold for practical importance.

The continuous-outcome comparison estimates and intervals are recalculated from posted means, standard deviations, and group sizes using a simple unadjusted approach. They are not described as adjusted for practice characteristics, clustering, repeated measures, or other covariates. The registry does not report whether the sponsor used a different analytic method, and no sponsor-reported analyses are listed for these outcomes. The results should not be attributed to a more complex model than the record documents.

Why does the primary count not have the same comparison?

The primary endpoint is a count of practices that started implementation. A group comparison would ordinarily need clearly defined numbers of practices at risk of starting in each group, together with a method for estimating uncertainty. Although the registry reports arm counts of 148 and 212, it does not clarify their relationship to the practice-level endpoint’s denominator. The results record also flags the count outcome as not estimable for a two-arm comparison. Without an interpretable denominator and a reported analysis, turning the counts into rates or a comparative effect would require assumptions not documented in the registry record. No such calculation is appropriate.

What do the planning illustrations mean?

The sample-size illustrations are planning scenarios derived from the observed secondary-outcome effects; they are not observed power and do not describe the trial’s original design justification. The stated assumptions are a two-sided significance level of 0.05, equal allocation, and a normal approximation. The scenarios consider detecting 100%, 75%, or 50% of the observed effect and report participants per arm for 80% and 90% power. These are hypothetical planning quantities, not evidence that the completed study had a particular probability of detecting an effect. The trial’s sample-size justification is not reported in the registry record.

Planning illustrations based on observed effects also depend on the stability of those estimates. Estimates from a study can be imprecise, and a smaller assumed target effect leads to a larger planning requirement under the stated method. Such calculations can help frame future study design, but they do not repair uncertainty in the observed comparison, account for unreported design features, or establish that the original trial was adequately powered. The registry record does not report whether its original design used these assumptions.

Multiplicity is another important consideration. The registry lists many outcome measures, while only a subset of secondary results is posted. If many outcomes are tested, unadjusted p-values can overstate the strength of evidence from any one nominally notable result. No adjustment or outcome hierarchy is reported. Therefore, the unadjusted comparisons are best treated as descriptive and exploratory rather than as a set of definitive tests.

10. Multiplicity, Missing Data and Analysis Populations

The registry record does not report a statistical analysis plan, a hierarchy of primary and secondary hypotheses, or a method for controlling multiplicity. It lists 23 outcome measures and 20 posted secondary outcomes, so the three continuous comparisons should not be treated as though they were the only outcomes considered. Their p-values are unadjusted. When researchers test many outcomes, the chance of observing at least one apparently notable result by chance can increase. Adjustment methods or a prespecified hierarchy can help address this, but neither is reported in the registry record.

Missing-data handling is also not reported. The posted sample sizes for the continuous questionnaire outcomes are 70 for IND and 95 for MOC, while the primary result lists arm counts of 148 and 212. The record does not explain why the available counts differ across outcomes, whether data were missing, or whether the populations analyzed differed by outcome. It does not describe imputation, complete-case analysis, exclusions, or sensitivity analyses. Accordingly, the posted means may represent the people with recorded scores rather than everyone randomized, but the analysis population is not reported in the registry record.

The record does not state whether the continuous results follow intention-to-treat assignment, a per-protocol definition, or another analysis population. It also does not report adjustment for clustering of practitioners within practices, despite the practice-level focus of the primary outcome. These gaps matter because allocation, unit of analysis, and missingness can affect both the interpretation and precision of results. The absence of details should not be filled by assuming a conventional approach. Without a documented analysis plan, complete outcome-specific populations, and a stated multiplicity strategy, the secondary comparisons are most appropriately read as unadjusted descriptive evidence.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry reports 0 serious adverse events and 0 other adverse events in both groups, and 0 deaths in both groups. The corresponding at-risk group counts are 148 for IND and 212 for MOC. No serious adverse-event terms are listed.

These figures mean that no events were recorded in the posted safety categories; they do not establish that the true risk of an event is zero. Interpretation depends on how events were defined, collected, and attributed, and those details are not reported in the registry record. The number of people or practices represented in each safety group and the period of safety follow-up are also not fully described. A zero count can be consistent with no events observed in the recorded data while still leaving uncertainty about uncommon events or incomplete reporting. The results should therefore be read as registry safety counts, not as proof that either training approach is risk-free or as a comparative safety analysis.

12. What the Result Does, and Does Not, Mean

The clearest primary finding is descriptive: the registry reports that 10 practices in IND and 9 practices in MOC started implementing SEEK. The endpoint is a practice-level startup count over up to 30 months. Since the record does not report a comparative effect estimate, uncertainty interval, or p-value, and does not clearly connect the posted arm counts to the endpoint denominator, the data do not quantify how precisely the groups differ in startup. The counts alone cannot establish whether the training approaches produce different implementation rates.

The continuous secondary results describe questionnaire averages at 30 months. Their unadjusted mean differences are small in scale units, and the intervals include zero. Those results do not demonstrate equivalence or rule out effects that the study could not estimate precisely. The registry does not report a threshold for a practically important change, and the posted outcome descriptions do not establish how differences in scale units translate to implementation quality or child outcomes.

The findings also do not establish that SEEK reduced child maltreatment or improved any particular child health outcome. The study’s purpose is described as understanding adoption and implementation, while the primary endpoint measures startup. The registry lists child-related secondary outcomes, but the full results and analyses for those outcomes are not available in the posted record. Nor can the results establish that one training method is superior, because the primary comparative analysis is not reported and the secondary comparisons are unadjusted.

In general, a result is most informative when its outcome definition, denominator, analysis population, uncertainty, and prespecified interpretation are clear. Several of those elements are not reported in the registry record. The posted counts and questionnaire summaries remain useful descriptive evidence, but conclusions should be limited to what those statistics directly convey.

13. Why This Trial Matters Statistically

This trial is statistically interesting because it evaluates implementation at a practice level while also reporting practitioner questionnaire measures and child-related outcomes. Those outcomes represent distinct units and questions. Startup is a count of practices; questionnaire measures are averages of scores among those with posted data; child-related outcomes concern possible abuse, neglect, care adherence, or immunization. Keeping these levels separate is essential when interpreting a complex implementation study.

The primary count illustrates why a clear denominator and analysis plan matter. The registry reports the numbers of practices that started, but does not establish how the recorded arm counts relate to the denominator for that endpoint or post an inferential comparison. Reporting counts can communicate observed activity, but a rate comparison requires a defined population at risk and a method that captures uncertainty. Without those elements, the numerical difference between counts cannot be interpreted as an effect estimate.

The secondary results also illustrate the distinction between an estimate and its precision. Mean differences summarize observed contrasts, while intervals describe uncertainty. Here the intervals include zero, and the comparisons are unadjusted. A nominal p-value alone cannot establish importance, particularly when many outcomes are listed and the multiplicity strategy is not reported. The planning illustrations further show how observed effect estimates can be used to frame hypothetical future sample sizes, but they are not observed power and do not reconstruct the original design rationale.

For biostatisticians and clinical researchers, the record highlights the value of complete reporting: outcome-specific denominators, analysis populations, handling of missing data, prespecified comparisons, and uncertainty estimates for the primary endpoint. Those details would make the implementation findings more reproducible and help distinguish descriptive evidence from confirmatory inference.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations are specific to the available registry results. The primary outcome is a count of practices that started, but the denominator for calculating a startup rate is not clearly reported. Although arm counts of 148 and 212 are listed, the record does not explain whether those counts are practices at risk for the primary outcome or another analysis population. The primary result also lacks a comparative estimate and uncertainty interval, limiting assessment of the magnitude and precision of any group difference.

The posted continuous secondary outcomes have smaller outcome-specific group sizes than the arm counts listed for the primary results. The reasons for this are not reported. No missing-data approach, analysis population, or sensitivity analysis is described. As a result, it is unclear how representative the observed questionnaire summaries are of the randomized groups. The registry does not report adjustments for clustering or other covariates, and it does not provide a statistical analysis plan to determine which analyses were prespecified.

Only a subset of the listed outcomes has detailed posted results. Although the registry lists 23 outcome measures and 20 secondary outcomes, the available results do not give a complete outcome-by-outcome account. The secondary comparisons are unadjusted recalculations, and a multiplicity strategy is not reported. The MOC arm description is incomplete, and the relationship between the study summary’s software-versus-paper-and-pencil comparison and the two training groups is not explained. These reporting gaps do not show that the study did not address those issues; they mean the relevant details are not reported in the registry record.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: SEEK: Dissemination and Implementation — NCT03642327.
• Registry record last updated 2025-05-29; status last verified 2025-05.
• Results first posted 2025-05-29.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT03642327, titled SEEK: Dissemination and Implementation, is a completed, randomized, parallel-group study with no masking. It lists an enrollment of 360 and compares Independent Online Training with Maintenance of Certification training for implementation of the SEEK model. The primary endpoint counts practices that started implementation over up to 30 months. The registry reports 10 practices starting in IND and 9 in MOC, without a comparative estimate or uncertainty interval.

At 30 months, three posted secondary questionnaire outcomes report group means and unadjusted recalculated comparisons. Their intervals include zero. The registry lists additional outcome measures but does not provide a complete set of results or a statistical analysis plan. Missing-data handling, the analysis populations, the primary endpoint denominator, and the multiplicity strategy are not reported in the registry record. Safety counts show 0 serious adverse events, 0 other adverse events, and 0 deaths in each group.