← Clinical Trials
Attention Deficit Hyperactivity Disorder NA Binary Endpoint NCT02271386

Educating and Supporting Primary Care Providers in the Implementation of Evidence-based Practices for ADHD: Complete Statistical Analysis

This randomized crossover trial examined whether Supporting Practice for ADHD was associated with greater use of evidence-based practices, using chart documentation of whether patients were sent parent rating scales as the primary measure. The registry results compare intervention and control clinicians during the baseline period.

Trial start: 2014-11  ·  Enrollment: 895  ·  Sponsor: Children's Hospital of Philadelphia
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for Educating and Supporting Primary Care Providers in the Implementation of Evidence-based Practices for ADHD (NCT02271386), a randomized na study. The primary outcome is Number of Patients Who Were Sent the Parents Rating Scale.

895
Enrollment
NA
2
Arms
In the primary outcome
-2.9 pts
Risk difference
Patients of Intervention Clinicians
-10.5 to 4.7 pts
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT02271386
PhaseNA
ConditionAttention Deficit Hyperactivity Disorder
AllocationRandomized
Design modelCrossover
MaskingNone (open label)
Enrollment895
Arms in the primary outcome2
Lead sponsorChildren's Hospital of Philadelphia
Results postedYes

2. Clinical Question

Population

Children aged 5 to 12 years with attention deficit hyperactivity disorder (ADHD) in primary care practices were the intended clinical population. The primary outcome was assessed through chart review of patients seen by clinicians assigned to the study arms. For the baseline comparison, the registry reports 200.0 patients of intervention clinicians and 191.0 patients of control clinicians.

Intervention

Supporting Practice for ADHD (SPA) was a behavioral intervention for clinicians. Its three components were education about ADHD management and communication training, collaborative consultation, and performance feedback about evidence-based practices.

Comparator

Control clinicians received no intervention during the first 8 months, after which they were to receive SPA during the last 8 months. The primary outcome comparison concerns the baseline period, before the intervention period described in the registry.

Primary question

Did the proportion of reviewed patient charts with evidence that a parent rating scale had been sent differ between patients of intervention clinicians and patients of control clinicians during the baseline period? The registry results describe the observed difference and its uncertainty; they do not establish that the intervention caused a difference in practice.

3. Trial Design

The registry describes a randomized, unmasked, crossover trial of SPA, a behavioral intervention intended to support primary care providers in applying evidence-based ADHD practices. The trial is listed as completed, with enrollment of 895. The listed start was 2014-11, and primary completion was 2016-07-31. The registry does not report a phase beyond “NA.”

Clinicians in the intervention arm were scheduled to receive SPA during the first 8 months. Clinicians in the control arm were to receive no intervention during that first 8 months, then receive SPA during the last 8 months. The record therefore describes a crossover in which the initial control group later received the intervention. The specific timing and procedures used for measuring outcomes across all periods are not fully detailed in the posted primary outcome description.

The primary outcome is reported for a baseline period of 8 months. Chart review covered patients of intervention clinicians and patients of control clinicians, with review described as up to 4 patients per clinician. For the primary comparison, the outcome counts are 33.0 among 200.0 patients of intervention clinicians and 37.0 among 191.0 patients of control clinicians. These are patient-level chart counts, while SPA was delivered to clinicians; the registry does not report additional details about the randomization unit or about how the analysis accounted for patients grouped under clinicians.

The record lists 28 outcome measures and 20 posted secondary outcomes. It does not report the trial’s original statistical analysis plan, a sample-size justification, or interim analyses. Those omissions limit what can be inferred about how the posted summaries relate to the trial’s prespecified analysis.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Baseline Period- 8 Months

StageIntervention CliniciansControl CliniciansPatients of Intervention CliniciansPatients of Control Clinicians
STARTED5352200191
COMPLETED5352200191
NOT COMPLETED0000

Intervention Period- 8 Months

StageIntervention CliniciansControl CliniciansPatients of Intervention CliniciansPatients of Control Clinicians
STARTED5352202197
COMPLETED5352202197
NOT COMPLETED0000

Analysis population for the primary outcome, as defined in the registry: We reviewed charts for a total of 402 patients of intervention clinicians (200 who had an office visit in the 8-month baseline period and 202 who had an office visit in the 8-month intervention period). We reviewed charts for a total of 388 patients of control clinicians (191 in the baseline period and 197 in the control period).

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

CharacteristicUnitIntervention CliniciansControl CliniciansPatients of Intervention CliniciansPatients of Control CliniciansTotal
Participants analysed5352402388895
Age, Continuousyears——9.3 (SD 1.9)9.2 (SD 2.0)9.3 (SD 2.0)
Sex: Female, Male: FemaleParticipants4345107129324
Sex: Female, Male: MaleParticipants107295259571
Ethnicity (NIH/OMB): Hispanic or LatinoParticipants——161834
Ethnicity (NIH/OMB): Not Hispanic or LatinoParticipants——386370756
Ethnicity (NIH/OMB): Unknown or Not ReportedParticipants——000
Race/Ethnicity, Customized: WhiteParticipants——248143391
Race/Ethnicity, Customized: BlackParticipants——104221325
Race/Ethnicity, Customized: OtherParticipants——502474
Insurance payer: MedicaidParticipants——169224393
Insurance payer: PrivateParticipants——230163393
Insurance payer: MissingParticipants——314
Provider Type: PediatricianParticipants4548——93
Provider Type: Nurse PractitionerParticipants84——12
Baseline Perception of Acceptability of Managing ADHD in Primary Care- Assessing ADHDunits on a scale3.2 (SD 0.6)3.3 (SD 0.4)——3.3 (SD 0.5)
Baseline Perception of Acceptability of Managing ADHD in Primary Care- Providing Mental Health Careunits on a scale3.4 (SD 0.5)3.4 (SD 0.5)——3.4 (SD 0.5)
Baseline Perception of Acceptability of Managing ADHD in Primary Care- Recommending Medicationsunits on a scale3.5 (SD 0.6)3.6 (SD 0.4)——3.6 (SD 0.5)
Perceived effectiveness at managing ADHD in primary careunits on a scale2.9 (SD 0.8)3.1 (SD 0.6)——3.0 (SD 0.7)

6. Primary Endpoint

The registry lists the primary outcome as Number of Patients Who Were Sent the Parents Rating Scale, measured over Baseline Period (8 months).

ElementRegistry entry
Result typeCount Of Participants
Measure of spreadnot applicable
UnitParticipants
Endpoint classificationBinary Endpoint

The primary endpoint is the number of patients whose charts showed evidence that a parent rating scale was sent during the baseline period, which lasted 8 months. The registry describes the measure as based on chart review, with up to 4 patients reviewed per clinician. The posted arm counts are 33.0 among 200.0 patients of intervention clinicians and 37.0 among 191.0 patients of control clinicians. The comparison summaries also express these counts as proportions: 0.165 and 0.194, respectively.

This endpoint focuses on a documented clinical process: whether a parent rating scale was sent. It is relevant to the trial’s aim of increasing evidence-based practice use because rating scales can form part of ADHD assessment and management. However, chart evidence of sending a scale does not establish that it was completed, returned, reviewed, or used in a clinical decision. The registry lists returned parent and teacher rating scales as secondary outcomes, which are related but distinct process measures.

The endpoint does not directly measure a child’s symptoms, functioning, or health outcome. It also does not by itself describe the quality or completeness of ADHD care. It provides a specific indicator of recorded practice during the stated baseline interval, rather than a comprehensive measure of all evidence-based care.

7. Results Posted in the Registry

Risk difference: Patients of Intervention Clinicians

-2.9

95% CI: -10.5 to 4.7 percentage points

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals10.613.917.120.423.726.9Patients of Intervention Clin… (n=200)Patients of Control Clinicians (n=191)% of participants (95% CI)
Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Patients of Intervention Clin…-12-8.17-4.64-1.112.425.95risk difference (percentage points) vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
Number of Patients Who Were Sent the Parent Rating Scale - by MOC StatusBinary EndpointRisk difference+1.1 pts (-9.4 to 11.7)16,357 per arm
Number of Patients Whose Parent Rating Scale Was Returned- by MOC StatusBinary EndpointRisk difference-2.1 pts (-12.3 to 8.1)4,563 per arm
Number of Patients Who Were Sent the Teacher Rating Scale- by MOC StatusBinary EndpointRisk difference+1.1 pts (-9.4 to 11.7)16,357 per arm

Number of Patients Who Were Sent the Parent Rating Scale - by MOC Status

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Patients of Intervention Clin…-11-6.18-1.303.598.4713risk difference (percentage points) vs control (95% CI)

Number of Patients Whose Parent Rating Scale Was Returned- by MOC Status

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Patients of Intervention Clin…-14-9.17-4.440.285.019.74risk difference (percentage points) vs control (95% CI)

Number of Patients Who Were Sent the Teacher Rating Scale- by MOC Status

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Patients of Intervention Clin…-11-6.18-1.303.598.4713risk difference (percentage points) vs control (95% CI)

The posted secondary outcomes concern whether parent and teacher rating scales were sent or returned, with results grouped by MOC status. They extend the primary outcome’s focus on recorded assessment practices, but they address different steps in the process and different comparisons. A scale being sent is not the same event as a scale being returned, and neither alone demonstrates how the information affected care.

Across the posted secondary comparison summaries, the estimated differences are close to zero in relation to their confidence intervals, and the intervals include no difference. The directions are not uniform across the outcomes: some estimates favor one subgroup and others favor the other. Taken together, these results do not provide a consistent signal that separates the groups. They should be considered alongside the primary baseline comparison, not as independent confirmation of an intervention effect.

The record lists 20 posted secondary outcomes, while the secondary outcomes named in the registry include repeated parent- and teacher-scale measures by MOC status. The three detailed comparison summaries in the record are unadjusted, and no reported secondary analyses are listed. It is not reported how these secondary measures were selected for analysis, whether all were planned in advance, or whether any multiplicity adjustment was used.

When many outcomes are examined, some estimates may look unusual by chance even if there is no underlying difference. This general issue means that unadjusted secondary p-values and intervals carry less weight than a single prespecified primary test, especially when the analysis plan and selection process are not reported. The secondary results can describe the posted data and suggest questions for further study, but they should not be treated as definitive evidence of subgroup differences or as a replacement for a prespecified analysis.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

During the baseline period, 33.0 of 200.0 reviewed patients of intervention clinicians had chart evidence that a parent rating scale was sent. For patients of control clinicians, the corresponding figures were 37.0 of 191.0. The posted proportions are 0.165 and 0.194. The risk difference, defined in the comparison as intervention clinicians minus control clinicians, is -0.029, with a 95% confidence interval from -0.105 to 0.047. The unadjusted p-value is 0.459.

The point estimate is a small negative difference in the intervention clinicians’ direction of comparison: the observed proportion was lower in that group during baseline. The confidence interval includes zero, so the data are compatible with no difference as well as with differences in either direction within the interval. The p-value does not provide evidence against the no-difference hypothesis at conventional significance thresholds, but it is not a probability that the null hypothesis is true. Nor does this baseline comparison test whether SPA changed practice after it began.

The comparison is descriptive of chart records during the baseline period. It should not be interpreted as an intervention effect, because the measure precedes the intervention period described in the record. The trial’s later crossover design means that both groups were scheduled to receive SPA at different times; interpretation of later contrasts would require attention to period effects, intervention exposure, and any persistence of effects. Details of such analyses are not reported in the registry record.

The endpoint is a process measure, not a direct measure of patient health. Evidence that a scale was sent does not establish that it was returned, interpreted, or acted upon. The registry lists returned-scale outcomes separately, reinforcing that these steps are not interchangeable. Likewise, the primary comparison does not establish whether clinicians followed other ADHD guidelines or whether patients’ symptoms or functioning changed.

The registry also reports risk-ratio and odds-ratio summaries for the primary comparison: the risk ratio is 0.852, with interval 0.557 to 1.303, and the odds ratio is 0.823, with interval 0.49 to 1.38. Their intervals include the respective no-association values of 1. These summaries tell a similar broad story to the risk difference: the estimate is imprecise and does not distinguish a difference from no difference. Because no adjusted analysis is reported, these summaries should be read as unadjusted comparisons of the posted chart counts.

Overall, the primary result documents similar observed levels of the recorded practice in the two groups at baseline, with uncertainty wide enough to include differences in either direction. It neither demonstrates a benefit of SPA nor answers whether the intervention changed clinician behavior during the subsequent periods.

10. Statistical Methods Explained

What does the risk difference describe?

The primary comparison uses a risk difference. In general, a risk difference subtracts the event proportion in one group from the event proportion in the reference group, using the stated direction of comparison. Here the contrast is intervention clinicians minus control clinicians. The posted proportions are 0.165 and 0.194, and the estimated risk difference is -0.029. A negative value means the chart-recorded event proportion was lower in the intervention clinician group during baseline under that subtraction order. This is a comparison of observed proportions, not a causal estimate of what SPA did.

Risk differences are expressed on the absolute probability scale. They can be useful for communicating how far apart two observed event proportions are, while relative measures describe a ratio between them. The scale matters: an absolute difference and a relative ratio answer related but distinct questions. No single measure captures every aspect of an effect, so interpretation should also consider the endpoint, study design, uncertainty, and analysis population.

What do confidence intervals and p-values contribute?

In general, a confidence interval describes the range of effect estimates reasonably compatible with the data under the calculation’s assumptions. It makes uncertainty visible rather than presenting the point estimate alone. For the primary risk difference, the 95% interval runs from -0.105 to 0.047. Because that interval crosses zero, values representing a negative difference, no difference, and a positive difference are all compatible with the interval. The interval does not assign a probability to each possible effect, and it cannot account for sources of bias that the analysis did not address.

A p-value measures how unusual data at least as inconsistent with a specified null model would be if that model and the test assumptions held. The primary unadjusted p-value is 0.459. It is not the probability that there is no difference, and it does not measure the importance of an observed difference. A p-value also does not correct for missing observations, clustering, selection, or an analysis that differs from the original plan.

How do the relative measures differ?

The risk ratio compares the event proportion in one group with that in another. The odds ratio compares the odds of the event rather than the proportions themselves. For the primary result, the risk ratio is 0.852, with interval 0.557 to 1.303; the odds ratio is 0.823, with interval 0.49 to 1.38. For both ratios, 1 represents no relative difference. Their intervals include 1, as the risk-difference interval includes zero. The registry does not report enough methodological detail to determine whether these estimates came from a particular regression model or to assess any model adjustment; they should therefore be read as posted comparison summaries, not as evidence of a more elaborate adjusted analysis.

Why does the design affect interpretation?

The trial is randomized and uses a crossover arrangement, but the primary outcome presented is from baseline. Random assignment can help balance groups in expectation, while a crossover can allow groups to receive the intervention at different times. Neither feature makes a baseline contrast an intervention effect. The record does not report the original statistical plan, the randomization unit, methods for accounting for patients clustered within clinicians, or analyses of period and carryover effects. Those details matter when interpreting repeated observations or later comparisons in a clinician-level intervention study.

In general, observations from patients treated by the same clinician may be correlated because they share clinical context and provider practices. When such dependence is present, methods that treat every patient record as fully independent can understate uncertainty. The registry does not report whether or how this issue was addressed, so the displayed intervals should not be assumed to incorporate a particular clustering adjustment.

What does “unadjusted” mean in this context?

An unadjusted comparison summarizes group differences without accounting for other measured characteristics through a statistical adjustment. This can make the contrast straightforward to read, but it does not ensure that groups are comparable on every factor in the observed sample. Randomization is relevant to group comparability, yet the registry does not provide a detailed analysis plan or covariate balance results for this endpoint. The posted unadjusted estimate therefore describes the available summary data and should not be treated as a fully adjusted treatment effect.

The record lists no reported analyses for the posted secondary outcomes. The comparisons displayed for the primary and secondary summaries should not be mistaken for a complete account of the prespecified statistical methods. The trial’s own choice of tests, model structure, handling of repeated measures, missing data, or multiplicity correction is not reported in the registry record. General statistical descriptions explain how the posted measures are commonly interpreted; they do not establish which methods were used beyond what the record states.

11. Confidence Intervals and Precision

Precision concerns how narrowly the data estimate the group difference. The primary risk difference is -0.029, and its 95% confidence interval extends from -0.105 to 0.047. This interval is broad enough to include zero and values on both sides of it. The point estimate alone would give an incomplete impression; the interval shows that the observed sample does not pin down the direction or size of the underlying difference precisely.

In general, a confidence interval is a range generated by a statistical procedure that, under its assumptions and repeated use, has the stated coverage. It is not a guarantee that the true effect lies inside a particular calculated interval, and it does not include uncertainty from every possible source. For example, it cannot by itself address whether chart review captured the relevant practice consistently, whether the statistical model accounted for clinician-level grouping, or whether missing records differed between groups. The registry does not report those methodological details.

The relative-effect intervals tell a similar story. The risk-ratio interval is 0.557 to 1.303, and the odds-ratio interval is 0.49 to 1.38; both include 1, the value representing no relative difference. The p-value of 0.459 is also consistent with the uncertainty conveyed by the risk-difference interval. These statistics do not show that the groups are equivalent. Establishing equivalence would require an appropriate design and a prespecified margin, neither of which is reported for this comparison.

The chart counts are drawn from patients seen by clinicians, and the registry says that chart review covered up to 4 patients per clinician. If patient outcomes within a clinician are correlated, that dependence could affect precision. Whether the posted interval accounts for such grouping is not reported in the registry record. Precision should therefore be interpreted in light of the reported interval while recognizing that the analysis details needed to evaluate its assumptions are unavailable.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect2,8013,749
75% of the observed effect5,0586,770
50% of the observed effect11,55315,465
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect2,0005,00010,00020,00050,00040%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Treat these figures cautiously. The observed difference is imprecise: its confidence interval includes no difference at all. A sample size built on a noisy estimate can be far too small or too large, so the smaller-effect rows are the more cautious guide.
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, pooled-variance normal approximation for two proportions (control proportion held at the observed value). Calculated for the comparison: Patients of Intervention Clinicians.

The sample-size planning figures are illustrations based on the observed effect, not a report of the trial’s original design assumptions. For the primary comparison, the figures are participants per arm. The stated planning assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is set to the stated fraction of the observed effect.

For planning for 100% of the observed effect, the illustration gives 2,801 participants per arm for 80% power and 3,749 participants per arm for 90% power. For planning for 75% of the observed effect, it gives 5,058 participants per arm for 80% power and 6,770 participants per arm for 90% power. For planning for 50% of the observed effect, it gives 11,553 participants per arm for 80% power and 15,465 participants per arm for 90% power.

In general, power is the chance that a study will detect an effect of a specified size under the planning assumptions if that effect is real. The 80% column represents a design targeting an 80% chance of detection under those assumptions; the 90% column represents a higher target. Smaller effects are harder to distinguish from random variation, so planning to detect a smaller effect requires more participants. The figures are planning illustrations, not observed power.

The original design assumptions and the trial’s own sample-size justification are not reported in the registry record. These illustrations use the observed effect and should not be read as evidence that the completed trial was designed around those assumptions. They also do not resolve questions about clinician-level grouping, missing chart data, or the relevance of the observed effect as a future planning target.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = [ z1−α/2 √(2p̄(1−p̄)) + z1−β √(p0(1−p0) + p1(1−p1)) ]2 / (p1 − p0)2

n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

  1. Control proportion p0 = 0.194; treatment proportion p1 = 0.165; average p̄ = 0.179.
  2. First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.5426 = 1.0634.
  3. Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.5422 = 0.4563.
  4. n = (1.0634 + 0.4563)2 / (-0.0287)2 = 2,800.6, rounded up to 2,801 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.193717277486911                          # control proportion
p1 <- 0.165                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(2271386)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 2,801 (80% power), 3,749 (90% power)
 75% of observed effect: n per arm = 5,058 (80% power), 6,770 (90% power)
 50% of observed effect: n per arm = 11,553 (80% power), 15,465 (90% power)
power.prop.test: 2800.6 
Simulated power at n = 2,801 per arm: 0.797 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry lists 28 outcome measures and 20 posted secondary outcomes. Examining multiple outcomes can increase the chance that at least one result appears noteworthy by chance. In general, multiplicity can be addressed through prespecified outcome hierarchies, adjusted significance thresholds, or other planned procedures. The record does not report whether such a procedure was planned or used, so the secondary comparisons should be interpreted as unadjusted summaries rather than as a family of results with a known overall error rate.

The registry reports an unadjusted p-value for the primary comparison and unadjusted p-values for the displayed secondary comparisons. It does not report the trial’s original statistical analysis plan, whether the primary endpoint and analysis were finalized before outcome data were examined, or whether any interim analyses occurred. These details are important for understanding how strongly to weigh individual results and whether the posted analyses match the intended confirmatory analysis.

Missing-data procedures and analysis populations are also not reported in detail. The primary measure is based on chart review, and the registry gives group-specific chart counts for the baseline comparison. It does not describe the number of eligible charts not reviewed, how incomplete records were treated, whether patients with unavailable documentation were classified as not having the event, or whether an imputation method was used. It also does not state whether the analysis followed an intention-to-treat principle or another defined population rule.

These gaps matter because a chart with no evidence of a scale being sent may represent a scale not sent, incomplete documentation, or an unavailable record. The endpoint is explicitly based on evidence in the chart, but the registry does not explain how ambiguous or missing documentation was handled. Nor does it describe an analysis accounting for patient clustering within clinicians or the repeated periods in a crossover design. Without these details, the posted estimates remain useful descriptions of the recorded comparison, but their assumptions and robustness cannot be fully assessed.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
Intervention Clinicians0530.0%
Control Clinicians0520.0%
Patients of Intervention Clinicians04020.0%
Patients of Control Clinicians03880.0%

Deaths

ArmDeaths (participants affected of at risk)
Intervention Clinicians0 of 53
Control Clinicians0 of 52
Patients of Intervention Clinicians0 of 402
Patients of Control Clinicians0 of 388

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

ArmParticipants affected of at risk
Intervention Clinicians0 of 53
Control Clinicians0 of 52
Patients of Intervention Clinicians0 of 402
Patients of Control Clinicians0 of 388
Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The registry reports serious adverse events, other adverse events, and deaths for intervention clinicians, control clinicians, and patients of clinicians in both groups. For each listed group, the number affected by serious adverse events is 0, the number affected by other adverse events is 0, and the number of deaths is 0. The corresponding numbers at risk are 53 and 53 for the clinician groups, and 402 and 388 for the patient groups.

These figures mean that the registry reports no affected participants in the listed categories during the recorded safety reporting. They do not show that an event was impossible or establish that SPA has no safety risks. Interpretation depends on the event definitions, period of observation, completeness of collection, and the populations included. Those details are not reported in the registry record. The trial’s primary outcome is a care-process measure, so the safety counts should be understood separately from the comparison of rating-scale practice.

16. What the Result Does, and Does Not, Mean

The primary result describes how often charts documented that a parent rating scale was sent during the 8-month baseline period. The observed proportion was 0.165 among patients of intervention clinicians and 0.194 among patients of control clinicians. The estimated risk difference was -0.029, with a 95% confidence interval from -0.105 to 0.047 and an unadjusted p-value of 0.459. The interval includes zero and spans differences in either direction.

These figures do not show that SPA improved or worsened practice. The primary comparison is from baseline, before the intervention period described in the record, and is not a direct estimate of the intervention’s effect. The result also does not establish equivalence between groups. A finding that does not distinguish a difference from no difference is not the same as proof that the groups behave identically.

The endpoint has a narrow meaning: chart evidence that a parent rating scale was sent. It does not establish that the scale was received, completed, returned, reviewed, or incorporated into care. It does not measure symptoms, functioning, or other patient outcomes. Secondary outcomes address some further steps, but the registry’s posted secondary comparisons are unadjusted and do not form a reported analysis plan.

The broader trial aim was to increase use of evidence-based practices for ADHD through clinician education, communication training, collaborative consultation, and feedback. The baseline figures alone cannot answer whether those components changed later practice. The crossover design and the planned timing of intervention for each arm make period-specific analyses important, but details of the relevant models and comparisons are not reported in the registry record.

The most defensible reading is therefore limited: the posted baseline data show no clear separation in the chart-recorded parent-scale sending measure, and the estimate is imprecise. They do not demonstrate an intervention effect, establish that care was or was not evidence-based overall, or show a patient health benefit or harm.

17. Why This Trial Matters Statistically

This trial illustrates why the outcome definition and timing of a comparison matter as much as the numerical estimate. Its primary measure is a documented clinical process, and the reported primary comparison is from baseline. A reader could mistake the randomized design for proof that every between-group contrast estimates the intervention effect; the timing makes that interpretation inappropriate for the primary result.

It also demonstrates the importance of distinguishing an observed point estimate from its precision. The risk difference is -0.029, but its 95% confidence interval extends from -0.105 to 0.047. The result is not a precise estimate of a single direction or magnitude. The related risk-ratio and odds-ratio intervals likewise include their no-difference value. Statistical interpretation should therefore center on the range of uncertainty and the design context, not on a point estimate or p-value alone.

The record further highlights a common measurement distinction: sending a rating scale, receiving a completed scale, and using its information in care are separate steps. An outcome that captures one part of a process cannot stand in for the whole clinical pathway or for a patient outcome. The separation is reflected in the registry’s additional parent- and teacher-scale measures.

Finally, multiple outcomes, unadjusted comparisons, chart-based measurement, and patient observations linked to clinicians all make clear reporting of analysis methods important. The registry does not report the original statistical plan, missing-data procedures, or details of adjustment for clinician-level grouping. Those omissions do not erase the value of the posted descriptive results, but they constrain the strength and scope of conclusions that can be drawn from them.

18. Limitations

Several limitations specific to this trial shape interpretation. First, the primary result concerns baseline, not a comparison of outcomes after one group received SPA and the other did not. It therefore cannot estimate the intervention’s effect. The crossover schedule may make later comparisons informative, but the registry does not report their analysis details or how time period and possible persistence of effects were handled.

Second, the outcome is based on chart documentation and records whether there is evidence that a parent rating scale was sent. Documentation may not perfectly reflect what occurred in practice, and the registry does not report procedures for resolving unclear records or handling missing charts. The measure also captures only one step in the assessment process, rather than whether a scale was returned, reviewed, or used.

Third, chart review was described as covering up to 4 patients per clinician. Patient records may share features because they are associated with the same clinician. The registry does not say whether the reported uncertainty accounts for that grouping. It also does not report how clinicians or patients were selected for review, or whether the reviewed charts represent all eligible care.

Fourth, the study is unmasked, and the primary outcome relies on chart review. The registry does not describe who performed the review, whether reviewers were aware of clinician assignment, or whether a blinded assessment procedure was used. Such details would help assess the possibility of differences in recording or assessment.

Finally, the record does not report the original statistical plan, missing-data methods, multiplicity procedures, or sample-size justification. These omissions limit assessment of whether analyses were prespecified, how secondary findings should be weighted, and whether the posted confidence intervals account for key design features.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

22. Record Summary

The registry describes a completed, randomized, unmasked crossover trial of SPA, a behavioral intervention combining ADHD management education and communication training, collaborative consultation, and performance feedback. The intended population was children aged 5 to 12 years with ADHD in primary care. Intervention clinicians were scheduled to receive SPA first; control clinicians were scheduled to receive no intervention during the first 8 months and SPA during the last 8 months.

The primary outcome was chart evidence that a parent rating scale was sent during the 8-month baseline period. The posted risk difference was -0.029, with a 95% confidence interval from -0.105 to 0.047 and an unadjusted p-value of 0.459. This baseline comparison does not estimate the effect of SPA. The registry does not report the original statistical analysis plan, missing-data procedures, or details needed to assess adjustment for clinician-level grouping.

Clinical Biostats