The Study Guide Cluster Randomized Control Trial: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for The Study Guide Cluster Randomized Control Trial (NCT03312218), a randomized na study. The primary outcome is Scores on the Certification Examination.

2. Clinical Question

3. Trial Design

The Study Guide Cluster Randomized Control Trial is registered as a randomized, double-masked, parallel trial. It is described as cluster randomized, meaning assignment was organized around groups rather than treating every resident as an independent randomization unit. The registry record does not specify the cluster definition, the number of clusters, how clusters were assigned, or how masking was implemented.

The planned contrast was between a spaced educational intervention and a control condition using the same app and educational content. Intervention-site residents received alerts to adaptively reinforce learning; control-group residents could access identical clinical cases and test questions on demand, with alerts inactivated. This makes the availability of alert-based spacing the stated difference between the conditions. The record does not describe adherence to the alerts, app use, or any departures from the assigned intervention.

The registry lists enrollment of 634 and a continuous primary outcome: examination scores expressed as a mean with standard deviation. The outcome timeframe is two years. The trial is marked completed. The listed start date is 2017-10-18 and the primary completion date is 2019-05-31.

Cluster allocation has consequences for analysis. Residents within the same cluster may have more similar outcomes than residents from different clusters. A statistical analysis generally needs to account for this dependence rather than treating all observations as independent. The registry record notes that a simple individual-level re-analysis would be misleading and indicates that only the sponsor’s own analysis is shown. However, it does not report the details or estimates of that analysis. The record also does not describe the planned sample-size justification or the method used to handle clustering.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Scores on the Certification Examination, measured over 2 years.

The primary endpoint is the score on the written component of the College of Family Physicians of Canada examination, specifically the Short Answer Management Problems section. Scores are described as ranging from 0 to 100, with higher values representing a better outcome. The outcome is measured on a continuous scale, and the registry identifies its summary as a mean with standard deviation. The timeframe is two years.

This endpoint captures performance on a written examination focused on management problems. It is used in the trial as an indicator of clinical knowledge or competence relevant to the educational intervention. The registry’s description frames the broader aim as improving clinical competence among family physicians, an outcome associated in the record with quality of care delivered to Canadians. The examination score is the specific measured outcome; it is not itself a direct measure of patient care or care quality.

The registry describes the minimum observed score as 53.9 and the maximum as 84.4. Those values describe the observed range, not the average score in either group. Group means, standard deviations, the between-group estimate, and its uncertainty are not reported in the registry record. Accordingly, the outcome definition is available, but the posted registry information does not establish how the groups scored relative to one another.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

Statistical interpretation starts with the outcome type and the design. The primary outcome is continuous: an examination score summarized as a mean and standard deviation. A mean describes the central location of scores in a group, while a standard deviation describes how dispersed individual scores are around that group’s mean. General explanation: comparing group means can describe an average difference, but the difference alone is not enough to show how precisely it has been estimated. A confidence interval is commonly used to express the range of values compatible with the data and the analysis assumptions. No confidence interval for this outcome is reported in the registry record.

Why does cluster allocation affect the analysis?

In an individually randomized study, the analysis can often treat each participant as an independent unit, subject to the design and model assumptions. In a cluster-randomized study, assignment occurs at a group level. Participants in one cluster may share experiences, environments, or other influences, so their outcomes may be more alike than outcomes from participants in different clusters. This within-cluster dependence is called correlation.

General explanation: ignoring positive within-cluster correlation can overstate the amount of independent information in the data and understate the uncertainty of a comparison. Appropriate approaches may model clustering directly or adjust uncertainty to reflect the cluster structure. Which approach was used in this trial is not reported in the registry record. The record specifically warns that a simple individual-level re-analysis would be misleading and says that only the sponsor’s own analysis is shown. It does not give that analysis’s model, cluster adjustment, estimate, or uncertainty.

What do a mean and standard deviation tell the reader?

A mean is an average score for a group. A standard deviation describes score variation within that group; it is not the uncertainty in the mean and is not the uncertainty in the difference between groups. To assess a randomized comparison, readers would ordinarily need the group-level results and a between-group estimate with an appropriate measure of uncertainty. Those results are not reported in the registry record. The observed minimum of 53.9 and maximum of 84.4 describe the score range in the study, but do not supply the missing group summaries.

How should the sample-size illustrations be read?

General explanation: a sample-size calculation describes how much information a future or planned study might need under chosen assumptions. It does not describe what was observed, and it is not a retrospective measure of whether an experiment was successful. The registry states that its planning illustration uses the observed effect as its reference, considers a stated fraction of that effect, and assumes a two-sided significance level of 0.05, equal allocation between arms, and a normal approximation. The calculated participant counts are not reported in the registry record.

Such an illustration depends on the effect used as the target and on the assumptions used to translate that target into a required sample size. For a cluster trial, the number and size of clusters and the within-cluster correlation are also important considerations for planning and precision. These details, and the trial’s own sample-size justification, are not reported in the registry record. The planning illustrations therefore should not be described as observed power or as proof that the trial could reliably detect a particular difference.

What can be concluded from the posted statistical information?

The registry supplies the design label, enrollment, endpoint definition, timeframe, and score range. It does not supply the inferential results needed to quantify the intervention-control comparison. In particular, an effect estimate and its uncertainty cannot be reconstructed from the outcome definition and overall range. General statistical principle: interpretation should distinguish the size of an estimated effect from its precision, and both depend on the analysis and the data. Neither is available for the primary outcome in the registry record.

10. Confidence Intervals for Each Arm

No two-arm comparison is shown for this trial, but each arm's mean can be given with a t-based 95% confidence interval computed from the posted mean, spread and number of participants.

R code and output

arm <- c("Intervention", "Control")
n   <- c(281, 241)
m   <- c(73.9, 73.1)
sp  <- c(4.4, 5.1)
sd  <- sp      # posted standard deviations
for (i in seq_along(n)) {
  se <- sd[i] / sqrt(n[i]); h <- qt(0.975, n[i] - 1) * se
  cat(sprintf("CARM|%s|%d|%.3f|%.3f|%.3f|%.3f\n", arm[i], n[i], m[i], sd[i], m[i] - h, m[i] + h))
}

CARM  Intervention  281  73.900  4.400  73.383  74.417
CARM  Control  241  73.100  5.100  72.453  73.747

11. Multiplicity, Missing Data and Analysis Populations

The registry lists one outcome measure and identifies the examination score as primary. It lists zero analyses. The record does not report whether any additional outcomes were collected, whether multiple comparisons were planned, or whether any multiplicity procedure was used. It also does not describe interim analyses, stopping rules, or adjustments to the analysis plan; those details are not reported in the registry record.

Missing-data methods are not described. The registry record does not state how many participants completed the examination, whether outcome data were unavailable for any enrolled residents, or how missing scores were handled. It does not report an analysis population definition, such as which randomized participants were included in the primary analysis. These omissions prevent assessment of whether the reported enrollment corresponds to the population contributing to the primary outcome analysis.

General explanation: missing outcomes can affect a comparison if the availability of measurements is related to assigned group or to the underlying score. The consequences depend on the amount and pattern of missingness and on the analysis assumptions. Without information on outcome availability and handling, the direction or magnitude of any such effect cannot be assessed. Similarly, a single registered primary outcome does not establish that no other analyses were performed; the registry record does not describe the full analysis plan or results.

The record states that a simple individual-level re-analysis would be misleading because of the cluster-randomized design and that only the sponsor’s own analysis is shown. Yet the sponsor’s analysis details and results are not reported in the registry record. The missing information includes the analysis population, handling of missing scores, and the specific method used to account for clustering.

12. What the Result Does, and Does Not, Mean

The available record establishes what comparison was intended: residents at intervention sites received app alerts for spaced education, while the control group had access to the same cases and questions without alerts. It also establishes that the primary outcome was the written examination score at the two-year timeframe. These details explain the question the trial was designed to address, but they do not reveal the answer from the observed results.

The score scale runs from 0 to 100, and the registry reports observed values from 53.9 to 84.4. Those details do not show whether the intervention group had a higher or lower mean than the control group. The group-specific means, standard deviations, estimated comparison, and confidence interval or other uncertainty measure are not reported in the registry record. No conclusion about the size or precision of a difference is supported by the information recorded.

The endpoint is an examination score. It should not be treated as a direct measure of patient outcomes or as proof of changes in clinical practice. The registry’s background statement connects clinical competence with quality of care, but the primary outcome described is examination performance. The record does not report a patient-care outcome.

The completed status and enrollment count do not substitute for outcome findings. Likewise, planning calculations based on an observed effect are not observed power and do not establish whether a comparison was statistically conclusive. The registry does not provide the sponsor’s estimates or enough detail to independently assess the primary analysis. The result therefore remains unquantified in the registry record, rather than indicating either a positive or a negative finding.

13. Why This Trial Matters Statistically

This trial raises a central design issue in clinical and educational research: the unit of assignment can differ from the unit of measurement. The intervention was organized by site or cluster, while the outcome was measured as an examination score for residents. That structure requires attention to dependence among observations when estimating uncertainty. A participant-level calculation that ignores clustering can give an overly optimistic impression of precision, which is why the registry cautions against a simple individual-level re-analysis.

The comparator also has statistical and interpretive value. Both groups had access to the same app-based clinical cases and test questions, while alerts were inactive for the control group. The intended contrast therefore focuses on spaced alerting rather than access to the educational content itself. A clear description of the comparator helps define the effect the trial sought to estimate, though it does not establish the size of that effect.

The entry illustrates the distinction between a registered question and a reported result. It specifies a continuous primary endpoint, a scale, a timeframe, and a cluster-randomized design. However, the outcome estimates and analysis details needed to interpret the comparison are not reported in the registry record. For statistical readers, this absence matters: design labels and enrollment alone cannot indicate the magnitude or precision of an effect.

It also underscores why sample-size illustrations must be kept separate from trial findings. Calculations based on an observed effect under assumed conditions are planning aids, not evidence about the certainty of the trial result. Without the sponsor’s estimate and cluster-adjusted uncertainty, the registered clinical question cannot be answered from this record.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The main trial-specific limitation for statistical interpretation is the lack of reported primary outcome results. Although the record defines the examination score and gives the overall observed range, it does not report group means, standard deviations, a comparison estimate, or an uncertainty interval. The size, direction, and precision of the group contrast are therefore unavailable.

The cluster structure is not described in enough detail to evaluate the analysis. The registry does not report the number or definition of clusters, the distribution of participants across them, or the method used to account for within-cluster dependence. It cautions that a simple individual-level re-analysis would be misleading and indicates that only the sponsor’s own analysis is shown, but the sponsor’s results and analytic details are not reported in the registry record.

Other information needed to assess the primary comparison is also absent. The record does not report outcome availability by group, missing-data handling, the analysis population, adherence to the app alerts, or the trial’s sample-size justification. Baseline characteristics and details of masking are not described. The record lists zero analyses, leaving the statistical procedures and results unspecified.

These limitations concern what can be concluded from the registry entry, not necessarily what was collected or performed during the trial. In particular, absence of a detail from the record does not establish that the procedure was not used. It means that readers cannot evaluate it from the information reported there.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: The Study Guide Cluster Randomized Control Trial — NCT03312218.
• Registry record last updated 2021-02-12; status last verified 2021-01.
• Results first posted 2021-02-12.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

The Study Guide Cluster Randomized Control Trial is registered as a completed, randomized, double-masked, parallel study sponsored by Jewish General Hospital. It lists enrollment of 634 and evaluates app-based spaced educational alerts against the same cases and test questions available on demand without alerts.

The primary outcome is the mean score, with standard deviation, on the Short Answer Management Problems component of the College of Family Physicians of Canada written examination, assessed at two years. Scores are described on a 0-to-100 scale, with higher values representing a better outcome; the recorded minimum and maximum are 53.9 and 84.4.

The registry record does not report group outcome summaries, the sponsor’s estimate, or the statistical analysis details. It cautions that simple individual-level re-analysis would be misleading because of the cluster-randomized design. The primary comparison’s direction and precision therefore cannot be determined from the registry information.