One Key Question: Pilot Study at NorthShore: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for One Key Question: Pilot Study at NorthShore (NCT03947788), a randomized na study. The primary outcome is Receipt of Any Reproductive Health Counseling.

2. Clinical Question

This is a question about patient-reported receipt of counseling, not about pregnancy outcomes, the quality of counseling, or whether patients followed any advice. The pilot assessed whether implementing the screening tool was associated with differences in counseling reported by patients. The registry record also lists contraceptive counseling and preconception counseling as separate secondary outcomes, allowing those counseling categories to be considered individually.

3. Trial Design

This was a randomized, parallel-group pilot study, listed as Phase II. Practices were assigned to an intervention or control condition: intervention practices received One Key Question training, while control practices continued usual care during the study period. Masking was listed as none. The registry record describes half of participating sites as randomly chosen for the intervention, so the allocation was at the practice level, not the individual patient level.

The study included a first phase in which patients at six NorthShore University HealthSystem practices were surveyed to establish baseline counseling rates. Phase II assessed the effects of implementing One Key Question: after intervention practices implemented the tool, patients at intervention and control practices were surveyed about preconception and contraceptive care. The primary outcome timeframe was after the clinic had implemented One Key Question for at least 4 weeks. The registry record does not report the number of practices participating in Phase II or how the Phase I survey data were used in the reported comparison.

The registry lists an enrollment of 143 and reports 69 participants in interventional practices and 74 in control practices for the primary outcome. The study is marked completed. Its recorded start date is 2019-01-17, and its primary completion date is 2021-01-06. The listed condition is reproductive health, and the sponsor is Endeavor Health.

The intervention was a behavioral training program for clinic staff, not a medication or procedure assigned to individual patients. Since staff and patients were grouped within practices, the practice-level assignment matters when interpreting the patient-level outcome. The registry record does not report whether the analysis accounted for similarities among patients attending the same practice. It also does not report the original sample-size justification, the planned statistical methods, or whether interim analyses were conducted.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: The population analyzed included patients with the ability to get pregnant that were between the ages of 18-49 years old. Patients that reported experiencing menopause, hysterectomy, or sterilization were excluded from analysis. Patients that were pregnant were not eligible for participation.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Baseline data was based upon the characteristics of 4 clinics. The 4 clinics continued their participation for the duration of Phase 2 of the study. The interventional as well as the control group each consisted of one family medicine practice, and one OB/GYN practice. At baseline, a total of 69 subjects participated in the interventional group, and a total of 74 subjects participated in the control group.

6. Primary Endpoint

The registry lists the primary outcome as Receipt of Any Reproductive Health Counseling, measured over After the clinic has implemented OKQ for at least 4 weeks.

The primary endpoint was receipt of any reproductive health counseling among patients with the ability to become pregnant who were eligible for the study. It was recorded as a participant count and defined using a yes/no rule: a patient counted as having received counseling if they reported any contraceptive counseling or any preconception counseling. The timeframe was after the clinic had implemented One Key Question for at least 4 weeks.

This combined endpoint captures whether a patient reported receiving at least one of the two counseling types relevant to the study’s focus. Using a combined outcome can describe the overall occurrence of either type of counseling without requiring that every patient receive both. The registry record separately lists receipt of contraceptive counseling and receipt of preconception counseling as secondary outcomes, so the component topics can also be examined individually.

The endpoint is patient-reported receipt, not a direct measure of what was documented in a medical record, the content or quality of counseling, or the patient’s subsequent decisions. It also does not measure pregnancy intention itself or a later health outcome. Those distinctions matter: a difference in reported counseling describes the outcome that was measured, and should not be interpreted as evidence about outcomes that were not measured. The registry record does not report the survey instrument or further details about how responses were collected.

7. Results Posted in the Registry

• Interventional Practices: 78.3% (54/69) versus 51.4% (38/74) on control; risk difference 26.9 percentage points (95% CI 11.9 to 41.9), risk ratio 1.52 (1.18–1.97); unadjusted two-proportion z-test p = <0.001.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Receipt of Contraceptive Counseling

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • Interventional Practices: 65.2% (45/69) versus 44.6% (33/74) on control; risk difference 20.6 percentage points (95% CI 4.7 to 36.6), risk ratio 1.46 (1.08–1.99); unadjusted two-proportion z-test p = 0.013.
• Interpretation:
  • The 95% confidence interval excludes no difference, so the proportion with the event is statistically higher for Interventional Practices than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.

Receipt of Preconception Counseling

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • Interventional Practices, Clinics Receiving One Key Question Training: 37.7% (26/69) versus 17.6% (13/74) on control; risk difference 20.1 percentage points (95% CI 5.8 to 34.5), risk ratio 2.14 (1.20–3.83); unadjusted two-proportion z-test p = 0.007.
• Interpretation:
  • The 95% confidence interval excludes no difference, so the proportion with the event is statistically higher for Interventional Practices, Clinics Receiving One Key Question Training than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.

The two secondary outcomes concern the individual counseling topics that contribute to the primary endpoint. In both comparisons, the observed direction favors intervention practices, matching the direction of the composite result. This consistency supports the view that the primary pattern was not confined to a directionally opposing result in one of the listed counseling categories. It does not show how much each category contributed to the primary outcome, because the registry record does not report participant-level overlap between the categories.

The secondary comparisons are related to, but not independent of, the primary result. The primary definition counts a participant who reported either contraceptive counseling or preconception counseling. The secondary endpoints examine those topics separately. A patient may potentially be represented in both component outcomes, but the registry record does not report how often that occurred. The secondary results therefore add detail about the measured counseling categories, while the primary outcome remains the broad summary of receipt of either form.

The secondary p-values are unadjusted. When several outcomes are examined, the chance of observing at least one apparently strong result can be affected by the number of comparisons. The registry record does not report whether a multiplicity adjustment was planned or used, or how the secondary outcomes were prioritized. Their results should consequently be treated as supportive descriptive evidence rather than as independent confirmatory findings. This caution does not erase the observed pattern; it limits how strongly the separate secondary comparisons can be interpreted.

The registry reports five outcome measures overall but posts two secondary counseling outcomes. No reported analyses are listed for those secondary endpoints. Details such as model choice, adjustment for practice-level grouping, missing-response handling, or sensitivity analyses are not reported in the registry record. The findings can be read as consistent group contrasts on the stated patient-reported measures, but the posted record does not support stronger conclusions about the underlying analysis or about counseling beyond the measured categories and timeframe.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Receipt of Contraceptive Counseling
• Receipt of Preconception Counseling

9. Statistical Interpretation

10. Statistical Methods Explained

The primary outcome is binary: each eligible participant either counted as reporting any reproductive health counseling or did not. For a binary outcome, a count describes how many participants in a group met the yes definition, while the group size gives the denominator for interpreting that count. The primary comparison reports counts for intervention and control practices and expresses their contrast as a risk difference. The registry record does not report a more detailed statistical analysis plan.

What does a risk difference describe?

A risk difference is the difference between the proportions with the outcome in two groups. A positive risk difference in this comparison means the observed proportion was higher in the intervention group than in the control group. It is an absolute measure of the contrast, unlike a ratio measure. The reported confidence interval describes a range of values compatible with the data under the method used to construct it. The registry record does not state the confidence level or the precise method used to calculate the interval, so those details should not be inferred.

How do relative risk and odds ratio differ?

The relative risk compares the proportion with the outcome in one group with the proportion in the other. The odds ratio compares the odds of the outcome between groups. Both are relative measures, and values greater than one indicate a higher observed outcome frequency in the intervention group for this comparison. They answer related but not identical questions. When outcomes are common, an odds ratio can be farther from one than the corresponding relative risk, so the two should not be treated as interchangeable. The registry reports both measures for the primary outcome and for the secondary outcomes.

What does the p-value tell the reader?

The posted primary p-value is unadjusted. A p-value describes how compatible the observed data are with a specified no-difference model under the assumptions of the analysis; it is not the probability that the null hypothesis is true, nor does it measure the size or importance of an effect. A small p-value does not correct for design limitations, missing-data concerns, or the fact that more than one outcome was considered. The registry record does not report the analysis model or any multiplicity adjustment.

Why does practice-level randomization matter?

Practices, rather than individual patients, were randomized. Patients attending the same practice may share features of care or clinical workflow. As a general statistical consideration, outcomes within the same practice may therefore be more alike than outcomes from unrelated practices. If that dependence is ignored, uncertainty estimates can be too narrow. Appropriate cluster-aware methods can account for practice-level assignment and within-practice similarity. The registry record does not report whether such methods were used, how many practices contributed to the primary comparison, or how the allocation process was implemented beyond the description of random selection of intervention sites.

The patient counts are not the number of independently randomized units: the randomized units were practices. This distinction is important for judging precision and for planning a later study. The posted participant counts describe how many patient responses contributed to each arm’s outcome, but they do not establish how much independent information was contributed by the practices. Without the number of practices and the analysis method, the degree to which clustering affected the reported interval cannot be assessed from the registry record.

What is the scope of the posted analysis?

The registry record reports outcome counts and group comparisons, but it does not list reported analyses for the primary or secondary endpoints. It does not report adjusted estimates, a prespecified model, handling of missing responses, or analysis populations beyond the eligibility description for the primary outcome. The posted results therefore support interpretation of the stated group contrasts, but do not allow readers to evaluate unreported modeling choices or sensitivity analyses.

In general, a well-described analysis would clarify the unit of assignment, the unit of outcome measurement, the method for handling practice-level grouping, the treatment of missing outcome responses, and whether the analysis followed a prespecified plan. These are especially relevant for a randomized study in which the intervention is delivered to practices and outcomes are collected from patients. Their absence from the registry record is a reporting limitation; it does not establish that a particular method was or was not used.

11. Confidence Intervals and Precision

The primary risk-difference confidence interval lies entirely on the positive side, consistent with a higher observed proportion reporting counseling in intervention practices. Its width reflects uncertainty around the estimated contrast: a narrower interval would indicate greater precision, while a wider interval would leave a broader range of compatible contrasts. The registry record does not state the confidence level or the interval-construction method, so the interval should be interpreted as posted rather than assigned an unstated convention.

The relative-risk interval runs from 1.182 to 1.965, and the odds-ratio interval runs from 1.641 to 7.087. Both intervals are entirely above one, consistent with the direction of the primary risk-difference result. The odds-ratio interval spans a wider range of relative values than its point estimate alone conveys. Measures on different scales should not be compared by interval width alone; the risk difference, relative risk, and odds ratio express different aspects of the contrast.

Precision depends on more than the number of patient responses. Because practices were randomized, the number of practices and the similarity of outcomes within practices can affect how much independent information the study contributes. Those details, along with the method used to account for practice grouping, are not reported in the registry record. The participant counts are stated, but the uncertainty in the posted estimates cannot be fully assessed without those design and analysis details.

The intervals describe uncertainty in the estimated comparison for the measured endpoint. They do not capture every source of uncertainty, such as limitations in outcome measurement or the representativeness of participating practices. Nor do they show whether the observed difference would persist over time: the primary outcome was assessed after implementation for at least 4 weeks, and longer-term follow-up is not reported in the registry record.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm for 80% power and participants per arm for 90% power. They are not total events. The illustration uses the observed primary effect as its reference and gives planning values for detecting 100%, 75%, and 50% of that observed effect. The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The trial’s own original sample-size assumptions and justification are not reported in the registry record.

For planning for 100% of the observed effect, the figures are 49 participants per arm for 80% power and 65 participants per arm for 90% power. For planning for 75% of the observed effect, they are 91 participants per arm for 80% power and 121 participants per arm for 90% power. For planning for 50% of the observed effect, they are 210 participants per arm for 80% power and 281 participants per arm for 90% power.

In plain terms, the 80% power column gives the estimated participants needed in each arm under the stated assumptions for a study designed to have an 80% chance of detecting the specified effect if that effect is present. The 90% power column describes the corresponding planning target for a higher chance of detection. Smaller effects are harder to distinguish from random variation and therefore require more participants under these assumptions. These values are planning illustrations, not observed power, and they do not establish that the pilot’s practice-level design was adequately powered.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.514; treatment proportion p₁ = 0.783; average p̄ = 0.648.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.6754 = 1.3237.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.6480 = 0.5454.
4. n = (1.3237 + 0.5454)² / (0.2691)² = 48.2, rounded up to 49 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.513513513513513                          # control proportion
p1 <- 0.782608695652174                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3947788)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 49 (80% power), 65 (90% power)
 75% of observed effect: n per arm = 91 (80% power), 121 (90% power)
 50% of observed effect: n per arm = 210 (80% power), 281 (90% power)
power.prop.test: 48.2 
Simulated power at n = 49 per arm: 0.805 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry lists a primary counseling outcome and two secondary counseling outcomes. The secondary comparisons have unadjusted p-values, and the registry record does not report a multiplicity strategy, a hierarchy of tests, or a prespecified rule for interpreting the secondary outcomes. When multiple outcomes are evaluated, unadjusted p-values do not account for the number of opportunities to observe a small value. The secondary findings therefore carry less confirmatory weight than a single prespecified primary comparison would carry on its own.

The registry record does not report how missing survey responses were handled, whether outcome data were complete for all eligible participants, or whether any imputation or sensitivity analysis was used. It does not report participant flow by practice or the number approached, eligible, and responding. The primary outcome population is described through eligibility criteria, but a full analysis population definition and the handling of exclusions beyond those criteria are not reported in the registry record.

The record also does not report a statistical analysis plan, the method used to account for practice-level assignment, adjusted analyses, interim analyses, or a sample-size justification. Those omissions matter because the intervention was delivered at the practice level while the outcome was measured for patients. The registry does not state whether the group comparisons accounted for within-practice similarity or other covariates.

These are reporting gaps, not evidence that a particular procedure was omitted in the conduct of the study. They limit independent assessment of how the posted estimates were obtained and how sensitive they might be to choices about missing data, multiplicity, or practice grouping. The unadjusted comparisons remain interpretable as the contrasts recorded in the registry, but conclusions should stay close to those posted results.

15. What the Result Does, and Does Not, Mean

The posted primary result describes a higher observed frequency of patient-reported receipt of any reproductive health counseling in intervention practices than in control practices. The confidence interval for the risk difference is entirely positive, and the unadjusted p-value is < 0.001. These statistics describe the evidence and precision for a difference in the measured outcome; they do not establish that the difference is clinically important or that the training caused it.

The study was randomized at the practice level, which is relevant when interpreting the comparison. However, the registry record does not report enough analysis detail to determine how the practice-level design was incorporated into the estimates and uncertainty. The pilot setting and the unreported number of Phase II practices also limit assessment of how broadly the observed pattern applies. The result should be understood in relation to participating practices and the surveyed patients, not assumed to represent every clinical setting.

The measured endpoint is patient-reported counseling after implementation for at least 4 weeks. It does not establish counseling quality, appropriateness, patient understanding, changes in pregnancy intention, or later reproductive health outcomes. The record also does not provide long-term follow-up information. A statistical difference in reports of counseling should not be translated into conclusions about outcomes that were not measured.

The separate secondary results are directionally consistent with the primary comparison, but they are unadjusted and relate to multiple outcomes. They help describe the pattern across the counseling topics, while the absence of a reported multiplicity plan means they should not be treated as independent confirmation. Overall, the registry supports a descriptive statement about the reported group difference in counseling within this pilot; it does not settle questions about mechanism, durability, generalizability, or effects on health outcomes.

16. Why This Trial Matters Statistically

This trial raises an important design issue for clinical researchers: the intervention was delivered to practices, while the outcome was measured from individual patients. That distinction determines the relevant randomized unit and can affect uncertainty estimates. A study may have many patient responses but comparatively fewer independent practice units; both levels are important when assessing design and analysis.

The outcome also illustrates the difference between a composite and its components. The primary measure combines reports of either contraceptive or preconception counseling, while the secondary outcomes distinguish those topics. Considering both levels can help describe the overall outcome and its components, but overlapping components and multiple testing require care in interpretation. The registry record does not report participant-level overlap or a multiplicity strategy.

Finally, the study provides a practical example of why effect estimates, confidence intervals, and p-values answer different questions. The estimate describes the direction and size of the recorded contrast, the interval conveys uncertainty, and the p-value summarizes compatibility with a no-difference model under analysis assumptions. None of these measures resolves missing reporting about clustering, survey response, or the original design rationale. The case shows why statistical interpretation depends on both the numerical result and the clarity of the trial record.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several limitations specific to this study affect interpretation. The primary endpoint was based on patients’ reports of counseling, and the registry record does not report the survey instrument, response process, or whether responses were verified against clinical documentation. The outcome therefore describes reported receipt rather than a documented assessment of counseling content or quality.

The outcome was assessed after practices had implemented One Key Question for at least 4 weeks. The registry record does not report a longer follow-up period or whether results changed over time. It also does not report the number of practices contributing to the Phase II comparison, the distribution of participants across practices, or practice-level outcome summaries. Those details are particularly relevant because randomization occurred at the practice level.

The registry record does not report the handling of missing responses, participant flow, adjusted analyses, or a method for accounting for clustering. It also does not report how Phase I baseline surveys were incorporated into the Phase II comparison, if at all. The original sample-size justification and statistical analysis plan are not reported in the registry record. These gaps limit evaluation of the precision and robustness of the posted contrasts.

Finally, this was a pilot study in practices associated with NorthShore University HealthSystem. The posted record does not establish how representative the participating practices or surveyed patients were of other settings. The findings concern the stated patient-reported counseling measures and should not be extended to unmeasured outcomes or populations without additional evidence.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: One Key Question: Pilot Study at NorthShore — NCT03947788.
• Registry record last updated 2021-12-03; status last verified 2021-12.
• Results first posted 2021-11-09.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT03947788, titled One Key Question: Pilot Study at NorthShore, is a completed randomized parallel-group Phase II pilot study sponsored by Endeavor Health. The recorded study dates are 2019-01-17 to 2021-01-06 for start and primary completion. Practices received One Key Question training or continued usual care, and patient-reported counseling was assessed after implementation for at least 4 weeks.

The primary outcome was receipt of any reproductive health counseling among eligible patients with the ability to become pregnant. The registry reports 54 of 69 participants in intervention practices and 38 of 74 in control practices with the outcome, with an unadjusted p-value of < 0.001. The posted secondary outcomes address contraceptive counseling and preconception counseling separately. Details of the statistical analysis plan, missing-data methods, and handling of practice-level grouping are not reported in the registry record.