This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for A Study Comparing User Experience of Different Delivery Devices for Glucagon (NCT03765502), a randomized phase 1 study. The primary outcome is Percentage of Trained Users That Performed a Successful Administration for Each Device.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT03765502 |
| Phase | Phase 1 |
| Condition | Healthy |
| Allocation | Randomized |
| Design model | Crossover |
| Masking | None (open label) |
| Enrollment | 99 |
| Arms in the primary outcome | 2 |
| Lead sponsor | Eli Lilly and Company |
| Results posted | Yes |
2. Clinical Question
Population
The primary outcome concerns trained users who found both devices and participated in at least one rescue-device simulation. The endpoint description specifies successful administration with both devices in both simulations. Participants with diabetes (PWD) did not have group data collected for this endpoint per protocol. The trial enrolled 99 participants overall, but the primary result applies to the specified trained-user population, not automatically to every enrolled participant.
Intervention
The nasal glucagon device (NG) was an empty device used on a medical manikin in simulations of severe hypoglycemia emergencies. No glucagon was administered to humans. The question is about the administration process and user experience, not the pharmacologic effect of glucagon.
Comparator
The comparator was a commercially available Glucagon Emergency Kit (GEK), delivered intramuscularly to a manikin. As with NG, no drug was administered to human participants. The study compared device use in simulated emergency scenarios.
Primary question
Among trained users, what proportion successfully administered each device in the simulation setting? The registry reports successful administration for 90.3% of the NG group and 15.6% of the GEK group. Those percentages describe performance under the study’s simulation conditions; they do not establish how devices perform in actual emergencies or measure a clinical effect in people receiving glucagon.
3. Trial Design
NCT03765502, titled “A Study Comparing User Experience of Different Delivery Devices for Glucagon,” was a Phase 1 randomized, unmasked crossover simulation study sponsored by Eli Lilly and Company. It enrolled 99 participants and is recorded as completed. The registry lists the start date as 2018-11-19 and primary completion as 2019-01-03. The listed condition was healthy.
The study used medical manikins to simulate severe hypoglycemia emergencies. The NG intervention was an empty nasal device, while GEK was a commercially available kit delivered intramuscularly to a manikin. No drug was administered to humans. The record describes Part A for trained users and participants with diabetes (PWD), lasting approximately 17 days, and Part B for untrained users, lasting approximately nine (9) days. It says each part was separated by approximately 7 days.
The primary outcome concerns trained users’ successful administration of both devices. Its stated timeframe was Part A: Days 8 - 9 and Days 15 - 17. The primary population description includes trained users who found both devices and participated in at least one simulation; it also describes successful administration of both rescue devices for both simulations. PWD group data were not collected per protocol. The registry does not report the sequence in which individual participants used the devices, the details of any crossover balancing, or how period effects were handled.
Randomization and crossover are design features, but the record does not provide enough detail to reconstruct the allocation sequence or participant flow. The numbers contributing to the primary device-specific results were 31 for NG and 32 for GEK. The trial’s original sample-size justification and its prespecified statistical analysis plan are not reported in the registry record.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: Trained users participated in Part A and untrained users participated in Part B.
Simulation 1
| Stage | Part A Trained Users NG/GEK | Part A Trained Users GEK/NG | Part A Person With Diabetes (PWD) | Part B Untrained Users NG/GEK | Part B Untrained Users GEK/NG |
|---|---|---|---|---|---|
| STARTED | 20 | 13 | 33 | 16 | 17 |
| COMPLETED | 20 | 13 | 33 | 16 | 17 |
| NOT COMPLETED | 0 | 0 | 0 | 0 | 0 |
Simulation 2
| Stage | Part A Trained Users NG/GEK | Part A Trained Users GEK/NG | Part A Person With Diabetes (PWD) | Part B Untrained Users NG/GEK | Part B Untrained Users GEK/NG |
|---|---|---|---|---|---|
| STARTED | 20 | 12 | 32 | 16 | 17 |
| COMPLETED | 20 | 12 | 32 | 16 | 17 |
| NOT COMPLETED | 0 | 0 | 0 | 0 | 0 |
Analysis population for the primary outcome, as defined in the registry: All trained user participants who found both devices and participated in at least one simulation of a rescue device. Participants with diabetes (PWD) reporting group data not collected per protocol.
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
Population: All randomized trained user participants, untrained user participants, and participants with diabetes.
| Characteristic | Unit | Part A Trained Users | Part A Participants With Diabetes | Part B Untrained Users | Total |
|---|---|---|---|---|---|
| Participants analysed | 33 | 33 | 33 | 99 | |
| Age, Categorical: <=18 years | Participants | 0 | 0 | 0 | 0 |
| Age, Categorical: Between 18 and 65 years | Participants | 29 | 32 | 33 | 94 |
| Age, Categorical: >=65 years | Participants | 4 | 1 | 0 | 5 |
| Sex: Female, Male: Female | Participants | 23 | 22 | 15 | 60 |
| Sex: Female, Male: Male | Participants | 10 | 11 | 18 | 39 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 0 | 0 | 0 | 0 |
| Race (NIH/OMB): Asian | Participants | 1 | 0 | 0 | 1 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 | 0 | 0 | 0 |
| Race (NIH/OMB): Black or African American | Participants | 17 | 17 | 13 | 47 |
| Race (NIH/OMB): White | Participants | 13 | 15 | 18 | 46 |
| Race (NIH/OMB): More than one race | Participants | 2 | 1 | 2 | 5 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 0 | 0 | 0 | 0 |
| Region of Enrollment: United States | Participants | 33 | 33 | 33 | 99 |
6. Primary Endpoint
The registry lists the primary outcome as Percentage of Trained Users That Performed a Successful Administration for Each Device, measured over Part A: Days 8 - 9 and Days 15 - 17.
| Element | Registry entry |
|---|---|
| Result type | Number |
| Measure of spread | not applicable |
| Unit | percentage of participants |
| Endpoint classification | Binary Endpoint |
The primary endpoint is the percentage of trained users who performed a successful administration for each device. In this study, success refers to completing the simulated administration with a manikin, not to delivering glucagon to a person or resolving a clinical emergency. The registry describes a population of trained users who found both devices and took part in at least one simulation, and specifies successful administration with both devices for both simulations.
This type of endpoint is useful for comparing whether users can complete a device-use task in a standardized scenario. A percentage makes the proportion of successful administrations understandable, while the device-specific denominators show how many participants contributed to each result. The posted results report 90.3% for NG among 31 participants and 15.6% for GEK among 32 participants.
Success is only one part of device use. It does not by itself describe the time taken, perceived ease, preference, performance in real emergencies, or clinical outcomes. The registry lists related secondary outcomes covering administration time, preference, and ease of use. Those outcomes address different aspects of usability and should not be treated as interchangeable with the primary success measure.
7. Results Posted in the Registry
- Glucagon Emergency Kit (GEK): 15.6% (5/32) versus 90.3% (28/31) on control; risk difference -74.7 percentage points (95% CI -91.0 to -58.4), risk ratio 0.17 (0.08–0.39); unadjusted two-proportion z-test p = <0.001.
Risk difference: Glucagon Emergency Kit (GEK)
95% CI: -91.0 to -58.4 percentage points
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Percentage of Untrained Users That Perform a Successful Administration for Each Device | Binary Endpoint | Risk difference | -90.9 pts (-100.7 to -81.1) | n/a |
| Average Time for Trained Users to Successfully Administer One Device Over the Other | Continuous Endpoint | Mean difference | 34.50 (14.19 to 54.81) | 20 per arm |
Percentage of Untrained Users That Perform a Successful Administration for Each Device
- Endpoint type: Binary: the percentage of participants with the event (counts rebuilt from the posted percentages and group sizes).
- Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
- Result:
- Glucagon Emergency Kit (GEK): 0.0% (0/31) versus 90.9% (30/33) on control; risk difference -90.9 percentage points (95% CI -100.7 to -81.1); unadjusted two-proportion z-test p = <0.001.
- Interpretation:
- The 95% confidence interval excludes no difference, so the proportion with the event is statistically lower for Glucagon Emergency Kit (GEK) than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
Average Time for Trained Users to Successfully Administer One Device Over the Other
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Glucagon Emergency Kit (GEK): mean 81.80 versus 47.30 on control; difference 34.50 (95% CI 14.19 to 54.81), standardized effect d = 0.91; z-test p = <0.001.
- Interpretation:
- The 95% confidence interval excludes no difference, so the mean is statistically higher for Glucagon Emergency Kit (GEK) than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
The secondary results with numerical data extend the same simulated-use picture as the primary endpoint: the posted untrained-user success proportions favor NG over GEK, and the trained-user administration-time means are lower for NG than for GEK. The secondary outcomes therefore add information about a distinct user group and about speed of task completion. They are not independent confirmation of performance in clinical emergencies because they remain simulation outcomes within this study.
The registry also lists preference outcomes for participants overall and for PWD, as well as ease-of-use outcomes. Numerical results for those listed measures are not reported in the registry record. Their presence in the outcome list cannot be interpreted as evidence that participants preferred one device or found it easier to use; that requires the actual results and their analysis.
Secondary outcomes generally carry less weight than a prespecified primary outcome when many measures are examined. In general, testing multiple outcomes without adjustment increases the chance that at least one result appears statistically notable by chance. The registry record does not report whether a multiplicity procedure was planned or used, nor does it provide a complete statistical analysis plan. The secondary p-values are reported by the sponsor as McNemar analyses for the success comparisons; the time result is an unadjusted recalculation from posted summary statistics, and a sponsor-reported time analysis is not listed.
The secondary pattern can provide context and suggest questions for further study, but the outcomes should be considered separately, with their denominators, variability, and missing reporting in view. They do not broaden the primary endpoint into a measure of clinical effectiveness or establish that the observed simulated-use pattern applies to every user group.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Percentage of Untrained Users That Perform a Successful Administration for Each Device
- Average Time for Trained Users to Successfully Administer One Device Over the Other
- Average Time for Untrained Users to Successfully Administer One Device Over the Other
- Percentage of Participants (Trained and Untrained Users) That Prefer One Device Over the Other
- Percentage of Participants (PWD) That Prefer One Device Over the Other
- Percentage of Participants Trained and Untrained Users That Find One Device Easy to Use Over the Other
9. Statistical Interpretation
For the primary outcome, successful administration was recorded for 90.3% of 31 trained-user participants using NG and 15.6% of 32 using GEK. The registry’s reported comparison used McNemar’s method and gave p <.0001. A simple unadjusted recalculation from the posted summary statistics gives a risk difference of -0.747 for GEK relative to NG, with p <0.001. The negative direction means the reported success proportion was lower for GEK than for NG in this comparison. The registry’s own analysis and the recalculation both indicate a marked difference in simulated task completion; neither turns a manikin exercise into evidence about clinical outcomes in patients.
The design is described as crossover, so participants may contribute observations under more than one device condition. McNemar’s method is a paired analysis for binary outcomes, making it relevant to a within-participant comparison. The registry does not report the participant-level paired success pattern, so the posted group percentages alone do not reveal how many participants succeeded with both devices, with one device only, or with neither. The reported analysis is therefore important for interpreting the comparison, while the group percentages convey the observed device-specific proportions.
The outcome denominators are not identical: 31 participants are listed for NG and 32 for GEK. The record does not explain the reason for that difference or provide a complete account of missing outcome data. It also states that PWD group data for the primary result were not collected per protocol. These details limit how far the result can be generalized beyond the recorded trained-user outcome population.
The secondary results have a similar broad pattern for untrained users’ successful administration: the registry reports 90.9% for NG among 33 participants and 0.0% for GEK among 31 participants, with a reported McNemar p <.0001. For trained users’ administration time, the posted means were 47.3 seconds for NG and 81.8 seconds for GEK, with standard deviations of 52.0 and 18.6 seconds, respectively. These are separate outcomes in a simulation study. They add context about task completion and time, but they do not establish usability in every setting or a clinical benefit.
The registry lists outcomes for preference and ease of use, but their numerical results are not reported in the registry record. The available results should therefore be read as a focused set of simulated-use findings, not as a complete account of all user-experience outcomes or as proof that either device will perform the same way outside the study scenarios.
10. Statistical Methods Explained
What does a percentage measure?
A percentage of participants with a successful administration describes the share of the analyzed group who met the endpoint definition. In this study, the posted percentages are 90.3% for NG and 15.6% for GEK among trained users. The percentage is descriptive: it tells readers how often the recorded task was completed in each group. It does not explain why a task succeeded or failed, and it is not a measure of drug action because no drug was administered to humans.
The denominators matter alongside the percentages. The registry lists 31 participants for NG and 32 for GEK. A percentage based on a limited group can be sensitive to the outcomes of individual participants. The posted percentages should therefore be interpreted with the study setting and the confidence interval in mind, rather than as exact estimates of performance in all potential users.
Why use McNemar’s method?
The registry reports McNemar’s method for the primary comparison and for the untrained-user success comparison. McNemar’s method is generally used for paired binary outcomes, where the same participant or matched unit has a yes-or-no result under each condition. A crossover design can produce paired observations because participants experience more than one condition. The analysis focuses on discordant pairs: cases where the outcome differs between conditions. Pairs with success under both conditions or failure under both conditions do not provide the same information about the direction of the paired contrast.
The record does not provide the paired outcome counts, so the McNemar result cannot be reconstructed from the marginal percentages alone. The sponsor’s reported primary p-value is p <.0001. The unadjusted recalculation from posted summary statistics is not a substitute for the paired analysis; it is a separate summary-based calculation. The record does not describe further adjustment or the precise implementation of the reported method.
What does a risk difference express?
A risk difference compares two proportions by subtracting one proportion from the other. It is expressed on the probability scale, so its sign depends on which device is treated as the reference and which is being compared. For the primary summary-statistic recalculation, the GEK-minus-NG risk difference is -0.747. The negative sign corresponds to a lower observed success proportion for GEK than for NG. This measure expresses an absolute contrast in the observed proportions; it is distinct from a relative measure such as a risk ratio or odds ratio.
A recalculated estimate and its interval are based on posted summary statistics, not on participant-level records. They can provide a compact description of the observed difference, but they cannot reproduce every feature of a paired crossover analysis. The registry’s own McNemar analysis remains the reported analysis for the comparison.
How should a p-value be understood?
A p-value describes how incompatible the observed data are with a specified null model, under the assumptions of the analysis. It is not the probability that the null hypothesis is true, and it does not measure the size or practical importance of a difference. The reported primary result is p <.0001; the unadjusted summary-based result is p <0.001. Both are reported as thresholds rather than exact p-values. The registry does not report a multiplicity adjustment for the outcomes.
Statistical significance does not remove design or measurement limitations. A strong statistical signal in a manikin simulation addresses the recorded simulated task, not the full set of conditions in a real emergency. Estimates, confidence intervals, participant population, and outcome definition are necessary context for interpreting a p-value.
Why can standard deviation matter for administration time?
The registry reports mean administration times and standard deviations for trained users. A mean is the arithmetic average of the recorded times. A standard deviation describes how dispersed individual observations are around that average. The NG mean was 47.3 seconds with a standard deviation of 52.0 seconds; the GEK mean was 81.8 seconds with a standard deviation of 18.6 seconds. The relatively larger reported spread for NG indicates greater variability around its mean in this sample.
When observations are highly variable, the mean may not describe a typical participant’s time particularly well. Distributional details such as individual values, medians, or a plot of the data can help assess whether a mean is representative. Those details are not reported in the registry record. The registry’s time comparison has a reported recalculated mean difference of 34.5 seconds, with a confidence interval from 14.19 to 54.81 seconds; this is an unadjusted comparison based on summary statistics.
What is over-dispersion, in general?
In general, over-dispersion means that the observed variation is greater than a statistical model assumes. It can make standard errors too small and confidence intervals too narrow if the model does not account for the extra variability. The registry record does not report an over-dispersion assessment, and it is not appropriate to infer one from the posted results. For this study, the reported standard deviations are relevant to the time outcome, while the binary success comparison is reported using McNemar’s method.
In general, paired analyses can be more informative than treating repeated observations as independent because they account for the relationship between measurements from the same participant. The crossover label and the reported McNemar method are consistent with paired comparison, but the registry does not report enough detail to evaluate sequence effects, period effects, or all assumptions of the analysis.
11. Confidence Intervals and Precision
A confidence interval describes a range of values reasonably compatible with the data and the method used to calculate it. It communicates uncertainty around an estimate; it is not a guarantee that the true value lies inside the interval. A wider interval indicates less precision than a narrower interval, all else being equal. Interpretation also depends on whether the interval is based on a method that matches the study design and analysis population.
For the primary recalculated risk difference, the registry-derived interval lies entirely below zero. This is consistent with the negative estimated contrast for GEK relative to NG and with the lower posted GEK success proportion. The interval is not a replacement for the registry’s paired McNemar analysis, and its endpoints are not repeated because the posted values have more than three decimal places.
For the untrained-user success comparison, the recalculated risk-difference interval runs from -1.007 to -0.811. It is also entirely below zero. For trained-user administration time, the recalculated mean-difference interval runs from 14.19 to 54.81 seconds. That interval is above zero for GEK relative to NG, consistent with the posted means being higher for GEK.
These intervals quantify statistical uncertainty under their respective calculation methods. They do not account for every uncertainty in applying simulated results to other users or settings. The primary outcome’s unadjusted interval is based on posted summary statistics, while the registry reports McNemar’s paired analysis without a confidence interval. The record does not report confidence intervals for every secondary outcome, so precision cannot be assessed uniformly across all listed measures.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 6 | 8 |
| 75% of the observed effect | 11 | 14 |
| 50% of the observed effect | 22 | 29 |
The sample-size illustrations are planning estimates, not descriptions of the trial’s original design. The original sample-size justification and design assumptions are not reported in the registry record. The illustrations use a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. They consider an effect equal to 100%, 75%, or 50% of the observed effect. All listed figures are participants per arm; these are not total event counts.
The 80% power column means the illustrated number of participants per arm that would give an 80% chance of detecting the specified effect under the stated planning assumptions, if that effect were the true effect and the assumptions held. The 90% power column expresses the corresponding planning target for a 90% chance. Neither column is a statement about the probability that the observed result is true.
For the primary success outcome, the illustrations range from 6 participants per arm for planning at 100% of the observed effect to 22 participants per arm for 80% power at 50% of the observed effect; the corresponding 90% power figures are 8 and 29 participants per arm. For the trained-user time outcome, the illustrations range from 20 participants per arm at 100% of the observed effect to 77 participants per arm at 50% for 80% power; the corresponding 90% figures are 26 and 102 participants per arm.
The general pattern is that planning for a smaller effect requires more participants to achieve the same power target. These values are sensitive to the observed effect, which can be an unstable basis for future planning, especially when estimated in a limited study. This is not observed power: it is an illustration of sample sizes under specified assumptions, not a retrospective measure of the study’s ability to detect its result.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.
Worked calculation for this trial
- Control proportion p0 = 0.903; treatment proportion p1 = 0.156; average p̄ = 0.530.
- First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.7059 = 1.3835.
- Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.4682 = 0.3941.
- n = (1.3835 + 0.3941)2 / (-0.7470)2 = 5.7, rounded up to 6 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
p0 <- 0.903225806451613 # control proportion
p1 <- 0.15625 # treatment proportion
n_per_arm <- function(power, frac = 1) {
q1 <- p0 + (p1 - p0) * frac # smaller effect = fraction of the observed difference
pbar <- (p0 + q1) / 2
z_b <- qnorm(power)
ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")
# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(3765502)
n <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))
Output from running the code
100% of observed effect: n per arm = 6 (80% power), 8 (90% power)
75% of observed effect: n per arm = 11 (80% power), 14 (90% power)
50% of observed effect: n per arm = 22 (80% power), 29 (90% power)
power.prop.test: 5.7
Simulated power at n = 6 per arm: 0.843 (20000 replicates)
The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| McNemar | not reported | < 0.001 |
The registry reports the sponsor’s primary comparison as a McNemar analysis with a superiority designation and p <.0001. It does not provide an estimate or confidence interval alongside that reported analysis. The simple unadjusted recalculation from the posted summary statistics gives a risk difference of -0.747 for GEK relative to NG and p <0.001. The recalculation and the sponsor-reported analysis point in the same direction, and their reported p-value thresholds are compatible.
The two analyses should not be treated as identical. McNemar’s method is a paired method, whereas the recalculation summarizes the posted device-specific results. The registry does not provide the participant-level paired outcomes needed to reproduce the McNemar analysis. The recalculated estimate and interval therefore offer a descriptive comparison based on summary data, while the reported McNemar p-value is the sponsor’s analysis recorded in the registry.
The registry marks no discrepancy between the recalculation and the reported analysis. There is consequently no conflicting result to resolve. The reported analysis is the relevant record of the sponsor’s method, and the recalculation is an additional unadjusted description rather than a replacement for that method.
15. Multiplicity, Missing Data and Analysis Populations
The registry lists seven outcome measures, including the primary outcome and six secondary outcomes. Multiple outcomes create a multiplicity issue: when many hypotheses are tested, the chance of obtaining at least one apparently notable result by chance can increase. In general, adjustment procedures can control some forms of this error, but they may also change the threshold used to interpret individual results. The registry record does not report a multiplicity adjustment, a hierarchy for secondary outcomes, or a prespecified plan for interpreting multiple tests.
The reported primary analysis uses McNemar’s method and reports p <.0001. The registry also reports McNemar results for the untrained-user success comparison. Other listed outcomes include administration time, preference, and ease of use, but the record does not report numerical results for every listed outcome. Because the full outcome-by-outcome analysis plan is not reported, it is unclear which outcomes were formally tested, whether all planned analyses were posted, or how any multiplicity was addressed.
Missing-data details are also limited. The primary endpoint description defines a trained-user population that found both devices and participated in at least one simulation, while the result denominators are 31 for NG and 32 for GEK. The registry does not explain the difference in denominators, describe reasons for missing outcomes, or report a missing-data analysis. It explicitly states that PWD group data were not collected per protocol for the primary endpoint. That is a defined omission, not a basis for inferring how PWD participants would have performed.
The record does not report the original statistical analysis plan, interim analyses, or the sample-size justification. It also does not provide enough participant-level information to assess the paired outcome pattern or reproduce the sponsor’s McNemar test. These omissions limit independent evaluation of the analysis choices, the handling of incomplete observations, and the role of secondary testing. They do not erase the posted results, but they make their scope and uncertainty important to state.
16. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Part A Trained Users Nasal Glucagon (NG) | 0 | 33 | 0.0% |
| Part A Trained Users Glucagon Emergency Kit (GEK) | 0 | 32 | 0.0% |
| Part A Participants With Diabetes (PWD) | 0 | 33 | 0.0% |
| Part B Untrained Users NG | 0 | 33 | 0.0% |
| Part B Untrained Users GEK | 0 | 33 | 0.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Part A Trained Users Nasal Glucagon (NG) | 0 of 33 |
| Part A Trained Users Glucagon Emergency Kit (GEK) | 0 of 32 |
| Part A Participants With Diabetes (PWD) | 0 of 33 |
| Part B Untrained Users NG | 0 of 33 |
| Part B Untrained Users GEK | 0 of 33 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 5% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Part A Trained Users Nasal Glucagon (NG) | 0 of 33 |
| Part A Trained Users Glucagon Emergency Kit (GEK) | 0 of 32 |
| Part A Participants With Diabetes (PWD) | 0 of 33 |
| Part B Untrained Users NG | 0 of 33 |
| Part B Untrained Users GEK | 0 of 33 |
The registry lists serious and other adverse-event counts for the trained-user NG group, trained-user GEK group, PWD group, and the untrained-user NG and GEK groups. In each listed group, the number at risk was 33, 32, 33, 33, and 33, respectively. The recorded counts for serious adverse events, other adverse events, and deaths were zero in every group.
These figures are event counts during a simulation study in which no drug was administered to humans. They describe what the registry recorded for the listed groups; they do not establish that a device has no risks in real-world use or in clinical treatment. The registry does not list serious adverse-event terms, consistent with the recorded count of zero, and the posted safety information should be understood within the study’s simulated setting and observation scope.
17. What the Result Does, and Does Not, Mean
The central finding is that trained users in this simulation study had a higher recorded successful-administration proportion with the empty NG device than with GEK. The primary percentages were 90.3% and 15.6%, respectively. The registry’s reported paired analysis gave p <.0001. These findings describe the task as performed by the study participants under the specified manikin scenarios.
They do not show that glucagon from one device is more effective in treating hypoglycemia, because no drug was administered to humans. They do not measure patient outcomes, time to recovery, or the consequences of a delayed or unsuccessful administration in an actual emergency. The study’s setting was designed to simulate device use, not to reproduce every feature of real-life emergencies.
The result also does not mean that every user will succeed with one device or fail with the other. The percentages summarize groups, and the registry does not provide individual paired results. The study includes trained users, untrained users, and PWD-related components, but the primary result is specifically for trained users; PWD group data for that endpoint were not collected per protocol. Applying the result to other populations, training conditions, or contexts requires caution.
In general, a statistically strong contrast can coexist with uncertainty about generalizability, measurement, and how results translate to other circumstances. The registry’s secondary outcome list includes preference and ease measures, but their numerical results are not reported in the registry record. Thus, the posted success result should not be taken as a complete account of user experience, nor as a clinical recommendation. It is evidence about observed simulated administration in this randomized crossover study, interpreted alongside its population and reporting limits.
18. Why This Trial Matters Statistically
This study illustrates why the statistical method should match the design. A crossover study can yield paired outcomes because participants use more than one device, and the registry reports McNemar’s method for the binary success comparison. That method uses the paired structure rather than treating all observations as unrelated. At the same time, the registry’s missing paired counts mean readers cannot independently reconstruct the sponsor’s analysis from the percentages alone.
It also shows the value of separating a large observed contrast from its scope. The primary percentages differ substantially in the posted results, and the reported analysis gives a small p-value threshold. Yet the endpoint is a simulated task, the population is specifically trained users, and the study did not administer glucagon to humans. Statistical evidence for a difference in one measured task does not automatically answer broader clinical questions.
The study also highlights the roles of precision and reporting completeness. Confidence intervals help express uncertainty around estimates, while denominators help readers judge how much information underlies each percentage. Reporting the paired outcome counts, analysis population, missing-data handling, and multiplicity approach would make the analysis easier to evaluate. Those details are not fully reported in the registry record.
Finally, planning illustrations based on an observed effect should be distinguished from the original trial design and from observed power. Their assumptions are explicit, but they cannot establish why the study enrolled its participants. Together, these points make the study useful for teaching interpretation of binary endpoints, paired analyses, simulated-use outcomes, and the limits of registry reporting.
19. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
- Participant counts were reconstructed from posted percentages and arm sizes, so small rounding differences are possible.
The primary result is limited to trained users completing a simulated task with manikins. No drug was administered to humans, so the study does not measure clinical response or effectiveness. The reported success percentages should not be generalized automatically to actual emergencies, other training levels, or settings not represented in the study.
The registry calls the design randomized and crossover but does not report the device-use sequence, how crossover order was handled, or whether period effects were assessed. It reports McNemar’s method but does not provide paired outcome counts. Without those counts, the sponsor’s paired comparison cannot be independently reproduced from the marginal success percentages alone.
The primary result uses different device-specific denominators, 31 for NG and 32 for GEK, and the registry does not explain that difference or describe missing-outcome handling. PWD group data for the primary endpoint were not collected per protocol. The result therefore cannot be read as a complete estimate across every participant group involved in the study.
Reporting is incomplete for the full user-experience picture. The record lists preference and ease-of-use outcomes, but their numerical results are not reported in the registry record. The time outcome has substantial reported variability for NG, and individual-level distributions are not available in the registry record. The original sample-size rationale, detailed statistical analysis plan, multiplicity approach, and interim analyses are also not reported. These limitations affect how fully the methods and broader implications can be assessed, even though the primary simulated-use contrast is clearly described.
20. Related Tutorials
21. Related Statistical Calculators
22. Sources
- ClinicalTrials.gov: A Study Comparing User Experience of Different Delivery Devices for Glucagon — NCT03765502.
- Registry record last updated 2020-03-20; status last verified 2019-06.
- Results first posted 2020-03-20.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
23. Record Summary
NCT03765502 is a completed Phase 1 randomized crossover simulation study sponsored by Eli Lilly and Company, with 99 participants enrolled. It compared an empty nasal glucagon device with a commercially available Glucagon Emergency Kit in manikin simulations; no drug was administered to humans.
For trained users, successful administration was reported for 90.3% of 31 participants using NG and 15.6% of 32 using GEK. The registry reports a McNemar comparison with p <.0001. The untrained-user success outcome shows the same broad direction in the posted results. Trained-user administration-time results are also reported, while numerical preference and ease-of-use results are not reported in the registry record.
The results describe simulated device use, not clinical treatment outcomes. Important details about the analysis plan, missing data, multiplicity, and crossover implementation are not reported in the registry record.