Online Grammatical Reasoning Training for Older Adults: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Online Grammatical Reasoning Training for Older Adults (NCT03661190), a randomized na study. The primary outcome is Change in Executive Function From Baseline.

2. Clinical Question

3. Trial Design

The registry describes a randomized, single-masked, parallel-group trial. It was sponsored by the University of Exeter and is listed as completed. The recorded start date is 2017-03-31, and the primary completion date is 2018-05-31. Enrollment is listed as 6544. The phase is recorded as not applicable.

Participants were assigned to grammatical reasoning training or a Card Pairs control. The training group was encouraged to complete the online task once daily for six weeks. The control task involved matching pictures and was described as offering the same level of engagement without the learning effects. Who was masked, and how masking was maintained, are not reported in the registry record. Details of the randomization procedure, allocation concealment, and how participants entered the study are also not reported in the registry record.

The primary outcome timeframe was baseline to six weeks. For that outcome, the registry reports 1739 participants in the grammatical reasoning group and 1736 in the control group. Those are the group sizes attached to the posted primary outcome values; the record does not explain how they relate to the enrollment figure or identify the analysis population used to obtain them.

The record lists five secondary outcomes: change in numerical working memory from six weeks, change in working memory from baseline, change in reaction time from baseline, change in episodic memory from baseline, and change in an overall composite measure of global cognition. The registry lists six outcome measures in total. Results for these secondary outcomes are not included in the posted result details, and the registry does not report the protocol’s original design assumptions or a full statistical analysis plan.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in Executive Function From Baseline, measured over Baseline to Six weeks.

The primary endpoint is mean change in executive function from baseline to six weeks, measured using the Trailmaking B task-switching test. The test presents participants with 24 circles containing numbers (0-13) or letters (A-L). They select the circles in alternating numerical and alphabetic order, such as 1, A, 2, B, 3, C, until all circles have been selected correctly. The task produces a total score, with a higher score indicating better performance. The stated score range is 0 - 300,000.

The registry identifies this as a validated task-switching test and describes it as a well-established computerized test of executive function. Executive function is relevant to the trial’s broad question about cognition, but the primary endpoint is a performance score on this particular task. It is not a direct measure of every aspect of cognition or day-to-day function. The record says that a negative mean score indicates a decline in performance and that the reported data are mean change per group from baseline to six weeks. It does not report a threshold for a meaningful change or explain how the task score maps onto everyday functioning.

Using mean change gives a group-level summary of the measured score over the specified timeframe. It allows the average change in each group, and the difference between those averages, to be described. The mean alone does not show how individual participants’ scores varied or whether the change was similar across participants.

7. Results Posted in the Registry

• Grammatical Reasoning: mean 5,436 versus 9,671 on control; difference -4,235 (95% CI -4,987 to -3,484), standardized effect d = -0.37; z-test p = <0.001.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary measure is continuous: it is a numerical task score, and the registry summarizes change using a mean. A mean is the arithmetic average of the observed values in a group. Comparing mean changes gives a straightforward summary of the average score movement in each randomized group over the stated timeframe. It does not describe every participant’s result, and it can be affected by unusually high or low values. The registry does not provide a distribution plot or other details about the shape of the scores.

What does the mean difference represent?

The between-group estimate is -4235.0, defined by the posted group means as grammatical reasoning minus Card Pairs control. This estimate summarizes the difference between the two group averages, rather than the change within one person or a percentage change. Its sign depends on the order of the comparison: a negative value means the first named group has a lower mean change than the second on this calculation. The arm-level means are 5436.0 for grammatical reasoning and 9671.0 for control.

The comparison also reports a pooled standard deviation of 11300.0 and a standardized effect size, d, of -0.375. A standardized effect expresses a mean difference relative to the spread of observations, allowing effects measured on different scales to be discussed in a common unit. It does not replace the original-scale estimate, and it does not show whether a difference matters to participants. Details of the method used to calculate the pooled spread and standardized effect are not reported in the registry record.

How should the confidence interval be understood?

The 95% confidence interval for the mean difference runs from -4987.0 to -3484.0. In general, a confidence interval expresses the uncertainty around an estimated quantity using a method with a stated long-run coverage property when its assumptions hold. It is not a range containing 95% of participant outcomes, and it does not mean that there is a 95% probability that the fixed true value lies inside this particular interval. The interval conveys more about the precision and range of plausible estimates than a point estimate alone.

The arm-level confidence intervals are also reported: 4906.0 to 5966.0 for grammatical reasoning and 9139.0 to 10200.0 for control. These intervals describe uncertainty around the group means, not the range of individual scores. The registry labels the outcome dispersion as a 95% confidence interval. It does not report further details about the method used to construct the intervals.

What does the p-value tell us?

The unadjusted p-value is <0.001. In general, a p-value assesses how compatible the observed result is with a statistical model that assumes no difference, conditional on the model and its assumptions. A small p-value is not the probability that the model or null hypothesis is true. It also does not convey the size or practical relevance of the estimate; those require attention to the estimate’s units, interval, and context. The registry does not report an adjusted p-value or a multiplicity procedure.

Randomization is intended, in general, to make groups comparable on average at assignment, reducing the potential for systematic baseline differences to explain a comparison. The registry states that allocation was randomized, but the sequence-generation and concealment methods are not reported. The record labels masking as single, without identifying the masked parties. Thus, the label alone does not explain how expectations or measurement could have been affected.

In general, over-dispersion occurs when observed variation is greater than a statistical model expects. It can make standard errors and confidence intervals too narrow if a model fails to account for the extra variation. The posted primary comparison is summarized using means and confidence intervals, but the registry does not provide enough detail about the analysis model or score distribution to assess whether over-dispersion was examined or relevant.

The results use an unadjusted comparison and the record lists multiple outcomes. In general, testing multiple outcomes can increase the chance of at least one apparently unusual result even if there are no true effects. An analysis plan may specify a primary endpoint, testing order, or adjustment method to address this concern. The registry identifies a primary endpoint and five secondary outcomes, but the multiplicity approach and detailed analysis plan are not reported in the registry record.

Finally, interpreting an estimate depends on how the endpoint was defined, which participants contributed data, and how missing measurements were addressed. The registry gives outcome group sizes, but does not state whether analyses followed intention-to-treat principles, whether any alternative populations were used, or what missing-data method was applied. Those omissions limit assessment of the robustness of the estimate beyond the numerical uncertainty represented by its confidence interval.

10. Confidence Intervals and Precision

The primary mean difference is -4235.0, and its 95% confidence interval extends from -4987.0 to -3484.0. The interval describes uncertainty in the estimated difference between group means, under the assumptions of the method used. Its endpoints are both negative, so the range of estimates represented by this interval is below zero for the grammatical reasoning minus control comparison. This says where the estimate lies on the score scale; it does not establish the practical or clinical importance of the difference.

The interval is substantially more informative than the point estimate on its own because it indicates a range of values compatible with the observed data and the calculation method. It does not show where most individual participants’ changes fell. The arm-level means have their own 95% confidence intervals: 4906.0 to 5966.0 for grammatical reasoning and 9139.0 to 10200.0 for control. These quantify uncertainty in the means, not the full spread of participant outcomes.

The reported group sizes for the primary outcome are 1739 and 1736. The record does not give details about missing observations, the analysis population, or the interval calculation. Those details matter when judging whether the reported interval captures all important sources of uncertainty. Nor does the registry report a threshold that would distinguish a small from a meaningful change on this scale. The estimate is therefore precise as a statistical summary of the posted comparison, but its real-world importance cannot be determined from the interval alone.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size estimates are planning illustrations based on fractions of the observed effect, not the original trial design. They are stated as participants per arm, rather than total events. The assumptions listed for these illustrations are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is the stated fraction of the observed effect. The trial’s own original sample-size assumptions are not reported in the registry record.

The column for 80% power gives the number of participants per arm that the illustration estimates would be needed to detect the chosen effect with 80% power under those assumptions. In plain terms, power is the chance that a study using the specified assumptions would produce a statistically significant result if the chosen effect were the true effect. The 90% power column gives the corresponding per-arm planning number for a higher chance of detecting that effect. These calculations are not observed power.

The rows use the full observed effect, 75% of the observed effect, and 50% of the observed effect. For the full observed effect, the illustration lists 112 participants per arm for 80% power and 150 participants per arm for 90% power. For 75% of the observed effect, the estimates are 199 and 266 participants per arm. For 50% of the observed effect, they are 447 and 599 participants per arm. Smaller effects require more participants to detect under the stated assumptions, which is why the planning estimates rise as the assumed effect becomes smaller.

These values are conditional on the planning assumptions and observed effect used for the illustration. They are not evidence that the trial was designed using those numbers, and they do not describe the power actually achieved by the completed trial. The registry does not report the original sample-size justification.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -4235.290; pooled standard deviation σ = 11298.398.
2. n = 2 × (1.96 + 0.842)² × 11298.398² / -4235.290² = 111.7, rounded up to 112 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -4235.29                      # observed difference in means
sigma <- 11298.3982036614                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(3661190)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 112 (80% power), 150 (90% power)
 75% of observed effect: n per arm = 199 (80% power), 266 (90% power)
 50% of observed effect: n per arm = 447 (80% power), 599 (90% power)
power.t.test: 112.7 
Simulated power at n = 112 per arm: 0.793 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry identifies one primary outcome and five secondary outcomes, for six outcome measures in total. The primary endpoint is change in executive function from baseline to six weeks. The secondary outcomes concern numerical working memory, working memory, reaction time, episodic memory, and an overall composite measure of global cognition. The posted primary result is accompanied by an unadjusted p-value. A procedure for accounting for multiple outcomes or multiple statistical tests is not reported in the registry record.

In general, when many outcomes are examined, the chance of at least one statistically unusual finding can increase. Prespecifying a primary endpoint can help distinguish the central question from additional outcomes, but it does not by itself show whether secondary-outcome testing was adjusted or interpreted under a planned hierarchy. The detailed statistical analysis plan, including any multiplicity strategy, is not reported in the registry record. The record’s listing of 0 analyses does not supply those missing methodological details.

The record reports primary-outcome group sizes of 1739 and 1736. It does not state whether these numbers represent all randomized participants, participants with observed endpoint data, or another analysis population. The handling of missing measurements, withdrawals, and protocol deviations is not reported in the registry record. The record also does not say whether the primary analysis followed intention-to-treat, per-protocol, or another principle, or whether sensitivity analyses were conducted.

These gaps affect how the posted estimate can be evaluated. Missing-data methods and analysis populations can influence the result if outcome availability is related to participants’ characteristics or responses. The registry does not provide enough information to assess that possibility. In general, a complete analysis plan and a clear account of participant flow help readers evaluate selective reporting and the robustness of a result; those details are not reported in the registry record.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

For the serious adverse event reporting, the registry lists 3279 participants at risk in the grammatical reasoning group and 3265 at risk in the Card Pairs control group. It records 0 serious adverse events in each group, 0 other adverse events in each group, and 0 deaths in each group. No leading serious adverse event terms are listed.

These figures describe what is recorded in the registry’s adverse-event fields. They do not by themselves establish that no adverse events occurred outside the reporting definitions, time period, or collection procedures used by the trial. The registry does not describe in the available record how events were solicited, classified, monitored, or attributed, nor does it provide narrative safety information. The number at risk in the adverse-event section differs from the group sizes attached to the primary outcome, and the record does not explain the relationship between those populations.

In general, zero recorded events should be read as an absence of events in the posted count, not as proof that an intervention has no risk. The posted figures do not support conclusions about uncommon events or safety beyond the trial’s recorded follow-up and reporting.

15. What the Result Does, and Does Not, Mean

The registry’s primary result is a group-level comparison of mean change in a task-based executive-function score from baseline to six weeks. It reports a lower mean change for grammatical reasoning than for Card Pairs control, with a mean difference of -4235.0 and a 95% confidence interval from -4987.0 to -3484.0. The unadjusted p-value is <0.001. These figures describe the measured outcome and statistical uncertainty for that comparison.

The result does not, on its own, explain why the group means differed. Random assignment supports a comparative design, but the available record does not report enough detail about implementation, missing data, analysis populations, or the statistical plan to resolve every concern about the estimate. A p-value does not measure the size or importance of the difference, and a confidence interval does not show the distribution of individual responses.

The endpoint is performance on Trailmaking B, not a direct measurement of all cognitive abilities or day-to-day functioning. Although the trial’s broad purpose included cognition and function, the primary result cannot be translated into a claim about everyday independence or functioning without additional evidence. The registry does not provide a threshold for a meaningful change on the score, nor does it report how score differences relate to outcomes that matter in daily life.

The posted comparison also does not establish what would happen over a longer period, in groups outside the studied population, or under different ways of delivering the tasks. The record describes the training as intended to be completed daily for six weeks, but adherence and exposure details are not reported. The results should therefore be understood as findings about the registered six-week score comparison, with the stated uncertainties and limitations, rather than as a broad conclusion about cognitive training generally.

16. Why This Trial Matters Statistically

This trial illustrates several statistical issues common in studies of cognitive training. The intervention and control were both online tasks, and the registry describes the control as intended to provide a similar level of engagement. That design makes the between-group contrast a comparison of two task experiences, rather than a simple comparison of activity with no activity. The degree to which the tasks were matched in practice is not reported in the registry record.

The primary endpoint is a continuous score, so the mean difference gives an interpretable measure in the outcome’s own units. Its confidence interval adds information about precision that a p-value alone cannot provide. At the same time, interpretation of a score difference requires knowledge of scale behavior and meaningful thresholds. The record describes the direction of better performance but does not identify a meaningful-change threshold or link score differences to daily functioning.

The presence of five secondary outcomes alongside a primary endpoint also highlights why an analysis plan matters. A clear prespecified approach helps readers understand which findings were confirmatory and how multiple tests were handled. Those details are not reported in the registry record. Missing-data methods and analysis populations are also not described, limiting the ability to evaluate how the posted outcome sizes relate to enrollment and whether the estimates are robust to incomplete measurements.

Finally, the sample-size display uses planning assumptions based on the observed effect, while the original trial design assumptions are not reported. Distinguishing those retrospective planning illustrations from the trial’s actual design is essential. Together, these features show why statistical interpretation depends not only on an estimate and p-value but also on study design, measurement, precision, multiplicity, and transparent reporting.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several trial-specific details needed to assess the result are not reported in the registry record. The method for generating the random sequence and concealing allocation is not described. Although the trial is labeled single-masked, the masked parties and procedures are not identified. The protocol’s original sample-size justification, detailed statistical analysis plan, and approach to interim analyses are not reported in the registry record.

The registry’s enrollment is 6544, whereas the primary outcome means are reported with group sizes of 1739 and 1736. The record does not explain the difference or state whether the reported outcome counts are the randomized groups, participants with complete endpoint measurements, or another analysis population. Participant flow, adherence to the encouraged daily training, withdrawals, and missing-data handling are not described in the posted information.

The primary measure is a specific task-switching score. The record explains the task and scoring direction, but does not provide a threshold for meaningful change or a direct interpretation in terms of day-to-day function. The listed secondary outcomes broaden the intended cognitive assessment, but their results are not included among the posted outcome details. The record does not say how multiple outcomes were handled statistically.

There is also limited detail about implementation and measurement: the registry does not report how participants’ use of the online tasks was measured, whether the two tasks achieved comparable engagement, or how task assessment was administered across groups. Such information would help readers judge the meaning and generalizability of the group comparison. These limitations do not change the posted numerical result, but they restrict conclusions that can be drawn from it.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Online Grammatical Reasoning Training for Older Adults — NCT03661190.
• Registry record last updated 2026-05-04; status last verified 2026-04.
• Results first posted 2026-05-04.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT03661190 is a completed randomized, single-masked, parallel-group trial sponsored by the University of Exeter. It enrolled 6544 adults in a study of online grammatical reasoning training and cognition and function in people over 50. The registered start date is 2017-03-31, and the primary completion date is 2018-05-31.

The primary outcome was mean change in executive function from baseline to six weeks, assessed using the Trailmaking B task-switching test. The registry reports mean changes of 5436.0 for grammatical reasoning and 9671.0 for Card Pairs control. The between-group mean difference is -4235.0, with a 95% confidence interval from -4987.0 to -3484.0 and an unadjusted p-value of <0.001.

Five secondary outcomes are listed. The record does not report the original sample-size assumptions, a detailed analysis plan, the missing-data approach, or the analysis populations. Adverse-event fields record 0 serious adverse events, 0 other adverse events, and 0 deaths in each group.