Introduction
The t-test is one of the most familiar statistical methods in biostatistics. Despite its simplicity, it remains useful in clinical research when the scientific question involves comparing a continuous outcome with a reference value or comparing the means of two groups.
Examples occur throughout clinical development. A study may ask whether the mean change in a laboratory measurement differs from zero, whether the mean change in a treatment group differs from a clinically relevant reference value, or whether the mean outcome differs between two treatment groups.
The correct version of the t-test depends primarily on the study design and scientific question, not simply on the fact that the outcome is continuous.
The Three Common t-Test Settings
| Test | Clinical question | Basic comparison |
|---|---|---|
| One-sample t-test | Does the sample mean differ from a specified value? | Mean vs. reference |
| Independent two-sample t-test | Do two independent groups have different means? | Mean A vs. Mean B |
| Paired t-test | Does the mean within-patient change differ from zero? | Mean of paired differences vs. zero |
One-Sample t-Test
Suppose a clinical study measures a continuous biomarker in a group of patients. The investigator wants to determine whether the population mean differs from a specified reference value.
Let the observations be:
The sample mean is:
\[ \bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i \]and the sample standard deviation is:
\[ s=\sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2} \]The one-sample t-test evaluates whether the population mean is equal to a specified value \(\mu_0\).
Hypotheses
For a two-sided test:
\[ H_0:\mu=\mu_0 \] \[ H_A:\mu\neq\mu_0 \]The null hypothesis states that the population mean equals the reference value. The alternative hypothesis states that the population mean differs from it.
For example, suppose a biomarker has a clinically relevant reference value of 50 units. The hypotheses could be:
\[ H_0:\mu=50 \] \[ H_A:\mu\neq50 \]The One-Sample t-Statistic
The test statistic is:
\[ t=\frac{\bar{X}-\mu_0}{s/\sqrt{n}} \]The numerator represents the observed difference between the sample mean and the hypothesized population mean. The denominator is the estimated standard error of the sample mean.
Under the null hypothesis, the statistic follows a t-distribution with:
\[ df=n-1 \]degrees of freedom.
One-Sample Clinical Example
Suppose 25 patients have a mean change in a laboratory measurement of \(-4.2\) units, with a standard deviation of \(8.0\) units. The investigator wants to test whether the mean change differs from zero.
The hypotheses are:
\[ H_0:\mu=0 \] \[ H_A:\mu\neq0 \]The standard error is:
\[ SE=\frac{8.0}{\sqrt{25}}=1.6 \]The t-statistic is therefore:
\[ t=\frac{-4.2-0}{1.6}=-2.625 \]The degrees of freedom are:
\[ df=25-1=24 \]The resulting p-value can then be compared with the prespecified significance level, such as \(\alpha=0.05\).
Confidence Interval for a One-Sample Mean
A confidence interval is often more informative than a p-value alone because it shows the range of population mean values compatible with the data under the chosen confidence level.
A two-sided \(100(1-\alpha)\%\) confidence interval for the population mean is:
\[ \bar{X}\pm t_{1-\alpha/2,\,n-1}\frac{s}{\sqrt{n}} \]For example, a 95% confidence interval is:
\[ \bar{X}\pm t_{0.975,\,n-1}\frac{s}{\sqrt{n}} \]If the confidence interval excludes zero, the corresponding two-sided test of \(H_0:\mu=0\) at the 0.05 significance level will reject the null hypothesis.
Two-Sample t-Test
The independent two-sample t-test is used when the outcome is measured in two independent groups.
A common clinical-trial example is comparison of a continuous endpoint between an experimental treatment arm and a control arm.
| Group | Example | Outcome |
|---|---|---|
| Treatment | Drug A | Change from baseline in biomarker |
| Control | Placebo | Change from baseline in biomarker |
Let:
\[ \bar{X}_1,\quad s_1,\quad n_1 \]represent the sample mean, standard deviation, and sample size for Group 1, and:
\[ \bar{X}_2,\quad s_2,\quad n_2 \]represent the corresponding quantities for Group 2.
The primary quantity of interest is the difference in means:
\[ \Delta=\mu_1-\mu_2 \]Hypotheses
For a two-sided comparison:
\[ H_0:\mu_1-\mu_2=0 \] \[ H_A:\mu_1-\mu_2\neq0 \]A one-sided hypothesis can also be specified when scientifically justified and prespecified.
Pooled Two-Sample t-Test
The traditional pooled two-sample t-test assumes that the two populations have the same variance.
The pooled variance estimate is:
\[ s_p^2= \frac{(n_1-1)s_1^2+(n_2-1)s_2^2} {n_1+n_2-2} \]The pooled standard error for the difference in means is:
\[ SE(\bar{X}_1-\bar{X}_2) = s_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}} \]The test statistic is:
\[ t= \frac{\bar{X}_1-\bar{X}_2} {s_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}} \]with:
\[ df=n_1+n_2-2 \]Welch's Two-Sample t-Test
Welch's t-test does not require the two populations to have equal variances. This makes it a particularly useful default for many two-group comparisons.
The Welch standard error is:
\[ SE= \sqrt{ \frac{s_1^2}{n_1} + \frac{s_2^2}{n_2} } \]The test statistic is:
\[ t= \frac{\bar{X}_1-\bar{X}_2} {\sqrt{ \frac{s_1^2}{n_1} + \frac{s_2^2}{n_2} }} \]The Welch-Satterthwaite approximation gives the degrees of freedom:
\[ df= \frac{ \left( \frac{s_1^2}{n_1}+ \frac{s_2^2}{n_2} \right)^2 }{ \frac{ \left(s_1^2/n_1\right)^2 }{n_1-1} + \frac{ \left(s_2^2/n_2\right)^2 }{n_2-1} } \]The resulting degrees of freedom need not be an integer.
Pooled vs. Welch: Which Should You Use?
| Feature | Pooled t-test | Welch t-test |
|---|---|---|
| Equal variances assumed? | Yes | No |
| Unequal group variances | Potential concern | Handled directly |
| Unequal sample sizes | More sensitive to variance differences | Generally robust |
| Degrees of freedom | \(n_1+n_2-2\) | Welch-Satterthwaite approximation |
| Common modern choice | When equal variance is justified | Often preferred when variance equality is uncertain |
Two-Sample Clinical Example
Suppose a randomized clinical trial compares change from baseline in a continuous biomarker.
| Group | n | Mean change | SD |
|---|---|---|---|
| Drug | 40 | −12.4 | 18.0 |
| Placebo | 40 | −4.8 | 14.0 |
The estimated treatment difference, defined here as Drug minus Placebo, is:
\[ \widehat{\Delta} = -12.4-(-4.8) = -7.6 \]Thus, the estimated mean change is 7.6 units lower in the Drug group than in the Placebo group.
Using Welch's method, the standard error is:
\[ SE= \sqrt{ \frac{18^2}{40} + \frac{14^2}{40} } \] \[ SE\approx3.63 \]The corresponding t-statistic is approximately:
\[ t= \frac{-7.6}{3.63} \approx-2.09 \]The precise p-value depends on the Welch degrees of freedom.
Confidence Interval for the Difference in Means
For an independent two-sample comparison, the confidence interval is centered on the estimated difference:
\[ (\bar{X}_1-\bar{X}_2) \pm t_{1-\alpha/2,df} \times SE \]For Welch's test, the standard error and degrees of freedom come from the Welch calculation.
A confidence interval provides three important pieces of information:
- Direction: whether the estimated difference favors one group or the other.
- Magnitude: the estimated size of the difference.
- Precision: how narrowly the plausible values are concentrated.
Paired t-Test
A paired t-test is appropriate when observations occur in natural pairs. Repeated measurements on the same patient are a common clinical example.
Suppose each patient has a baseline measurement and a Week 12 measurement. Define the within-patient difference:
\[ D_i=X_{i,\mathrm{Week\,12}}-X_{i,\mathrm{Baseline}} \]The paired t-test is then simply a one-sample t-test applied to the differences.
The hypotheses are:
\[ H_0:\mu_D=0 \] \[ H_A:\mu_D\neq0 \]The test statistic is:
\[ t= \frac{\bar{D}-0}{s_D/\sqrt{n}} \]with:
\[ df=n-1 \]Independent vs. Paired Data
| Situation | Appropriate approach |
|---|---|
| Drug group vs. independent placebo group | Independent two-sample t-test |
| Patient baseline vs. same patient's follow-up | Paired t-test |
| Matched subjects with one observation in each condition | Paired t-test |
| One group compared with a fixed reference value | One-sample t-test |
Why Pairing Can Increase Precision
Suppose baseline and follow-up measurements within the same patient are positively correlated.
The variance of a paired difference is:
\[ \operatorname{Var}(D) = \operatorname{Var}(X_{\mathrm{post}}) + \operatorname{Var}(X_{\mathrm{baseline}}) - 2\operatorname{Cov}(X_{\mathrm{post}},X_{\mathrm{baseline}}) \]Equivalently, in terms of the correlation \(\rho\):
\[ \operatorname{Var}(D) = \sigma_{\mathrm{post}}^2 + \sigma_{\mathrm{baseline}}^2 - 2\rho\sigma_{\mathrm{post}}\sigma_{\mathrm{baseline}} \]When baseline and follow-up measurements are strongly positively correlated, the variance of the within-patient difference can be substantially smaller than the variance that would result from treating the observations as independent.
Assumptions of the t-Test
The assumptions depend somewhat on which t-test is being used, but several principles are common.
The t-test is designed for quantitative measurements such as laboratory values, blood pressure, biomarker concentrations, or change from baseline.
For an independent two-sample test, observations between groups should be independent according to the study design.
The t-test relies on the sampling distribution of the relevant mean or mean difference being appropriately represented by the t-distribution.
The pooled two-sample t-test assumes equal population variances. Welch's test does not.
Does the Raw Data Have to Be Perfectly Normal?
No. The t-test does not require every observed value to be exactly normally distributed in practice.
The robustness of the method depends on factors such as sample size, the degree of skewness, the presence of extreme observations, and whether the observations are independent.
For sufficiently large samples, the sampling distribution of the mean can be approximately normal under broad conditions.
Outliers
Extreme observations deserve particular attention because a mean and standard deviation can be sensitive to outliers.
An unusually large observation should not automatically be deleted simply because it makes a p-value less favorable.
In clinical research, investigate whether the observation represents:
- A legitimate biological measurement
- A data-entry error
- A laboratory or measurement problem
- A protocol deviation
- An unusual but clinically plausible response
Any treatment of influential observations should follow the prespecified analysis strategy and appropriate data-review procedures.
One-Sided vs. Two-Sided t-Tests
A two-sided alternative is:
\[ H_A:\mu\neq\mu_0 \]A one-sided alternative might be:
\[ H_A:\mu>\mu_0 \]or:
\[ H_A:\mu<\mu_0 \]The choice should be driven by the scientific question and specified before examining the results.
Relationship Between the t-Test and the Confidence Interval
For a two-sided test at significance level \(\alpha\), the corresponding \(100(1-\alpha)\%\) confidence interval provides an equivalent decision rule for the null value.
For example, when testing:
\[ H_0:\mu_1-\mu_2=0 \]at \(\alpha=0.05\):
- If the 95% confidence interval excludes 0, the two-sided p-value is below 0.05.
- If the 95% confidence interval includes 0, the two-sided p-value is at least 0.05.
The confidence interval, however, communicates substantially more than the binary reject/do-not-reject decision.
Effect Size
A p-value does not quantify the magnitude of a treatment difference. Effect-size measures can provide additional context.
For two groups, one standardized measure is Cohen's \(d\):
\[ d= \frac{\bar{X}_1-\bar{X}_2}{s_p} \]where \(s_p\) is the pooled standard deviation.
A standardized effect can be useful when comparing effects measured on different scales, although the raw treatment difference is often more directly clinically interpretable.
Statistical Significance vs. Clinical Significance
Consider two hypothetical results.
| Scenario | Estimated difference | p-value | Potential interpretation |
|---|---|---|---|
| A | 0.4 units | <0.001 | Statistically significant but potentially clinically small |
| B | 8.0 units | 0.08 | Potentially important but imprecisely estimated |
These examples illustrate why clinical interpretation should consider the estimated effect and its confidence interval rather than relying exclusively on whether \(p<0.05\).
Change From Baseline in Clinical Trials
A common use of t-tests in clinical data is analysis of change from baseline. For patient \(i\):
\[ CHG_i=POST_i-BASE_i \]The mean change in a treatment group is:
\[ \overline{CHG} = \frac{1}{n}\sum_{i=1}^{n}CHG_i \]A one-sample test could evaluate whether the mean change differs from zero.
However, in a randomized controlled trial, the primary treatment comparison is often a comparison between treatment groups rather than simply asking whether one treatment group's change differs from zero.
Why "Significant Within Treatment but Not Between Treatment" Can Happen
Suppose the treatment group has a statistically significant mean change from baseline, while the control group does not.
It is incorrect to conclude automatically that the treatment is superior to control.
The appropriate question is whether the difference between the groups is statistically and clinically supported.
In symbols, the relevant contrast is:
\[ \Delta= \mu_{\mathrm{Treatment}} - \mu_{\mathrm{Control}} \]rather than testing the two within-group changes independently.
Baseline Adjustment and ANCOVA
Although a t-test can compare change from baseline, clinical-trial analyses sometimes use a regression or ANCOVA framework that adjusts for baseline.
A simple ANCOVA model can be written as:
\[ Y_{\mathrm{post},i} = \beta_0 + \beta_1 Treatment_i + \beta_2 Baseline_i + \epsilon_i \]Here, \(\beta_1\) represents the treatment effect conditional on baseline.
The choice between a simple t-test on change and a baseline-adjusted model depends on the endpoint, design, analysis plan, efficiency considerations, and regulatory/statistical expectations.
When a t-Test May Not Be Appropriate
A t-test may be unsuitable or incomplete when the endpoint or study design has features such as:
- Highly skewed continuous data requiring a different analysis strategy
- Ordinal outcomes
- Binary or categorical outcomes
- Time-to-event endpoints
- Repeated longitudinal measurements
- Clustered observations
- Substantial informative missingness
- Complex covariate adjustment
For example, a survival endpoint such as time to disease progression is not appropriately analyzed with a simple two-sample t-test because the statistical structure is fundamentally different.
Nonparametric Alternatives
When a t-test is not appropriate because of the endpoint characteristics or analysis assumptions, nonparametric methods may sometimes be considered.
| Situation | Possible method |
|---|---|
| One-sample continuous/ordinal comparison | Wilcoxon signed-rank test |
| Two independent groups | Wilcoxon rank-sum test |
| Paired observations | Wilcoxon signed-rank test |
These methods do not simply represent "the same t-test without the normality assumption." They generally address different statistical quantities and should be interpreted accordingly.
Implementation in R
One-Sample t-Test
Suppose the variable change contains change from
baseline.
t.test(change, mu = 0)
This tests:
\[ H_0:\mu=0 \]Two-Sample Welch t-Test
Suppose change is the continuous endpoint and
treatment identifies the two groups.
t.test(change ~ treatment,
data = analysis_data,
var.equal = FALSE)
The argument:
var.equal = FALSE
requests Welch's unequal-variance version.
Pooled Two-Sample t-Test in R
t.test(change ~ treatment,
data = analysis_data,
var.equal = TRUE)
The distinction is important because the two commands correspond to different variance assumptions.
Paired t-Test in R
For paired baseline and follow-up measurements:
t.test( analysis_data$week12, analysis_data$baseline, paired = TRUE )
Equivalently, the analysis can be expressed explicitly in terms of patient-level differences:
analysis_data$change <- analysis_data$week12 - analysis_data$baseline t.test(analysis_data$change, mu = 0)
Implementation in SAS
One-Sample t-Test
A one-sample t-test can be performed with PROC TTEST.
proc ttest data=analysis_data h0=0;
var change;
run;
Here, h0=0 specifies the hypothesized population mean.
Two-Sample t-Test
proc ttest data=analysis_data;
class treatment;
var change;
run;
SAS provides results for the relevant two-group comparisons, including tests under different variance assumptions.
Paired t-Test
proc ttest data=analysis_data;
paired week12*baseline;
run;
This evaluates the mean within-patient difference.
Clinical-Trial Reporting
A useful statistical presentation should generally include more than a p-value. For a two-group continuous endpoint, a concise presentation might include:
| Component | Purpose |
|---|---|
| Sample size | Shows the number of analyzed patients |
| Mean | Describes the central tendency |
| Standard deviation | Describes variability |
| Estimated treatment difference | Quantifies the between-group effect |
| 95% confidence interval | Shows precision and plausible effect sizes |
| p-value | Quantifies evidence against the specified null hypothesis |
For example, a clinical report might describe a treatment difference as:
Common Mistakes
1. Using a Two-Sample Test for Paired Data
Treating repeated measurements from the same patient as independent observations ignores the pairing structure.
2. Testing Each Treatment Group Against Zero
In a randomized trial, the scientifically relevant question is often the between-treatment contrast rather than whether each group individually changed from baseline.
3. Automatically Assuming Equal Variances
The pooled test requires an equal-variance assumption. Welch's method avoids that requirement.
4. Reporting Only the p-Value
A p-value does not communicate the size or precision of the treatment effect.
5. Ignoring the Direction of the Contrast
A treatment difference should be defined explicitly, such as:
\[ \Delta= \mu_{\mathrm{Drug}} - \mu_{\mathrm{Placebo}} \]Otherwise, the sign of the reported estimate can be misinterpreted.
6. Calling a Result Clinically Important Because It Is Significant
Statistical significance and clinical importance are different concepts.
7. Changing the Analysis After Seeing the Data
The test type, sidedness, estimand, population, and handling of important data features should be specified according to the statistical analysis plan rather than selected opportunistically after observing the results.
A Practical Decision Framework
Are you comparing one mean with a fixed reference, comparing two groups, or evaluating within-subject change?
If the same patients contribute observations under two conditions, consider a paired analysis rather than an independent two-sample test.
Define precisely what difference or mean is being estimated and the direction of the contrast.
Consider the endpoint distribution, study design, repeated measurements, covariates, missing data, and prespecified analysis methodology.
Use a one-sample, independent two-sample, Welch, pooled, or paired t-test as appropriate.
Present the estimated mean or treatment difference, confidence interval, sample size, and p-value.
Worked Comparison of the Three Tests
| Question | Null hypothesis | Typical test statistic |
|---|---|---|
| Does one mean equal 0? | \(H_0:\mu=0\) | \(\frac{\bar X}{s/\sqrt n}\) |
| Do two independent means differ? | \(H_0:\mu_1-\mu_2=0\) | \(\frac{\bar X_1-\bar X_2}{SE}\) |
| Does mean within-patient change equal 0? | \(H_0:\mu_D=0\) | \(\frac{\bar D}{s_D/\sqrt n}\) |
The Most Important Concept
The most important step in applying a t-test is not calculating the t-statistic. It is identifying the correct comparison.
A useful way to think about the three common settings is:
Is the population mean different from a specified value?
Is the population mean in Group 1 different from the population mean in Group 2?
Is the mean within-pair difference different from zero?
Once the comparison is correctly defined, the mathematics follows naturally.
Key Takeaways
Independent two-sample t-test: compares means from two independent groups.
Welch t-test: compares two independent means without assuming equal population variances.
Paired t-test: analyzes within-pair differences and is equivalent to a one-sample t-test on those differences.
Confidence intervals: show the magnitude and precision of the estimated effect and should generally accompany hypothesis tests.
Clinical significance: should not be inferred from statistical significance alone.
Clinical-trial analysis: the t-test is useful for appropriate continuous endpoints, but more complex designs may require regression, ANCOVA, mixed models, survival analysis, or other methods.
Summary
The t-test remains an important tool for clinical statisticians because many clinical research questions involve means of continuous measurements.
The one-sample t-test evaluates a mean against a specified reference. The independent two-sample t-test compares means between two independent groups, with Welch's version avoiding the equal-variance assumption required by the pooled test. The paired t-test handles naturally paired observations by reducing the analysis to a one-sample test of within-pair differences.
In clinical trials, however, the statistical test should follow the scientific question and prespecified analysis strategy. A p-value alone is rarely enough. The estimated treatment difference, confidence interval, variability, sample size, and clinical relevance should all contribute to interpretation.
Understanding these distinctions allows the t-test to be used appropriately rather than simply applied whenever two means appear in a dataset.