Non-Parametric Tests: Wilcoxon and Mann-Whitney U

Introduction

Clinical trial data do not always behave like the idealized normal distributions assumed by many classical statistical methods.

Biomarkers may be strongly skewed. Laboratory measurements can contain extreme values. Pain scores and other patient-reported outcomes may be ordinal rather than truly continuous. Pharmacokinetic measurements such as exposure can span several orders of magnitude.

In these situations, a non-parametric test can provide a useful alternative to a conventional parametric test.

Two of the most important non-parametric procedures are the:

• Wilcoxon signed-rank test for paired or one-sample data
• Wilcoxon rank-sum test / Mann-Whitney U test for two independent groups

Parametric vs. Non-Parametric Tests

A parametric test typically makes assumptions about the distribution of the outcome or about the parameters describing that distribution.

For example, the two-sample t-test is commonly motivated by a normal-theory model.

A rank-based non-parametric procedure instead transforms observations into their relative ordering.

However, it is a mistake to think of non-parametric tests as assumption-free.

They still require appropriate study design, independence or pairing as applicable, meaningful ordering of observations, and assumptions specific to the interpretation of the test.

The Two Tests Answer Different Questions

The first decision is whether the observations are paired or independent.

Quick Test Selection Guide

The Wilcoxon Signed-Rank Test

The Wilcoxon signed-rank test is designed for paired observations or a one-sample location problem.

A common clinical application is comparing a patient's measurement before and after treatment.

For patient \(i\), define the paired difference:

The test then considers the absolute magnitudes and signs of the nonzero differences.

The absolute differences are ranked from smallest to largest, and the signs of the original differences are retained.

Example: Pain Score Before and After Treatment

Suppose ten patients have pain scores measured before and after treatment.

Patient 5 has a zero difference.

In the conventional Wilcoxon signed-rank procedure, zero differences are excluded from the ranking calculation.

The remaining differences are ranked according to their absolute values.

Ranking the Differences

Ignoring the zero difference, the absolute differences are:

Ties receive the average of the ranks they would occupy.

The resulting ranking structure is:

The sign of each original difference is then attached to its rank.

The positive and negative ranks are summed separately.

The Wilcoxon Signed-Rank Statistic

Let \(R_i\) denote the rank of the absolute value of the nonzero difference for subject \(i\).

Define:

and:

A common test statistic is the smaller of the two signed-rank sums:

Software may report a related statistic or use a different sign convention, so the exact printed value should always be interpreted according to the software documentation.

Wilcoxon Signed-Rank Hypotheses

The exact formulation depends on the assumptions and the scientific question.

A common two-sided formulation is:

Under a symmetric difference distribution, this can be interpreted as a test of whether the population median difference is zero.

What Does the Wilcoxon Signed-Rank Test Use?

The procedure uses two pieces of information:

• The magnitude of each nonzero difference, through its rank
• The direction of each difference, through its positive or negative sign

This is different from a sign test, which uses only whether each difference is positive or negative and ignores the magnitude information.

Wilcoxon Signed-Rank vs. Sign Test

The Mann-Whitney U Test

The Mann-Whitney U test is used when comparing two independent groups.

It is also commonly called the Wilcoxon rank-sum test.

The two names refer to closely related formulations of the same rank-based comparison.

Clinical Example: Two Treatment Groups

Suppose a clinical trial compares a biomarker response between two independent treatment groups.

There are 10 observations in total.

The Mann-Whitney procedure combines the observations from both groups and ranks them from smallest to largest.

Rank All Observations Together

The rank sum for Treatment A is:

The rank sum for Treatment B is:

If \(n_A=5\), the Mann-Whitney U statistic for Group A is:

Therefore:

The corresponding statistic for Group B is:

Since \(n_A=n_B=5\):

Thus the smaller U statistic is:

A U statistic this extreme indicates that every observation in Treatment A is below every observation in Treatment B in this small example.

What Does the Mann-Whitney U Test Actually Test?

This is one of the most frequently misunderstood points about the test.

The Mann-Whitney U test is fundamentally a test of whether observations from one population tend to be systematically larger or smaller than observations from the other population.

It can be expressed using the probability:

where \(X\) is a randomly selected observation from one group and \(Y\) is a randomly selected observation from the other group.

With appropriate handling of ties, the corresponding probability of superiority is often expressed as:

This quantity is sometimes called the common-language effect size or probability of superiority.

When Can Mann-Whitney Be Interpreted as a Median Comparison?

Suppose the two populations have similarly shaped distributions and differ primarily by a shift in location.

Under that type of assumption, the Mann-Whitney test can reasonably be interpreted as evidence that one population tends to have larger values than the other, and the location difference can be summarized using medians.

If the distributions have substantially different shapes, however, the median alone may not capture what the test is detecting.

Mann-Whitney U and Rank Sums Are Equivalent

The Wilcoxon rank-sum statistic for Group A is:

The corresponding U statistic is:

Thus, once the rank sum is known, the U statistic follows directly.

The two statistics contain the same information.

Relationship Between U and Pairwise Comparisons

Another useful interpretation of \(U\) is that it counts the number of cross-group comparisons in which observations from one group exceed observations from the other group, subject to the convention used for ties.

If:

then there are:

possible cross-group pairs.

A U statistic near the middle of its possible range indicates substantial overlap between the groups.

A U statistic near zero or near \(n_An_B\) indicates strong separation.

The Null Distribution of U

Under the null hypothesis that the two groups are exchangeable with respect to their outcome distributions, the rank assignments have a known distribution.

For sufficiently small samples, an exact test can be used when the data structure permits.

For larger samples, the U statistic is often approximated using a normal distribution.

Normal Approximation for Mann-Whitney U

For independent groups with sample sizes \(n_1\) and \(n_2\), the expected value of \(U\) under the null is:

The variance without ties is:

The standardized statistic is approximately:

A continuity correction may be applied depending on the software and analysis method.

Ties Require Special Handling

Clinical data frequently contain tied values.

For example, patient-reported outcomes may use a limited integer scale:

Many patients can therefore have exactly the same observed value.

When ties occur, observations with the same value receive their average rank.

The variance used for the normal approximation must also be adjusted for ties.

Zero Differences in the Signed-Rank Test

The Wilcoxon signed-rank test also requires special handling of zero differences.

If:

there is no direction to rank.

Such observations are generally omitted from the signed-rank calculation.

This means that the effective sample size for the test can be smaller than the number of paired observations originally enrolled.

Exact vs. Asymptotic Tests

For small samples, exact inference can be particularly useful.

For larger samples, asymptotic methods are usually computationally convenient and often highly accurate.

The exact options available depend on the statistical software and the specific structure of the data.

One-Sided vs. Two-Sided Tests

As with parametric hypothesis tests, non-parametric tests can be formulated as one-sided or two-sided.

For example, suppose lower biomarker values are clinically desirable.

A two-sided alternative is:

A directional alternative could instead be:

The direction must be determined before looking at the observed results if a one-sided confirmatory test is being used.

Clinical Example: Paired Biomarker Measurements

Suppose a biomarker is measured at baseline and Week 12 in the same patients.

The appropriate non-parametric comparison is generally the Wilcoxon signed-rank test because the observations are paired.

The analysis begins by calculating:

The differences are then ranked according to their absolute values.

The signs are retained, and the positive and negative rank sums are compared.

Clinical Example: Two Independent Treatments

Now suppose Treatment A and Treatment B are administered to different patients, and the biomarker is measured at Week 12.

The observations are independent between treatment groups.

The appropriate rank-based comparison is therefore:

It would be incorrect to use the signed-rank test simply because both groups have the same type of measurement.

Ordinal Clinical Outcomes

Non-parametric tests are particularly attractive for ordinal outcomes.

Examples include:

• Severity grades
• Symptom scores
• Functional scales
• Patient-reported outcome categories
• Clinician-rated ordinal assessments

For two independent groups, a Mann-Whitney test can compare the rank ordering of the outcome.

However, investigators should consider whether an ordinal regression model would provide a more informative analysis when the outcome structure and clinical question support it.

Non-Parametric Does Not Mean "Always Better"

A common misconception is:

"If the data are not perfectly normal, use Wilcoxon or Mann-Whitney."

That is too simplistic.

Parametric methods can be remarkably robust, particularly with adequate sample sizes and well-behaved designs.

A rank-based test can also answer a different question from a t-test.

The choice should therefore be based on:

• The measurement scale
• The study design
• The scientific estimand
• The distribution of the outcome
• The presence of outliers
• The sample size
• The assumptions required for the intended interpretation

Wilcoxon Signed-Rank vs. Paired t-Test

Mann-Whitney vs. Two-Sample t-Test

What a P-Value Means

Suppose a Mann-Whitney test produces:

This does not mean there is a 1.8% probability that the null hypothesis is true.

Rather, under the null model and the assumptions of the test, a result at least as extreme as the observed result would be relatively unlikely.

The p-value provides evidence against the null hypothesis.

Effect Size for Mann-Whitney: Probability of Superiority

One useful effect size is the probability that a randomly selected observation from one group exceeds a randomly selected observation from the other.

For example:

An estimate based on the U statistic is:

depending on which group-specific U statistic is used.

If \(\hat{\theta}=0.75\), the interpretation is approximately that a randomly selected observation from the designated group will be larger than a randomly selected observation from the comparison group 75% of the time, counting ties as half.

Rank-Biserial Correlation

Another effect-size measure for the Mann-Whitney procedure is the rank-biserial correlation.

One common formulation is:

The exact sign depends on which group's U statistic is used.

The measure ranges from approximately \(-1\) to \(+1\), with the sign indicating the direction of the group difference.

Effect Size for Wilcoxon Signed-Rank

For paired rank-based analyses, a standardized effect measure can be derived from the test statistic.

A commonly reported measure is:

where \(Z\) is the standardized test statistic and \(N\) is the relevant number of nonzero paired differences.

This should be reported with enough information to make the direction and definition clear.

Reporting Medians and IQRs

Because non-parametric tests are often used for skewed or ordinal data, descriptive statistics frequently include:

For example:

The statistical test can then be reported alongside the descriptive comparison.

Example Clinical Results Statement

A concise clinical-trial report might state:

A stronger report would additionally include an appropriate effect estimate and confidence interval when available.

Confidence Intervals Are Still Important

A p-value answers a hypothesis-testing question.

A confidence interval provides information about the magnitude and uncertainty of the effect.

For rank-based analyses, confidence intervals can be constructed for several useful quantities, including:

• Hodges-Lehmann location shift estimates
• Median paired differences
• Probability of superiority
• Other prespecified rank-based effect measures

The Hodges-Lehmann Estimator

The Hodges-Lehmann estimator provides a useful estimate associated with Wilcoxon-type procedures.

For two independent groups, it is based on the collection of pairwise differences:

The estimator is a median of these pairwise differences, using the appropriate Walsh-average or pseudomedian construction depending on the setting.

For paired data, the estimator is based on the paired differences.

Important Assumptions

The assumptions should be considered carefully rather than treating non-parametric methods as assumption-free.

Wilcoxon Signed-Rank

• Observations are meaningfully paired.
• Pairs are independent of one another.
• The outcome is at least ordinal so that ranking is meaningful.
• The distribution of paired differences is approximately symmetric if the test is interpreted as a test of a median/location shift.
• Differences are appropriately handled when they are zero or tied.

Mann-Whitney U

• The two groups are independent.
• Observations within and between groups are appropriately sampled for the intended inference.
• The outcome is at least ordinal.
• Observations can be meaningfully ranked.
• If interpreted as a pure location comparison, the distributions should have reasonably similar shapes.

Independence Is Critical

One of the most important assumptions is independence.

Suppose a clinical trial contains 100 patients but each patient's biomarker is measured five times.

Those 500 observations are not 500 independent patients.

Simply pooling all measurements and performing a Mann-Whitney test can produce incorrect inference.

Common Clinical Applications

Wilcoxon and Mann-Whitney procedures are frequently encountered in exploratory and clinical research settings involving:

• Biomarker measurements
• Patient-reported outcomes
• Pain scores
• Laboratory values
• Pharmacokinetic parameters
• Time-to-event summaries used descriptively
• Ordinal disease severity scores
• Change-from-baseline analyses
• Small-sample pilot studies

The appropriateness of the test depends on the actual estimand and study design, not simply on the fact that a variable is skewed.

Pharmacokinetic Example

Pharmacokinetic parameters such as AUC and \(C_{\max}\) can be highly skewed.

Suppose two independent formulations are compared.

A Mann-Whitney test may be useful as an exploratory rank-based comparison, depending on the scientific question.

However, pharmacokinetic bioequivalence analyses generally have specific regulatory methods, commonly involving logarithmic transformation and confidence intervals for geometric mean ratios.

Change From Baseline: A Common Trap

Suppose a trial measures a continuous endpoint at baseline and Week 12.

One possibility is to calculate:

and compare the changes between two independent treatment groups using a Mann-Whitney procedure.

That may be reasonable for a specific exploratory question.

However, in a confirmatory clinical trial, baseline-adjusted methods such as ANCOVA or an appropriate mixed model may provide greater efficiency and better alignment with the prespecified estimand.

Therefore, the choice should not be made solely because the change scores are not normally distributed.

Outliers and Rank-Based Tests

One advantage of rank-based procedures is that a very large numerical outlier does not receive proportionally enormous influence simply because of its magnitude.

For example, consider:

The value 1000 is much larger numerically than the other observations, but ranking treats it primarily as the largest observation.

This can make rank-based tests more stable in certain settings.

However, an outlier may indicate:

• A data-entry error
• An assay problem
• A protocol deviation
• A genuinely unusual biological response

It should therefore be investigated rather than automatically neutralized through a non-parametric test.

Common Mistakes

1. Using Wilcoxon signed-rank for independent groups. The signed-rank test is designed for paired observations.
2. Using Mann-Whitney for paired observations. If the same patients are measured twice, the pairing should generally be preserved.
3. Calling Mann-Whitney a test of means. It is a rank-based distributional comparison, not a mean comparison.
4. Automatically calling it a test of medians. That interpretation requires appropriate distributional assumptions.
5. Assuming non-parametric tests have no assumptions. They still require appropriate independence, pairing, ordering, and distributional conditions for certain interpretations.
6. Ignoring ties. Tied observations affect ranking and the variance of asymptotic statistics.
7. Ignoring zero paired differences. Zero differences generally do not contribute ranks to the signed-rank statistic.
8. Reporting only a p-value. A clinically meaningful analysis should also provide descriptive statistics and an effect estimate when possible.
9. Using a non-parametric test simply because a normality test is significant. Test selection should consider the study design, estimand, sample size, robustness, and clinical interpretation rather than relying on one diagnostic test.
10. Ignoring repeated measurements. Repeated observations from the same patient are correlated and should not automatically be treated as independent.

R: Wilcoxon Signed-Rank Test

In R, the base function for the Wilcoxon signed-rank test is wilcox.test().

For paired observations:

before <- c(8, 7, 6, 8, 5, 7, 6, 5, 8, 7)

after <- c(6, 5, 5, 7, 5, 4, 4, 4, 6, 8)

wilcox.test(
  before,
  after,
  paired = TRUE
)

The key argument is:

paired = TRUE

This tells R that the observations should be analyzed as matched pairs.

One-Sample Wilcoxon Test in R

Suppose the scientific question is whether the median or location of a single-sample outcome differs from a prespecified reference value of 50.

x <- c(
  42, 47, 51, 55, 48,
  45, 53, 49, 46, 52
)

wilcox.test(
  x,
  mu = 50
)

Here, mu = 50 specifies the reference location.

Two-Sided vs. One-Sided R Tests

The default alternative in wilcox.test() is two-sided.

wilcox.test(
  x,
  mu = 50,
  alternative = "two.sided"
)

A directional test can be specified with:

wilcox.test(
  x,
  mu = 50,
  alternative = "greater"
)

or:

wilcox.test(
  x,
  mu = 50,
  alternative = "less"
)

The direction must correspond to the scientific hypothesis and the way the data have been coded.

R: Mann-Whitney U Test

For two independent groups, R also uses wilcox.test().

treatment_a <- c(12, 15, 13, 16, 11)

treatment_b <- c(18, 21, 17, 25, 20)

wilcox.test(
  treatment_a,
  treatment_b
)

Although R calls this a Wilcoxon test, the two-sample procedure is the Wilcoxon rank-sum test, which is equivalent to the Mann-Whitney U test.

Using a Formula in R

With a data frame, a formula interface is convenient:

wilcox.test(
  biomarker ~ treatment,
  data = trial_data
)

For independent groups, the grouping variable should identify the treatment groups.

Exact P-Values in R

For suitable small-sample problems, R can calculate an exact p-value.

wilcox.test(
  treatment_a,
  treatment_b,
  exact = TRUE
)

However, exact calculations may not be available or appropriate in the same way when ties or other complications are present.

Always inspect the output and understand the method used.

Confidence Intervals in R

For some Wilcoxon procedures, R can provide a confidence interval for a location parameter such as a Hodges-Lehmann estimate.

wilcox.test(
  treatment_a,
  treatment_b,
  conf.int = TRUE
)

This can be considerably more informative than reporting only the p-value.

Using the Test With Clinical Trial Data

A typical analysis workflow might be:

Wilcoxon Signed-Rank Worked Calculation

Consider five paired differences:

The absolute differences are:

Their ranks are:

Restoring the signs gives:

The positive rank sum is:

The negative rank sum is:

The smaller signed-rank statistic is therefore:

The exact p-value is obtained from the null distribution of the signed-rank statistic rather than simply comparing 6 with a conventional normal critical value.

Why Ranks Matter

Consider two differences:

A mean-based procedure treats the second value as 100 times the first in magnitude.

A rank-based procedure treats them as the smallest and largest observations, respectively.

This is the central reason rank-based methods are less sensitive to extreme magnitudes.

But Ranks Also Discard Information

The same feature that protects the analysis from extreme numerical values can also reduce efficiency.

Suppose the actual numerical magnitudes contain reliable information.

A rank-based procedure compresses those magnitudes into an ordering.

For example:

and:

have very different numerical structures, but their rank ordering is identical.

A parametric method can potentially exploit the additional magnitude information.

Power Considerations

Non-parametric tests are not automatically less powerful than parametric tests.

When data are strongly skewed or contain extreme observations, a rank-based procedure may perform very well.

When the parametric assumptions are approximately satisfied, a parametric test can be more efficient because it uses the actual numerical magnitudes.

Clinical Interpretation Should Come Before the P-Value

Suppose a Mann-Whitney test produces:

This is strong statistical evidence of a distributional difference.

But the clinical importance depends on the magnitude of the difference and the endpoint itself.

A very large trial can produce a small p-value for a tiny effect.

Conversely, a clinically important difference may fail to achieve statistical significance in a small exploratory study.

Multiplicity Still Matters

Using a non-parametric test does not eliminate multiplicity concerns.

Suppose a clinical trial evaluates ten biomarkers and performs a Mann-Whitney test for every biomarker.

Testing many endpoints increases the probability of obtaining at least one small p-value by chance.

Depending on the confirmatory strategy, investigators may need:

• Hierarchical testing
• Multiplicity-adjusted p-values
• False discovery rate control
• Prespecified primary and secondary endpoints
• Other appropriate multiplicity strategies

Missing Data

A Wilcoxon analysis generally requires appropriate outcome information for the observations contributing to the comparison.

Missing data should not simply be handled by silently deleting observations without considering the missingness mechanism and the prespecified analysis strategy.

In confirmatory clinical trials, missing-data handling should be specified in the statistical analysis plan.

Non-Parametric Tests in Small Samples

Small samples are a common reason investigators consider non-parametric tests.

For example, a pilot study might contain only 10 patients per treatment group.

Rank-based procedures can be attractive because their validity does not depend on estimating a large number of distributional parameters.

However, a small sample still means limited information.

A non-significant p-value in a small study does not demonstrate that the treatments are equivalent.

Common Reporting Language

For a Wilcoxon signed-rank analysis:

For a Mann-Whitney analysis:

A stronger report can additionally provide a Hodges-Lehmann estimate, probability of superiority, or another prespecified effect measure.

Decision Tree

Wilcoxon and Mann-Whitney: Side-by-Side

A Practical Analysis Checklist

• Confirm whether the observations are paired or independent.
• Define the primary endpoint and estimand.
• Inspect the outcome distribution.
• Identify ties and zero differences.
• Consider whether the outcome is ordinal, continuous, or discrete.
• Determine whether an exact or asymptotic procedure is appropriate.
• Prespecify the direction of a one-sided test.
• Report descriptive statistics.
• Report the p-value.
• Report an effect estimate when possible.
• Consider a confidence interval.
• Address missing data according to the analysis plan.
• Account for multiplicity when multiple hypotheses are tested.

What to Put in a Statistical Analysis Plan

For a clinical trial using a Wilcoxon or Mann-Whitney procedure, the SAP should clearly define:

• The endpoint
• The analysis population
• The comparison of interest
• Whether observations are paired or independent
• The null and alternative hypotheses
• Whether the test is one-sided or two-sided
• The handling of ties
• The handling of zero differences
• The method for calculating the p-value
• The use of exact or asymptotic inference
• The descriptive statistics
• The effect measure
• The confidence interval method, if applicable
• The handling of missing observations
• Multiplicity adjustments, if required

When a Parametric Method May Still Be Preferable

There are situations in which a t-test or model-based approach may be more appropriate even when the raw data are not perfectly normal.

For example, with a sufficiently large sample and a well-behaved endpoint, the mean may be the scientifically relevant estimand.

In a randomized clinical trial, an ANCOVA or mixed-effects model may also provide better adjustment for baseline values and repeated measurements.

Therefore, "non-normal data" should not automatically trigger a switch to Wilcoxon or Mann-Whitney.

Why the Test Choice Matters

Consider a randomized trial with a continuous primary endpoint.

If the scientific question is:

then a rank-based test may not directly answer the primary question.

If instead the question is:

then a Mann-Whitney procedure may be much more naturally aligned with the estimand.

The statistical test should follow the scientific question rather than the other way around.

Summary of the Worked Examples

The Most Important Distinction

If there is one rule to remember from this tutorial, it is:

The tests are both rank-based, but they solve different problems.

The Most Important Interpretation Point

The second key concept is that these tests are not simply non-parametric versions of the t-test.

The Wilcoxon signed-rank and Mann-Whitney procedures operate on ranks and therefore test distributional or location-related hypotheses.

Under additional assumptions, these can be interpreted as tests of median or location differences.

Without those assumptions, a significant result can reflect differences in distributional shape, spread, or other aspects of the outcome distribution.

Bottom Line

References

Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.
Mann, H.B. & Whitney, D.R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18(1), 50–60.
Hollander, M., Wolfe, D.A. & Chicken, E. (2014). Nonparametric Statistical Methods. 3rd ed. Wiley.
Conover, W.J. (1999). Practical Nonparametric Statistics. 3rd ed. Wiley.
Lehmann, E.L. (1998). Nonparametrics: Statistical Methods Based on Ranks. Prentice Hall.
Hodges, J.L. & Lehmann, E.L. (1963). Estimates of location based on rank tests. Annals of Mathematical Statistics, 34(2), 598–611.