Notes · StatisticsUK · A-Levels

One and two sample non-parametric tests

This topic covers distribution-free tests used when the assumptions of the parametric tests fail: the sign test for a median, the Wilcoxon signed-rank test for a median or matched pairs, and the Wilcoxon rank-sum (Mann-Whitney) test for two independent samples. It develops how to rank data, handle ties, and choose the appropriate test.

5 sections~13 min reading time3 competenciesLevel Standard 2 · Advanced 3

T·161616 / 18

Exam profile

AO1 · Carry out the sign test and the rank-based non-parametric testsAO2 · Choose the appropriate non-parametric test and justify itAO3 · Apply and interpret distribution-free tests on real data

Operators:testrankfinddetermineconcludecomment

basic level

Non-parametric tests are principally full A-Level (Paper 3) content within the statistical-enquiry strand.

higher level

The full A-Level expects the sign test, both Wilcoxon tests, the handling of ties, and a reasoned choice of test.

Depth

Text

Contents · 5 sections

One and two sample non-parametric tests

Why non-parametric tests?#

StandardPearson Edexcel 9ST0, Topic 15 (Paper 3)

Choosing a hypothesis test

Fig. 1A guide to choosing a test: parametric versus non-parametric, then by design.

Non-parametric (distribution-free) tests make no assumption that the data come from a particular distribution such as the normal. They are the right choice when the parametric assumptions fail — for markedly skewed data, small samples from an unknown distribution, ordinal data, or data with outliers that would distort a mean. Because they typically work with signs or ranks rather than the raw values, they are resistant to outliers and applicable to ordinal measurements.

The price of this robustness is a modest loss of power WHEN the parametric assumptions actually hold: if the data really are normal, a

t

-test uses more of the information and is slightly more likely to detect a true effect than the corresponding rank test. The choice is therefore a trade-off — use the parametric test when its assumptions are defensible, and the non-parametric test when they are not.

The decision tree below maps the common choices. The first question is whether the data justify a parametric test (roughly normal, quantitative) or call for a non-parametric one (ranks, non-normal, outliers). Within the non-parametric branch, the second question is the design: one sample or matched pairs points to the sign or Wilcoxon signed-rank test, while two independent samples points to the Wilcoxon rank-sum test.

These tests generally concern the MEDIAN rather than the mean, because the median is the natural measure of centre for skewed or ordinal data. Framing the hypotheses in terms of the median, and matching the test to the design, are the first steps of any non-parametric analysis and are explicitly assessed.

Worked example

Selecting a test

Twelve judges rank two products on a 1-to-10 preference scale, and the two sets of ordinal scores are to be compared for the same judges. Which test is appropriate?

Data type
The scores are ordinal, so a non-parametric test is appropriate.
Design
The same judges rate both products, so the data are paired.
Choice
A paired non-parametric test — the sign test or the Wilcoxon signed-rank test on the differences.

Result: Because the data are ordinal and paired, a sign or Wilcoxon signed-rank test on the differences is appropriate.

Typical mistakes

Using a $t$ -test on clearly non-normal small-sample data.
Framing hypotheses about the mean when the median is the natural centre.

Active revision

A small sample of skewed reaction times is to be tested for a change in centre. Explain which family of test is appropriate and why.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The sign test#

StandardPearson Edexcel 9ST0, Topic 15 (Paper 3)

Signed differences for the sign test

Fig. 2Differences from the hypothesised median: nine positive and one negative, an imbalance the sign test assesses.

The sign test is the simplest non-parametric test, using only the SIGNS of the differences from a hypothesised median. To test

H_0

: median

= m_0

, subtract

m_0

from each observation, discard any zeros, and count the positive signs. Under

H_0

a value is equally likely to be above or below the median, so the number of positive signs follows a binomial distribution

B(n, 0.5)

, where

n

is the number of non-zero differences.

The number line below shows the differences from the hypothesised median for a sample: nine lie above (positive) and one below (negative). If the median really were

m_0

, positives and negatives would be roughly balanced, so a large imbalance is evidence against

H_0

. The test statistic is the smaller of the counts of positive and negative signs, and the p-value is a binomial tail probability.

For a two-tailed test the binomial tail is doubled, because an imbalance in either direction counts. With nine positive and one negative sign (

n = 10

), the smaller count is 1, and

P(X \leq 1) = \dfrac{1 + 10}{2^{10}} = 0.0107

under

B(10, 0.5)

, so the two-tailed p-value is

2 \times 0.0107 = 0.0215

. Being below 5%, this rejects

H_0

The sign test's great virtue is that it needs almost no assumptions — only that the observations are independent — which makes it applicable to ordinal data and robust to outliers. Its weakness is low power, because throwing away the SIZES of the differences and keeping only their signs discards information; the Wilcoxon signed-rank test recovers some of that information.

\text{Under } H_0: \text{median} = m_0, \quad (\text{number of } + \text{ signs}) \sim B(n, 0.5)

The sign test

Count signs of the differences from $m_0$ ; the count is binomial with $p = 0.5$ .

Worked example

A two-tailed sign test

Ten non-zero differences from a hypothesised median give 9 positive and 1 negative sign. Test at the 5% level whether the median differs from $m_0$ .

Hypotheses
$H_0$ : median $= m_0$ against $H_1$ : median $\neq m_0$ (two-tailed).
Test statistic
Under $H_0$ , the number of positives is $B(10, 0.5)$ ; the smaller count is 1.
Tail probability
$P(X \leq 1) = \dfrac{1 + 10}{1024} = 0.0107$ , so the two-tailed p-value is $2 \times 0.0107 = 0.0215$ .
$\text{p-value} = 2 \times \frac{11}{1024} = 0.0215$
Conclude
Since $0.0215 < 0.05$ , reject $H_0$ : there is evidence the median differs from $m_0$ .

Result: The two-tailed p-value 0.0215 is below 0.05, so reject $H_0$ : the median appears to differ from $m_0$ .

Typical mistakes

Including zero differences in $n$ ; they should be discarded.
Forgetting to double the binomial tail for a two-tailed test.

Active revision

Twelve differences from a hypothesised median give 10 positive and 2 negative signs. Carry out a two-tailed sign test at the 5% level, given $P(X \leq 2) = 0.0193$ for $B(12, 0.5)$ .

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The Wilcoxon signed-rank test#

AdvancedPearson Edexcel 9ST0, Topic 15 (Paper 3)

Ranking for the Wilcoxon signed-rank test

Fig. 3Ranking the absolute differences: T- = 1 (the single negative) and T+ = 35, so T = 1.

The Wilcoxon signed-rank test improves on the sign test by using the SIZES of the differences as well as their signs. The differences from the hypothesised median (or the paired differences) are taken, their absolute values are ranked from smallest to largest, and the ranks are then attached to the original signs. The test statistic

T

is the smaller of

T_+

(the sum of ranks of positive differences) and

T_-

(the sum for negative differences).

The ranking is shown in the table below. Absolute differences are ranked 1 (smallest) upward, with tied values sharing the mean of their positions; here the single negative difference receives rank 1, so

T_- = 1

and

T_+ = 35

. As a check,

T_+ + T_- = \dfrac{n(n + 1)}{2} = \dfrac{8 \times 9}{2} = 36

, which the values satisfy.

The statistic

T = \min(T_+, T_-) = 1

is compared with a critical value from Wilcoxon tables indexed by

n

; the null hypothesis is rejected when

T

is LESS THAN OR EQUAL to the critical value (small

T

means the ranks pile up on one side). For

n = 8

at the two-tailed 5% level the critical value is 3, and since

1 \leq 3

H_0

is rejected.

Because it uses both sign and magnitude, the Wilcoxon signed-rank test is more powerful than the sign test while still being distribution-free, needing only that the differences are symmetric about the median under

H_0

. Ties in the absolute differences are handled by shared mean ranks, and any zero differences are discarded before ranking — the standard conventions the examination expects.

T = \min(T_+, T_-), \qquad T_+ + T_- = \frac{n(n + 1)}{2}; \quad \text{reject } H_0 \text{ if } T \leq T_{\text{crit}}

The Wilcoxon signed-rank statistic

Rank the absolute differences; sum ranks by sign; reject for a small $T$ .

Worked example

A Wilcoxon signed-rank test

Eight paired differences are -1, 2, 3, 4, 5, 6, 7, 8. Test at the two-tailed 5% level whether the median difference is zero (critical value 3 for n = 8).

Rank absolute values
$|d|$ = 1, 2, 3, 4, 5, 6, 7, 8 rank as 1 to 8; the negative difference $-1$ has rank 1.
Sum by sign
$T_- = 1$ and $T_+ = 2+3+4+5+6+7+8 = 35$ (and $1 + 35 = 36 = \tfrac{8 \times 9}{2}$ ).
Test statistic
$T = \min(35, 1) = 1$ .
$T = \min(T_+, T_-) = \min(35, 1) = 1$
Conclude
Since $T = 1 \leq 3$ , reject $H_0$ : the median difference is not zero.

Result: $T = 1 \leq 3$ , so reject $H_0$ : there is evidence of a non-zero median difference.

Typical mistakes

Rejecting for a large $T$ ; the Wilcoxon signed-rank test rejects for a SMALL $T$ .
Ranking the signed differences instead of their absolute values.

Active revision

Eight paired differences give $T_+ = 31$ and $T_- = 5$ . State the test statistic and, using a critical value of 3 (n = 8, two-tailed 5%), give the conclusion.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The Wilcoxon rank-sum (Mann-Whitney) test#

AdvancedPearson Edexcel 9ST0, Topic 15 (Paper 3)

The Wilcoxon rank-sum test (equivalent to the Mann-Whitney test) compares two INDEPENDENT samples to test whether they come from populations with the same median. The two samples are pooled, all the values are ranked together from smallest to largest, and the ranks belonging to one sample are added to give the rank-sum statistic

W

. If the two populations are alike, the ranks of each sample should be well mixed; if one tends to be larger, its ranks — and hence its rank-sum — will be systematically high.

Consider small samples A = 4, 7, 9 and B = 5, 8, 11. Pooling and ranking gives 4 (rank 1, A), 5 (2, B), 7 (3, A), 8 (4, B), 9 (5, A), 11 (6, B), so the rank-sum for A is

W_A = 1 + 3 + 5 = 9

and for B is

W_B = 2 + 4 + 6 = 12

. As a check the two sum to

\dfrac{N(N + 1)}{2} = \dfrac{6 \times 7}{2} = 21

The rank-sum of the smaller sample (or a specified sample) is compared with critical values from Wilcoxon rank-sum tables indexed by the two sample sizes. A rank-sum far from the value expected under equal populations — either much smaller or much larger — is evidence against

H_0

. Care is needed to use the correct sample's rank-sum and the correct row of the table.

The test assumes only that the two samples are independent and drawn from populations of the same shape differing (if at all) by a shift in location. It is the distribution-free counterpart of the two-sample

t

-test, and it is the right choice for comparing two independent groups of skewed or ordinal data, where a

t

-test's normality assumption is untenable.

W = \text{sum of the pooled ranks of one sample}, \qquad W_A + W_B = \frac{N(N + 1)}{2}

The Wilcoxon rank-sum statistic

Rank the pooled data; sum one sample's ranks; compare with tabulated critical values.

Worked example

Forming the rank-sum statistic

Samples A = 4, 7, 9 and B = 5, 8, 11 are independent. Pool and rank the data and find each rank-sum.

Order the pooled data
4, 5, 7, 8, 9, 11 receive ranks 1 to 6.
Identify sources
Ranks 1, 3, 5 belong to A (4, 7, 9); ranks 2, 4, 6 belong to B (5, 8, 11).
Rank-sums
$W_A = 1 + 3 + 5 = 9$ and $W_B = 2 + 4 + 6 = 12$ .
$W_A = 9, \quad W_B = 12, \quad W_A + W_B = 21$
Interpret
$W_A = 9$ is compared with the tabulated critical value for samples of sizes 3 and 3; the check $9 + 12 = 21$ confirms the ranking.

Result: $W_A = 9$ and $W_B = 12$ (summing to 21); $W_A$ is compared with the rank-sum critical value for the two sample sizes.

Typical mistakes

Ranking each sample separately instead of pooling before ranking.
Using the wrong sample's rank-sum or the wrong row of the table.

Active revision

Two independent samples of sizes 4 and 5 are pooled and ranked. Explain how to form the rank-sum statistic and what it is compared with.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Choosing a test and interpreting results#

AdvancedPearson Edexcel 9ST0, Topic 15 (Paper 3)

Selecting the right test follows the decision tree from the first section, and the reasoning must be explicit. Identify the data type (quantitative and roughly normal, versus ordinal or non-normal) and the design (one sample, matched pairs, or two independent samples), then name the test and justify it. A common exam demand is to explain WHY a non-parametric test is preferred over a

t

-test in a given context.

Among the non-parametric tests, the sign test is the most robust but least powerful, the Wilcoxon signed-rank test is more powerful for paired or single-sample data with symmetric differences, and the Wilcoxon rank-sum test handles two independent samples. Where the data would also support a parametric test, mentioning the power trade-off shows the mature understanding the higher marks reward.

The conclusion is always contextual and framed in terms of the median. 'There is sufficient evidence at the 5% level that the median differs' says more than a bare rejection, and it should be tied to the substantive question — did the treatment change the outcome, do the two groups differ. The direction of any effect, read from which side the signs or ranks favour, completes the interpretation.

Finally, the limitations should be acknowledged: non-parametric tests detect differences in location but say little about the size of an effect, small samples give low power, and, as ever, a significant result is evidence of a difference, not of its cause. This honest framing within the statistical enquiry cycle is exactly the AO3 outcome the topic targets.

Worked example

Recommending and framing a test

A study measures pain scores (ordinal) for two independent groups given different treatments. Recommend a test and outline the conclusion.

Data and design
The scores are ordinal and the two groups are independent.
Choose the test
The Wilcoxon rank-sum (Mann-Whitney) test compares the two independent groups without assuming normality.
Conclusion form
If the rank-sum falls beyond the critical value, conclude there is evidence that the two treatments' median pain scores differ; state the direction.
Caveat
The test shows a difference in location, not causation or effect size.

Result: Use the Wilcoxon rank-sum test for the two independent ordinal samples, concluding about a difference in median pain score with the appropriate caveats.

Typical mistakes

Choosing a two-independent-sample test for paired data (or vice versa).
Concluding about the mean when the test concerns the median.

Active revision

A researcher compares recovery times for two independent groups of patients, with strongly skewed data. Recommend a test, justify it, and describe how you would conclude.

Active recall

Recall the key points — then reveal.

Sources: GCE AS and A level subject content (Statistics) (Department for Education / Ofqual)

References & sources

Sources

Pearson Edexcel

Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification

Department for Education / Ofqual

GCE AS and A level subject content (Statistics)

One and two sample non-parametric tests

5 sections~13 min reading time3 competenciesLevel Standard 2 · Advanced 3

One and two sample non-parametric tests

Key points

Selecting a test

Key points

A two-tailed sign test

Key points

A Wilcoxon signed-rank test

Key points

Forming the rank-sum statistic

Key points

Recommending and framing a test

Sources

One and two sample non-parametric tests

Key points

Selecting a test

Key points

A two-tailed sign test

Key points

A Wilcoxon signed-rank test

Key points

Forming the rank-sum statistic

Key points

Recommending and framing a test

Sources