Notes · StatisticsUK · A-Levels

Analysis of variance

This topic uses one-way analysis of variance (ANOVA) to test whether several population means are equal. It partitions the total variation into between-groups and within-groups components, builds the ANOVA table of sums of squares, mean squares and the F-statistic, compares it with the F-distribution, and interprets the outcome, its assumptions and an effect size.

5 sections~12 min reading time3 competenciesLevel Standard 1 · Advanced 4

T·171717 / 18

Exam profile

AO1 · Build an ANOVA table and compute the F-statisticAO2 · Use the F-distribution and interpret the outcomeAO3 · Apply one-way ANOVA to compare groups and comment on assumptions and effect size

Operators:testcalculatepartitiondetermineconcludecomment

basic level

One-way ANOVA is principally full A-Level (Paper 3) content within the statistical-enquiry strand.

higher level

The full A-Level expects the partition of variation, the ANOVA table, the F-test, its assumptions and an effect-size interpretation.

Depth

Text

Contents · 5 sections

Analysis of variance

Comparing several means#

StandardPearson Edexcel 9ST0, Topic 17 (Paper 3)

Comparative box plots of three groups

Fig. 1Three groups with medians (and means) 7, 10, 13 and similar spread; the separation suggests the means differ.

When three or more groups are to be compared, running many two-sample

t

-tests is a mistake: each test carries a chance of a Type I error, and doing several inflates the overall false-positive rate. Analysis of variance solves this with a SINGLE test of the null hypothesis that all the group means are equal,

H_0

\mu_1 = \mu_2 = \cdots = \mu_k

, against the alternative that at least one mean differs.

The name is apt: ANOVA compares means by analysing VARIANCES. Its insight is that if the group means really are equal, the spread of the group means about the overall mean should be no larger than would be expected from the natural variation within the groups; if the group means differ, the between-group spread will be inflated. The test compares these two sources of variation.

The box plots below show three groups with medians (and means) 7, 10 and 13 and similar spreads. The clear separation of the boxes relative to their widths is visual evidence that the group means differ — precisely the comparison ANOVA formalises by contrasting the variation BETWEEN the groups with the variation WITHIN them.

One-way ANOVA has a single factor (the grouping variable) with several levels (the groups), and one measured response. It generalises the two-sample

t

-test to more than two groups, and reduces to it when there are exactly two. Recognising this structure — one factor, several groups, one response — signals that ANOVA is the appropriate analysis.

H_0: \mu_1 = \mu_2 = \cdots = \mu_k \quad\text{against}\quad H_1: \text{at least one mean differs}

The ANOVA hypotheses

A single test of equality of all group means, avoiding many pairwise tests.

Worked example

Setting up an ANOVA

Three fertilisers are compared for crop yield across independent plots. State the hypotheses and the structure of the analysis.

Factor and levels
The single factor is fertiliser, with three levels (the three fertilisers); the response is yield.
Hypotheses
$H_0$ : $\mu_1 = \mu_2 = \mu_3$ against $H_1$ : at least one mean yield differs.
Method
Use one-way ANOVA, comparing between-group and within-group variation in a single test.

Result: One-way ANOVA tests $H_0$ : $\mu_1 = \mu_2 = \mu_3$ against 'at least one differs', avoiding multiple pairwise tests.

Typical mistakes

Running multiple two-sample $t$ -tests and inflating the Type I error rate.
Writing $H_1$ as 'all means differ' rather than 'at least one differs'.

Active revision

Explain why comparing four teaching methods with six separate two-sample $t$ -tests is statistically unsound, and what should be done instead.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Partitioning the sum of squares#

AdvancedPearson Edexcel 9ST0, Topic 17 (Paper 3)

ANOVA splits the total variation in the data into two additive parts. The total sum of squares

\text{SST} = \sum\sum (x_{ij} - \bar{x})^2

measures the spread of every observation about the grand mean

\bar{x}

. It decomposes as

\text{SST} = \text{SSB} + \text{SSW}

, where SSB is the between-groups sum of squares and SSW the within-groups sum of squares.

The between-groups sum of squares

\text{SSB} = \sum n_i(\bar{x}_i - \bar{x})^2

measures how far the GROUP means

\bar{x}_i

lie from the grand mean, weighted by group size; it is large when the groups are well separated. The within-groups sum of squares

\text{SSW} = \sum\sum (x_{ij} - \bar{x}_i)^2

measures the spread of observations about their OWN group mean; it reflects the natural, unexplained variation.

For three groups A = {5, 7, 9}, B = {8, 10, 12}, C = {11, 13, 15}, the group means are 7, 10, 13 and the grand mean is 10. Then

\text{SSB} = 3[(7-10)^2 + (10-10)^2 + (13-10)^2] = 3(9 + 0 + 9) = 54

, and

\text{SSW} = 8 + 8 + 8 = 24

(each group contributes

(-2)^2 + 0^2 + 2^2 = 8

), giving

\text{SST} = 54 + 24 = 78

The logic of the test is now visible: SSB captures variation EXPLAINED by the grouping, while SSW captures variation WITHIN groups that the grouping does not explain. If

H_0

is true, both estimate the same underlying variance; if the group means differ, SSB is inflated relative to SSW. Turning the sums of squares into comparable variance estimates is the next step.

\text{SST} = \text{SSB} + \text{SSW}, \quad \text{SSB} = \sum n_i(\bar{x}_i - \bar{x})^2, \quad \text{SSW} = \sum\sum (x_{ij} - \bar{x}_i)^2

Partition of the total sum of squares

Total variation splits into between-groups (explained) and within-groups (unexplained).

Worked example

Partitioning the variation

For A = {5, 7, 9}, B = {8, 10, 12}, C = {11, 13, 15} (group means 7, 10, 13; grand mean 10), find SSB, SSW and SST.

Between groups
$\text{SSB} = 3[(7-10)^2 + (10-10)^2 + (13-10)^2] = 3(9 + 0 + 9) = 54$ .
Within groups
Each group gives $(-2)^2 + 0^2 + 2^2 = 8$ , so $\text{SSW} = 8 \times 3 = 24$ .
Total
$\text{SST} = 54 + 24 = 78$ .
$\text{SST} = \text{SSB} + \text{SSW} = 54 + 24 = 78$

Result: $\text{SSB} = 54$ , $\text{SSW} = 24$ and $\text{SST} = 78$ .

Typical mistakes

Forgetting the group-size weight $n_i$ in the between-groups sum of squares.
Measuring within-group deviations from the grand mean instead of the group mean.

Active revision

For groups A = {5, 7, 9}, B = {8, 10, 12}, C = {11, 13, 15}, compute SSB, SSW and SST.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The ANOVA table and the F-statistic#

AdvancedPearson Edexcel 9ST0, Topic 17 (Paper 3)

The ANOVA table

Fig. 2One-way ANOVA table for the three-group data: F = 27/4 = 6.75 on (2, 6) degrees of freedom.

Each sum of squares is converted to a mean square by dividing by its degrees of freedom. With

k

groups and

N

observations in total, the between-groups degrees of freedom are

k - 1

and the within-groups degrees of freedom are

N - k

. The mean squares are

\text{MSB} = \dfrac{\text{SSB}}{k - 1}

and

\text{MSW} = \dfrac{\text{SSW}}{N - k}

, and the within-groups mean square MSW is an unbiased estimate of the common variance whether or not

H_0

is true.

The F-statistic is the ratio of these two mean squares,

F = \dfrac{\text{MSB}}{\text{MSW}}

. Under

H_0

both mean squares estimate the same variance, so

F

is around 1; if the group means differ, MSB is inflated and

F

grows. The test is upper-tailed: only a large

F

is evidence against equality of means.

These results are laid out in the ANOVA table below. For the three-group data,

\text{MSB} = \dfrac{54}{2} = 27

and

\text{MSW} = \dfrac{24}{6} = 4

, giving

F = \dfrac{27}{4} = 6.75

(2, 6)

degrees of freedom. The table's structure — source, sum of squares, degrees of freedom, mean square, F — is the standard way to present the calculation.

The two degrees-of-freedom values,

k - 1

and

N - k

, index the F-distribution used for the critical value and must be quoted in that order (numerator then denominator). Assembling the ANOVA table correctly, with each sum of squares divided by the right degrees of freedom, is the core computational skill of the topic.

\text{MSB} = \frac{\text{SSB}}{k - 1}, \quad \text{MSW} = \frac{\text{SSW}}{N - k}, \quad F = \frac{\text{MSB}}{\text{MSW}}

Mean squares and the F-statistic

The F-statistic is the ratio of between- to within-groups mean squares, on $(k-1, N-k)$ df.

Worked example

Building the ANOVA table

With $\text{SSB} = 54$ , $\text{SSW} = 24$ , three groups of three, complete the mean squares and the F-statistic.

Degrees of freedom
Between: $k - 1 = 2$ ; within: $N - k = 9 - 3 = 6$ .
Mean squares
$\text{MSB} = \dfrac{54}{2} = 27$ ; $\text{MSW} = \dfrac{24}{6} = 4$ .
F-statistic
$F = \dfrac{27}{4} = 6.75$ on $(2, 6)$ degrees of freedom.
$F = \frac{27}{4} = 6.75$

Result: $\text{MSB} = 27$ , $\text{MSW} = 4$ and $F = 6.75$ on $(2, 6)$ degrees of freedom.

Typical mistakes

Dividing a sum of squares by the wrong degrees of freedom.
Quoting the F degrees of freedom in the wrong order (denominator before numerator).

Active revision

Given $\text{SSB} = 54$ ( $k = 3$ ) and $\text{SSW} = 24$ ( $N = 9$ ), complete the ANOVA table and find $F$ .

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The F-test, assumptions and conclusions#

AdvancedPearson Edexcel 9ST0, Topic 17 (Paper 3)

F-distribution with the critical region

Fig. 3F-distribution (2, 6): reject beyond the critical value 5.14; the test statistic 6.75 lies in this tail.

The F-statistic is compared with the critical value of the F-distribution on

(k - 1, N - k)

degrees of freedom at the chosen significance level. The F-distribution, shown below for

(2, 6)

degrees of freedom, lives on the positive axis and is right-skewed; the rejection region is the upper tail beyond the critical value. For the three-group data,

F = 6.75

exceeds the 5% critical value 5.14, so

H_0

is rejected.

The test rests on three assumptions: the observations are independent, each group is (approximately) normally distributed, and the groups share a common variance (homogeneity of variance). ANOVA is fairly robust to mild departures, especially with equal group sizes, but strong skew or very unequal variances undermine it, in which case a transformation or a non-parametric alternative is preferable.

The conclusion must be stated in context and, crucially, with the right scope. Rejecting

H_0

says that at least one group mean differs from the others — it does NOT identify WHICH, nor how many. Determining which specific groups differ requires further (post-hoc) comparisons; the ANOVA is the omnibus test that licenses looking further.

As with every test, the F-test detects a difference, not a cause, and statistical significance is distinct from practical importance. A very large sample can make a trivial difference in means significant, so the size of the effect — pursued in the next section — should always accompany the verdict for an honest interpretation.

\text{Reject } H_0 \iff F > F_{\text{crit}}(k-1,\, N-k,\, \alpha); \qquad 6.75 > 5.14 \Rightarrow \text{means differ}

The F-test decision

Upper-tailed: a large F rejects equality of means; the test does not say which groups differ.

Worked example

Completing the F-test

For the three-group data, $F = 6.75$ on $(2, 6)$ degrees of freedom. Using the 5% critical value 5.14, conclude the test.

Compare
$6.75 > 5.14$ , so the statistic lies in the rejection region.
Decision
Reject $H_0$ at the 5% level.
$6.75 > 5.14 \Rightarrow \text{reject } H_0$
Conclude in context
There is sufficient evidence at the 5% level that at least one group mean differs.
Scope
The test does not identify which groups differ; further comparisons would be needed, and the assumptions of normality and equal variance should be checked.

Result: $F = 6.75 > 5.14$ , so reject $H_0$ : at least one group mean differs (without saying which).

Typical mistakes

Concluding which specific groups differ directly from a significant ANOVA.
Ignoring the normality and equal-variance assumptions.

Active revision

Given $F = 6.75$ on $(2, 6)$ degrees of freedom with a 5% critical value of 5.14, complete the F-test with a contextual conclusion.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Effect size and interpretation#

AdvancedPearson Edexcel 9ST0, Topic 17 (Paper 3)

A significant F-test tells us the means differ, but not by how much relative to the overall variation. An effect size answers that. The simplest is

\eta^2 = \dfrac{\text{SSB}}{\text{SST}}

, the PROPORTION of the total variation explained by the grouping. For the three-group data,

\eta^2 = \dfrac{54}{78} = 0.69

, so about 69% of the variation in the response is explained by the group — a large effect.

Effect size and significance are complementary. Significance (the p-value or the F-comparison) indicates whether an effect is likely to be real; effect size indicates whether it is large enough to matter. A small

\eta^2

can be statistically significant in a very large study yet be of little practical importance, while a large

\eta^2

in a small study may be practically striking even if only marginally significant.

Reporting both is best practice: 'the group means differ significantly (

F(2, 6) = 6.75

p < 0.05

), with the grouping explaining about 69% of the variation'. This pairs the statistical verdict with a measure of its magnitude, exactly the honest, informative reporting the enquiry cycle demands and the higher marks reward.

Finally, the interpretation returns to the substantive question and its limits: the ANOVA establishes that a difference in means exists and how large it is, but not its cause, and not which specific groups differ. Acknowledging these boundaries — association not causation, omnibus not pairwise — completes a mature analysis and frames what further investigation would be needed.

\eta^2 = \frac{\text{SSB}}{\text{SST}} = \frac{54}{78} = 0.69

Effect size (eta squared)

The proportion of total variation explained by the grouping; here about 69%.

Worked example

Effect size for the ANOVA

For the three-group data with $\text{SSB} = 54$ and $\text{SST} = 78$ , compute $\eta^2$ and interpret it alongside the significant F-test.

Compute
$\eta^2 = \dfrac{54}{78} = 0.69$ (2 s.f.).
Interpret
About 69% of the total variation is explained by the group, a large effect.
$\eta^2 = \frac{54}{78} = 0.69$
Report together
The means differ significantly ( $F(2,6) = 6.75$ , $p < 0.05$ ) and the effect is large ( $\eta^2 = 0.69$ ), so the difference is both real and substantial.

Result: $\eta^2 = 0.69$ : the grouping explains about 69% of the variation, a large and practically important effect.

Typical mistakes

Reporting significance alone with no measure of the size of the effect.
Treating a significant result as automatically important regardless of effect size.

Active revision

For $\text{SSB} = 54$ and $\text{SST} = 78$ , compute the effect size and comment on its practical importance.

Active recall

Recall the key points — then reveal.

Sources: GCE AS and A level subject content (Statistics) (Department for Education / Ofqual)

References & sources

Sources

Pearson Edexcel

Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification

Department for Education / Ofqual

GCE AS and A level subject content (Statistics)

Analysis of variance

5 sections~12 min reading time3 competenciesLevel Standard 1 · Advanced 4

Analysis of variance

Key points

Setting up an ANOVA

Key points

Partitioning the variation

Key points

Building the ANOVA table

Key points

Completing the F-test

Key points

Effect size for the ANOVA

Sources

Analysis of variance

Key points

Setting up an ANOVA

Key points

Partitioning the variation

Key points

Building the ANOVA table

Key points

Completing the F-test

Key points

Effect size for the ANOVA

Sources