Notes · StatisticsUK · A-Levels

Hypothesis testing, confidence intervals and power

This topic develops the main body of parametric inference for a mean: the z-test and confidence interval when the variance is known, the t-test and t-interval when it is unknown, and the extension to two-sample and paired designs. It treats the equivalence between a confidence interval and a two-tailed test, and the power of a test and the factors that govern it.

5 sections~14 min reading time3 competenciesLevel Standard 2 · Advanced 3

T·131313 / 18

Exam profile

AO1 · Carry out z- and t-tests and construct confidence intervals for a meanAO2 · Choose between one-sample, two-sample and paired designs and interpret intervalsAO3 · Reason about the power of a test and the trade-offs in test design

Operators:testcalculateestimatedetermineconcludecomment

basic level

The AS foundation introduces the z-test and confidence interval for a mean with known variance.

higher level

The full A-Level expects the t-test, two-sample and paired tests, the interval-test equivalence and the power of a test.

Depth

Text

Contents · 5 sections

Hypothesis testing, confidence intervals and power

The z-test and z-confidence interval for a mean#

StandardPearson Edexcel 9ST0, Topic 13 (Paper 2)

Two-tailed rejection region

Fig. 1Two-tailed 5% test: reject H0 if |z| > 1.96; the shaded tails carry 2.5% each.

When the population variance

\sigma^2

is known and either the population is normal or the sample is large, a hypothesis about the mean is tested with the

z

-statistic

z = \dfrac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}

. Under

H_0

\mu = \mu_0

this has a standard normal distribution, so it is compared with the standard normal critical values —

\pm 1.96

for a two-tailed test at the 5% level, as shown by the shaded rejection tails below.

The test measures how many standard errors the sample mean lies from the hypothesised mean. A value of

z

beyond the critical value falls in a tail so improbable under

H_0

that

H_0

is rejected; a value inside the critical values does not. The two-tailed rejection region is the pair of tails whose total area is the significance level, split equally between them.

The corresponding confidence interval is

\bar{x} \pm z\dfrac{\sigma}{\sqrt{n}}

, using the same critical value. This is the direct dual of the test: the values of

\mu_0

that would NOT be rejected by a two-tailed test at level

\alpha

are exactly those inside the

100(1 - \alpha)\%

confidence interval. Computing the interval therefore answers the test and reports the precision in one step.

The conditions must be checked: the variance must genuinely be known (not estimated from the sample), and normality of

\bar{X}

must hold, either because the population is normal or because the sample is large enough for the central limit theorem. When

\sigma

is estimated from a small sample, the

t

-procedure of the next section replaces the

z

z = \frac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}, \qquad \text{CI}: \; \bar{x} \pm z\,\frac{\sigma}{\sqrt{n}}

z-test statistic and confidence interval

The number of standard errors from $\mu_0$ ; the interval uses the same critical value.

Worked example

A two-tailed z-test and its interval

Cartons should contain 500 ml with known $\sigma = 8$ ml. A sample of 16 gives mean 496 ml. Test at the 5% level whether the mean differs from 500, and give a 95% confidence interval.

Hypotheses
$H_0$ : $\mu = 500$ against $H_1$ : $\mu \neq 500$ (two-tailed).
Test statistic
$z = \dfrac{496 - 500}{8/\sqrt{16}} = \dfrac{-4}{2} = -2$ .
$z = \frac{496 - 500}{8/4} = -2$
Compare
$|{-2}| = 2 > 1.96$ , so reject $H_0$ .
Confidence interval
$496 \pm 1.96 \times 2 = (492.1, 499.9)$ ; 500 lies outside, agreeing with the rejection.

Result: $z = -2$ , so reject $H_0$ at the 5% level; the 95% interval (492.1, 499.9) excludes 500, confirming the mean has changed.

Typical mistakes

Using $\sigma$ rather than the standard error $\frac{\sigma}{\sqrt{n}}$ in the denominator.
Comparing $|z|$ with 1.645 (one-tailed) when a two-tailed test needs 1.96.

Active revision

A machine should fill cartons to 500 ml with $\sigma = 8$ . A sample of 16 has mean 496 ml. Test at the 5% level whether the mean has changed.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The t-distribution and the t-test#

AdvancedPearson Edexcel 9ST0, Topic 13 (Paper 2)

The t-distribution compared with the normal

Fig. 2The t-distribution (5 df) has a lower peak and heavier tails than the standard normal.

When the population variance is unknown — the usual case — it is estimated by the sample variance

s^2

, and the extra uncertainty this introduces means the standardised mean no longer follows a normal distribution but a

t

-distribution with

n - 1

degrees of freedom. The test statistic is

t = \dfrac{\bar{x} - \mu_0}{s/\sqrt{n}}

, identical in form to the

z

but using the estimated standard error.

The

t

-distribution is symmetric and bell-shaped like the normal but with heavier tails and a slightly lower peak, as the comparison below shows, because estimating

\sigma

adds variability. The fewer the degrees of freedom, the heavier the tails; as

n

increases the

t

-distribution approaches the standard normal, and for large samples the two are practically indistinguishable.

Critical values come from

t

-tables (or a calculator) indexed by the degrees of freedom

n - 1

, and they are always slightly larger in magnitude than the corresponding

z

-values, reflecting the heavier tails. This makes the

t

-test more conservative than a

z

-test would be, correctly demanding stronger evidence when the variance is only estimated.

The

t

-procedure requires the population to be approximately normal, an assumption that matters most for small samples where the central limit theorem cannot rescue a skewed population. The corresponding confidence interval

\bar{x} \pm t\dfrac{s}{\sqrt{n}}

uses the

t

critical value, and is the interval to quote whenever

\sigma

has been estimated from the data.

t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}, \quad \text{df} = n - 1; \qquad \text{CI}: \; \bar{x} \pm t\,\frac{s}{\sqrt{n}}

t-test statistic and interval

Uses the estimated standard error; $t$ has heavier tails than the normal, indexed by $n - 1$ .

Worked example

A one-tailed t-test

A random sample of 10 measurements from a normal population has mean 52 and sample standard deviation 4. Test at the 5% level whether the population mean exceeds 50 (t-critical for 9 df, one-tailed, is 1.833).

Hypotheses
$H_0$ : $\mu = 50$ against $H_1$ : $\mu > 50$ (one-tailed).
Test statistic
$t = \dfrac{52 - 50}{4/\sqrt{10}} = \dfrac{2}{1.2649} = 1.581$ .
$t = \frac{52 - 50}{4/\sqrt{10}} = 1.581$
Compare
With 9 degrees of freedom the critical value is 1.833; since $1.581 < 1.833$ , do not reject $H_0$ .
Conclude
There is insufficient evidence at the 5% level that the population mean exceeds 50.

Result: $t = 1.581 < 1.833$ , so do not reject $H_0$ : insufficient evidence that the mean exceeds 50.

Typical mistakes

Using $z$ -critical values when $\sigma$ has been estimated by $s$ .
Taking the degrees of freedom as $n$ rather than $n - 1$ .

Active revision

A sample of 10 gives mean 52 and $s = 4$ . Test at the 5% level whether the population mean exceeds 50.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Two-sample and paired tests#

AdvancedPearson Edexcel 9ST0, Topic 13 (Paper 2)

Comparing two population means uses the difference of the sample means. For independent samples with known variances, the standardised statistic is

z = \dfrac{\bar{x}_1 - \bar{x}_2}{\sqrt{\sigma_1^2/n_1 + \sigma_2^2/n_2}}

, because the variance of a difference of independent means is the SUM of their variances. With unknown variances estimated from the samples, the corresponding

t

-test is used with a suitable degrees-of-freedom rule.

A paired design is fundamentally different and must be recognised. When each unit provides two measurements — before and after a treatment, or left and right — the two samples are NOT independent, and the correct approach is to reduce each pair to its difference

d

and perform a ONE-sample

t

-test on the differences, testing

H_0

: mean difference

= 0

. This removes the between-unit variation and is usually far more powerful than treating the data as two independent samples.

Choosing between a two-sample and a paired test is an examined judgement that depends on the design, not on the data. If the same subjects (or naturally matched pairs) are measured twice, the design is paired; if two separate groups are compared, it is two-sample. Applying an independent two-sample test to paired data throws away the pairing and typically loses sensitivity.

In every case the conclusion is stated in context, and a confidence interval for the difference of means often accompanies the test. An interval for the difference that excludes zero corresponds to a significant two-tailed test, giving both the decision and an estimate of the size of the effect — which is frequently more useful than the test alone.

z = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\dfrac{\sigma_1^2}{n_1} + \dfrac{\sigma_2^2}{n_2}}}; \qquad \text{paired: one-sample } t \text{ on the differences } d

Two-sample and paired tests

Independent differences add variances; paired data are reduced to differences first.

Worked example

A paired-sample test

Eight students sit a test before and after a revision course; the differences (after minus before) have mean 3.5 marks and sample standard deviation 4.0. Test at the 5% level whether the course improves marks (t-critical for 7 df, one-tailed, is 1.895).

Recognise the design
Each student is measured twice, so the data are paired; work with the differences $d$ .
Hypotheses
$H_0$ : mean difference $= 0$ against $H_1$ : mean difference $> 0$ .
Test statistic
$t = \dfrac{3.5 - 0}{4.0/\sqrt{8}} = \dfrac{3.5}{1.414} = 2.475$ .
$t = \frac{3.5}{4.0/\sqrt{8}} = 2.475$
Conclude
With 7 df the critical value is 1.895; since $2.475 > 1.895$ , reject $H_0$ : the course appears to improve marks.

Result: A paired $t$ -test gives $t = 2.475 > 1.895$ , so reject $H_0$ : there is evidence the course improves marks.

Typical mistakes

Treating paired data as two independent samples, discarding the pairing.
Subtracting the two variances instead of adding them for a difference of means.

Active revision

Ten athletes have their times measured before and after a training programme. Explain which test is appropriate and outline its steps.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Confidence intervals and their interpretation#

StandardPearson Edexcel 9ST0, Topic 13 (Paper 2)

A confidence interval on a number line

Fig. 395% confidence interval (492.1, 499.9) about the sample mean 496; the hypothesised 500 lies outside.

A confidence interval reports a plausible range for a parameter with a stated confidence level, and it is the natural partner of a hypothesis test. The interval below,

496 \pm 3.92

, is centred on the sample mean 496 and gives the 95% interval

(492.1, 499.9)

; the hypothesised value 500 lies outside it, which is exactly why the two-tailed test at the 5% level rejected

H_0

This equivalence is exact and worth stating: a value

\mu_0

is rejected by a two-tailed test at level

\alpha

if and only if it lies outside the

100(1 - \alpha)\%

confidence interval. The interval therefore does the job of infinitely many tests at once, showing every value that the data are and are not consistent with, and reporting the precision of the estimate through its width.

The confidence LEVEL is a property of the method, not of a single interval. Over many samples,

95\%

of the intervals constructed this way will contain the true parameter; a particular interval either contains it or does not. Saying 'there is a 95% probability the mean is in (492.1, 499.9)' is the classic misinterpretation, because the mean is a fixed (if unknown) number, not a random one.

Confidence intervals are often preferred to bare tests because they convey effect size and uncertainty together. A very narrow interval far from a hypothesised value signals a large, well-estimated effect; a wide interval, even one excluding the value, warns that the estimate is imprecise. Reporting the interval alongside any test is best practice and is rewarded in extended answers.

\mu_0 \text{ rejected (two-tailed, level } \alpha) \iff \mu_0 \notin 100(1 - \alpha)\% \text{ confidence interval}

Interval-test equivalence

The interval shows every hypothesised value the data are consistent with.

Worked example

Reading a test from an interval

A 95% confidence interval for a population mean is (492.1, 499.9). State the outcome of a two-tailed test at the 5% level of $H_0$ : $\mu = 500$ .

Locate the hypothesised value
$\mu_0 = 500$ .
Check membership
500 lies outside the interval (492.1, 499.9).
Apply the equivalence
A value outside the 95% interval is rejected by the two-tailed 5% test, so reject $H_0$ .
$500 \notin (492.1, 499.9) \Rightarrow \text{reject } H_0$

Result: Since 500 lies outside the 95% interval, the two-tailed 5% test rejects $H_0$ : $\mu = 500$ .

Typical mistakes

Claiming a 95% probability that the parameter lies in a specific interval.
Reporting the interval without linking it to the corresponding test decision.

Active revision

A 99% confidence interval for a mean is (48.2, 55.8). State whether a two-tailed test at the 1% level would reject $\mu_0 = 50$ , and why.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The power of a test#

AdvancedPearson Edexcel 9ST0, Topic 13 (Paper 2)

Power as an area under the alternative

Fig. 4Power is the shaded area of the alternative distribution beyond the critical value 1.645; here about 0.64.

The power of a test is the probability that it correctly rejects a false null hypothesis,

\text{power} = P(\text{reject } H_0 \mid H_1 \text{ true}) = 1 - \beta

, where

\beta

is the probability of a Type II error. A powerful test is one likely to detect a real effect. Power can only be computed against a SPECIFIC alternative value, because how easy an effect is to detect depends on how large it is.

The figure shows the idea: the left curve is the sampling distribution of the test statistic under

H_0

, the right curve under a specific alternative, and the shaded area is the power — the part of the alternative distribution that falls in the rejection region (beyond the critical value). The further apart the two curves, the more of the alternative lies in the rejection region and the greater the power.

Three levers increase power. A larger true effect (greater separation of the curves) is easier to detect; a larger sample size shrinks both standard errors and separates the standardised curves; and a larger significance level moves the critical value inward, enlarging the rejection region. The first is fixed by nature, but sample size is the designer's main tool for achieving adequate power.

There is an unavoidable tension with the Type I error rate: lowering

\alpha

to reduce false alarms moves the critical value outward and lowers the power, raising

\beta

. For a fixed sample size the two error rates trade off, so the only way to reduce both is to collect more data. Designing a study with enough power to detect an effect worth finding is a central practical concern, and understanding this trade-off is the key AO3 outcome of the topic.

\text{power} = 1 - \beta = P(\text{reject } H_0 \mid H_1 \text{ true})

Power of a test

Increases with effect size, sample size and significance level; computed against a specific alternative.

Worked example

Reasoning about power

A one-tailed 5% test uses the critical value 1.645 for the standardised mean. Under a specific alternative the standardised mean is distributed as $N(2, 1)$ . Find the power and comment.

Power as an area
Power $= P(\text{statistic} > 1.645 \mid H_1)$ where the statistic is $N(2, 1)$ .
Standardise
$P(Z > 1.645 - 2) = P(Z > -0.355)$ .
Evaluate
$= \Phi(0.355) = 0.639$ (3 s.f.).
$\text{power} = \Phi(0.355) = 0.639$
Comment
About 64% power: increasing the sample size would separate the curves further and raise this.

Result: The power is about 0.639, so the test would detect this alternative about 64% of the time; a larger sample would increase it.

Typical mistakes

Trying to state a single power without a specific alternative value.
Believing power can be raised without cost; lowering $\alpha$ reduces power for a fixed $n$ .

Active revision

Explain two changes to a study's design that would increase the power of its hypothesis test, and state the cost of each.

Active recall

Recall the key points — then reveal.

Sources: GCE AS and A level subject content (Statistics) (Department for Education / Ofqual)

References & sources

Sources

Pearson Edexcel

Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification

Department for Education / Ofqual

GCE AS and A level subject content (Statistics)

Hypothesis testing, confidence intervals and power

5 sections~14 min reading time3 competenciesLevel Standard 2 · Advanced 3

Hypothesis testing, confidence intervals and power

Key points

A two-tailed z-test and its interval

Key points

A one-tailed t-test

Key points

A paired-sample test

Key points

Reading a test from an interval

Key points

Reasoning about power

Sources

Hypothesis testing, confidence intervals and power

Key points

A two-tailed z-test and its interval

Key points

A one-tailed t-test

Key points

A paired-sample test

Key points

Reading a test from an interval

Key points

Reasoning about power

Sources