Notes · StatisticsUK · A-Levels

Sampling, estimates and resampling

This topic explains how a statistic computed from a sample behaves as an estimate of a population parameter. It develops the sampling distribution of the mean and its standard error, the central limit theorem, unbiased point estimators including the (n - 1)-divisor sample variance, the idea of a confidence interval, and resampling by the bootstrap.

5 sections~12 min reading time3 competenciesLevel Standard 1 · Advanced 4

T·121212 / 18

Exam profile

AO1 · Compute point estimates, standard errors and unbiased variance estimatesAO2 · Explain unbiasedness and the central limit theoremAO3 · Use sampling distributions and resampling to quantify uncertainty

Operators:estimatecalculatefinddetermineexplaininterpret

basic level

The AS foundation introduces the sample mean as an estimate; sampling distributions and resampling are principally full A-Level (Paper 2) content.

higher level

The full A-Level expects the standard error, the central limit theorem, unbiased estimators and the bootstrap idea.

Depth

Text

Contents · 5 sections

Sampling, estimates and resampling

The sampling distribution of the mean#

StandardPearson Edexcel 9ST0, Topic 12 (Paper 2)

The sampling distribution narrows as n increases

Fig. 1Sampling distribution of the mean for sigma = 10: standard error 2 at n = 25 and 1 at n = 100.

A statistic such as the sample mean varies from sample to sample, and the distribution of its values over all possible samples is its sampling distribution. For the mean of a random sample of size

n

from a population with mean

\mu

and variance

\sigma^2

, the sampling distribution has mean

E(\bar{X}) = \mu

and variance

\text{Var}(\bar{X}) = \dfrac{\sigma^2}{n}

, results that follow directly from the rules for combining independent variables.

The standard deviation of the sampling distribution,

\dfrac{\sigma}{\sqrt{n}}

, is called the standard error of the mean. It measures how much the sample mean typically differs from the population mean, and — crucially — it shrinks as

n

grows, in proportion to

\dfrac{1}{\sqrt{n}}

. Quadrupling the sample size halves the standard error, which is why larger samples give more precise estimates.

The figure shows this narrowing: for

\sigma = 10

, the sampling distribution for

n = 25

has standard error 2, while for

n = 100

it has standard error 1 — taller and tighter about the mean. The estimate becomes more concentrated on the true value as the sample grows, though with diminishing returns because of the square root.

That

\bar{X}

has mean exactly

\mu

makes it an unbiased estimator (the next-but-one section), and that its spread is

\dfrac{\sigma}{\sqrt{n}}

makes precision improvable by sampling more. Together these two facts are the foundation on which confidence intervals and hypothesis tests for a mean are built.

E(\bar{X}) = \mu, \qquad \text{Var}(\bar{X}) = \frac{\sigma^2}{n}, \qquad \text{standard error} = \frac{\sigma}{\sqrt{n}}

Sampling distribution of the mean

The mean is unbiased; the standard error shrinks as $1/\sqrt{n}$ .

Worked example

A probability for the sample mean

A population has mean 50 and standard deviation 10. For a random sample of 25, find $P(\bar{X} > 52)$ , assuming $\bar{X}$ is approximately normal.

Standard error
$\dfrac{\sigma}{\sqrt{n}} = \dfrac{10}{\sqrt{25}} = 2$ .
Standardise
$P(\bar{X} > 52) = P\!\left(Z > \dfrac{52 - 50}{2}\right) = P(Z > 1)$ .
$P(\bar{X} > 52) = P(Z > 1) = 0.1587$
Evaluate
$= 1 - \Phi(1) = 0.1587$ .

Result: The standard error is 2 and $P(\bar{X} > 52) = 0.1587$ .

Typical mistakes

Using $\sigma$ instead of $\frac{\sigma}{\sqrt{n}}$ when standardising the sample mean.
Thinking the standard error falls as $\frac{1}{n}$ rather than $\frac{1}{\sqrt{n}}$ .

Active revision

A population has $\mu = 50$ , $\sigma = 10$ . For a sample of 25, find the standard error and $P(\bar{X} > 52)$ .

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The central limit theorem#

AdvancedPearson Edexcel 9ST0, Topic 12 (Paper 2)

The central limit theorem (CLT) states that for a random sample of size

n

from ANY population with finite mean

\mu

and variance

\sigma^2

, the sampling distribution of the mean

\bar{X}

is approximately normal,

\bar{X} \approx N\!\left(\mu, \dfrac{\sigma^2}{n}\right)

, and the approximation improves as

n

increases. Remarkably, the population itself need not be normal — the normality emerges from averaging.

This is what makes the normal-based methods of inference so widely applicable. Even for a skewed or discrete population, the mean of a reasonably large sample is close to normal, so probabilities for

\bar{X}

, confidence intervals and

z

-tests can be computed as if the mean were exactly normal. A common working rule is that

n \geq 30

is large enough for most populations, with smaller

n

sufficing when the population is already near-normal.

When the population IS normal, the sampling distribution of the mean is exactly normal for every

n

, not merely approximately. The CLT is needed precisely for the non-normal case, and the further the population is from normal (heavily skewed, say), the larger the

n

required for a good approximation.

The CLT justifies treating the standardised sample mean

\dfrac{\bar{X} - \mu}{\sigma/\sqrt{n}}

as a standard normal variable in large samples. This single result underpins the

z

-tests and confidence intervals for a mean in the next topic, and it explains why the normal distribution is so central to statistics despite the variety of real populations.

\bar{X} \approx N\!\left(\mu, \frac{\sigma^2}{n}\right) \quad\text{for large } n, \text{ any population}; \qquad \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \approx N(0, 1)

The central limit theorem

The mean of a large sample is approximately normal whatever the population's shape.

Worked example

Using the CLT for a skewed population

Times between events have mean 8 and standard deviation 6, from a right-skewed population. For a random sample of 50, estimate $P(\bar{X} > 9)$ .

Justify normality
$n = 50 \geq 30$ , so by the CLT $\bar{X} \approx N\!\left(8, \dfrac{6^2}{50}\right)$ despite the skew.
Standard error
$\dfrac{6}{\sqrt{50}} = 0.8485$ .
Standardise
$P(\bar{X} > 9) = P\!\left(Z > \dfrac{9 - 8}{0.8485}\right) = P(Z > 1.179)$ .
$P(\bar{X} > 9) \approx P(Z > 1.179) = 0.119$
Evaluate
$= 1 - \Phi(1.179) = 0.119$ (3 s.f.).

Result: By the CLT, $P(\bar{X} > 9) \approx 0.119$ , valid because $n = 50$ is large enough to overcome the skew.

Typical mistakes

Applying a normal model to a single observation rather than to the sample mean.
Believing the CLT requires the population itself to be normal.

Active revision

A population is strongly right-skewed with mean 8 and variance 36. Explain whether $P(\bar{X} > 9)$ for a sample of 50 can be found using a normal model.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Unbiased point estimators#

AdvancedPearson Edexcel 9ST0, Topic 12 (Paper 2)

A point estimator is a statistic used to estimate a parameter, and it is unbiased when its expected value equals the parameter it estimates, so that it is correct on average over all samples. The sample mean is an unbiased estimator of the population mean, since

E(\bar{X}) = \mu

, which is one of the reasons it is the default estimator of the centre.

The sample variance requires care. The natural formula

\dfrac{S_{xx}}{n}

systematically underestimates the population variance, because the deviations are measured from the sample mean rather than the true mean. Dividing instead by

n - 1

corrects this bias, giving the unbiased estimator

s^2 = \dfrac{S_{xx}}{n - 1}

, where

S_{xx} = \sum x^2 - \dfrac{(\sum x)^2}{n}

The

n - 1

divisor is understood through degrees of freedom: once the sample mean is fixed, only

n - 1

of the deviations are free to vary, so the corrected sum of squares carries

n - 1

independent pieces of information. This is why

s^2

uses

n - 1

, and the same idea reappears in the degrees of freedom of the

t

, chi-squared and

F

tests.

In practice, whenever a sample is used to estimate a population variance — as it is for a confidence interval or a

t

-test — the

n - 1

version is the one to use. Reserving the

n

divisor for describing a complete data set, and the

n - 1

divisor for estimating a population, is a distinction that is explicitly examined and easy to get wrong.

\hat{\mu} = \bar{x} \;(\text{unbiased}); \qquad s^2 = \frac{S_{xx}}{n - 1} \;(\text{unbiased estimate of } \sigma^2)

Unbiased estimators

The sample mean estimates $\mu$ ; the $(n-1)$ -divisor sample variance estimates $\sigma^2$ .

Worked example

Unbiased estimates from a sample

A sample of $n = 10$ has $\sum x = 250$ and $\sum x^2 = 6350$ . Find the unbiased estimates of $\mu$ and $\sigma^2$ .

Estimate the mean
$\hat{\mu} = \bar{x} = \dfrac{250}{10} = 25$ .
Corrected sum of squares
$S_{xx} = 6350 - \dfrac{250^2}{10} = 6350 - 6250 = 100$ .
Estimate the variance
$s^2 = \dfrac{S_{xx}}{n - 1} = \dfrac{100}{9} = 11.1$ (3 s.f.).
$s^2 = \frac{100}{9} = 11.1$

Result: $\hat{\mu} = 25$ and $s^2 = \dfrac{100}{9} = 11.1$ (3 s.f.).

Typical mistakes

Dividing $S_{xx}$ by $n$ when estimating a population variance from a sample.
Confusing the estimate $s^2$ with the descriptive variance $\frac{S_{xx}}{n}$ .

Active revision

A sample of 10 gives $\sum x = 250$ and $\sum x^2 = 6350$ . Find the unbiased estimates of the population mean and variance.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The idea of a confidence interval#

AdvancedPearson Edexcel 9ST0, Topic 12 (Paper 2)

A point estimate gives a single value but no sense of its uncertainty; a confidence interval gives a range of plausible values for the parameter together with a stated level of confidence. For a mean with known

\sigma

, the interval is

\bar{x} \pm z\dfrac{\sigma}{\sqrt{n}}

, centred on the sample mean and extending by a multiple of the standard error, where

z

is the standard normal value for the required confidence (1.96 for 95%).

The width of the interval is

2z\dfrac{\sigma}{\sqrt{n}}

, so it is narrower — a more precise estimate — for larger samples and lower confidence levels. There is a genuine trade-off: demanding 99% confidence rather than 95% widens the interval, and only a larger sample can restore the precision without lowering the confidence.

The confidence level must be interpreted carefully. A 95% confidence interval means that the METHOD produces an interval containing the true parameter in 95% of samples in the long run; it does NOT mean there is a 95% probability that this particular interval contains the parameter, which is either in it or not. This subtle but examined point distinguishes the frequentist confidence statement from a probability about the parameter.

The interval is fully developed, with the

t

-distribution for unknown

\sigma

, in the next topic; introduced here it makes the purpose of the standard error concrete. A confidence interval turns the sampling distribution into a statement about the parameter, and its symmetry about

\bar{x}

shows the estimate flanked by its margin of error.

\bar{x} \pm z\,\frac{\sigma}{\sqrt{n}} \qquad (z = 1.96 \text{ for } 95\%)

Confidence interval for a mean (known sigma)

Centred on $\bar{x}$ , with margin of error a multiple of the standard error.

Worked example

A 95% confidence interval

A random sample of 25 items from a population with known $\sigma = 10$ has mean 50. Find a 95% confidence interval for the population mean.

Standard error
$\dfrac{\sigma}{\sqrt{n}} = \dfrac{10}{\sqrt{25}} = 2$ .
Margin of error
$1.96 \times 2 = 3.92$ .
Interval
$50 \pm 3.92 = (46.1, 53.9)$ (3 s.f.).
$50 \pm 1.96 \times 2 = (46.1, 53.9)$
Interpret
The method gives an interval containing the true mean in 95% of samples; here that interval is (46.1, 53.9).

Result: The 95% confidence interval is (46.1, 53.9) to 3 significant figures.

Typical mistakes

Saying there is a 95% probability the parameter lies in a specific computed interval.
Using the standard deviation $\sigma$ instead of the standard error $\frac{\sigma}{\sqrt{n}}$ in the margin.

Active revision

A sample of 25 from a population with $\sigma = 10$ has mean 50. Construct a 95% confidence interval for the population mean.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Resampling and the bootstrap#

AdvancedPearson Edexcel 9ST0, Topic 12 (Paper 2)

The bootstrap procedure

Fig. 2The bootstrap: resample the original data with replacement, recompute the statistic, and read off its variability.

Resampling estimates the variability of a statistic directly from the data, without assuming a distribution. The bootstrap, the standard method, treats the observed sample as a stand-in for the population: it draws many new samples of the same size FROM THE ORIGINAL SAMPLE, with replacement, recomputes the statistic on each, and uses the spread of those recomputed values to estimate the statistic's standard error and to form intervals.

The flow below sets out the procedure: from the original sample, draw a large number of resamples with replacement, compute the statistic for each, and read the standard error and interval from the resulting bootstrap distribution. Because sampling is with replacement, each resample differs from the original, and the variation among the recomputed statistics mimics the sampling variability that repeated real samples would show.

The bootstrap is powerful because it applies to statistics for which no simple standard-error formula exists — a median, a trimmed mean, a correlation — and it makes no assumption of normality. It relies instead on the original sample being representative of the population and reasonably large, since a small or biased sample gives a correspondingly unreliable bootstrap.

Resampling complements the formula-based methods rather than replacing them: where a neat standard error exists (the mean), the two broadly agree, and where none exists the bootstrap extends inference to new statistics. Its computational, assumption-light spirit is characteristic of modern applied statistics and of the enquiry-cycle emphasis of this qualification.

\text{bootstrap SE} = \text{standard deviation of the statistic over many resamples (with replacement)}

The bootstrap standard error

Estimated from the data directly, with no distributional assumption.

Worked example

Applying the bootstrap idea

A sample of 40 waiting times is available and the standard error of the sample median is required, for which no simple formula is used. Outline a bootstrap estimate.

Resample
Draw, say, 1000 resamples of size 40 from the original 40 values, each sampled with replacement.
Recompute
Find the median of each resample, giving 1000 bootstrap medians.
Summarise
The standard deviation of those 1000 medians estimates the standard error of the sample median.
Interval
The middle 95% of the bootstrap medians gives an approximate 95% confidence interval for the population median.

Result: The bootstrap estimates the median's standard error as the spread of the medians of many with-replacement resamples of the original data.

Typical mistakes

Resampling WITHOUT replacement, which just reproduces the original sample.
Trusting a bootstrap from a small or unrepresentative original sample.

Active revision

Explain how you would use the bootstrap to estimate the standard error of the median of a sample of 40 house prices.

Active recall

Recall the key points — then reveal.

Sources: GCE AS and A level subject content (Statistics) (Department for Education / Ofqual)

References & sources

Sources

Pearson Edexcel

Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification

Department for Education / Ofqual

GCE AS and A level subject content (Statistics)

Sampling, estimates and resampling

5 sections~12 min reading time3 competenciesLevel Standard 1 · Advanced 4

Sampling, estimates and resampling

Key points

A probability for the sample mean

Key points

Using the CLT for a skewed population

Key points

Unbiased estimates from a sample

Key points

A 95% confidence interval

Key points

Applying the bootstrap idea

Sources

Sampling, estimates and resampling

Key points

A probability for the sample mean

Key points

Using the CLT for a skewed population

Key points

Unbiased estimates from a sample

Key points

A 95% confidence interval

Key points

Applying the bootstrap idea

Sources