Notes · StatisticsUK · A-Levels

Correlation and linear regression

This topic analyses bivariate data through scatter diagrams, the product-moment and Spearman's rank correlation coefficients, and the least-squares regression line. It develops the meaning and calculation of each measure, the use of a regression model for prediction, the difference between interpolation and extrapolation, and the crucial distinction between correlation and causation.

5 sections~12 min reading time3 competenciesLevel Foundation 1 · Standard 2 · Advanced 2

T·101010 / 18

Exam profile

AO1 · Calculate the PMCC, Spearman's coefficient and the least-squares regression coefficientsAO2 · Interpret correlation and regression in context, including their limitationsAO3 · Use a regression model to make and criticise predictions

Operators:calculatefinddetermineinterpretcommentpredict

basic level

The AS foundation expects scatter diagrams, the interpretation of the PMCC, and the use of a given regression line.

higher level

The full A-Level expects computing the PMCC and Spearman's coefficient, deriving the least-squares line, and reasoning about prediction and causation.

Depth

Text

Contents · 5 sections

Correlation and linear regression

Scatter diagrams and correlation#

FoundationPearson Edexcel 9ST0, Topic 7 (Paper 1)

A scatter diagram with strong negative correlation

Fig. 1Points falling from upper left to lower right show strong negative correlation (here r = -0.995).

Bivariate data record two variables on each unit, and a scatter diagram plots one against the other to reveal any relationship. Correlation describes the strength and direction of a LINEAR association: positive correlation means

y

tends to rise with

x

, negative correlation means

y

tends to fall as

x

rises, and no correlation means no linear trend. The scatter below shows a strong negative correlation — the points fall almost along a downward line.

The strength of the association is judged by how closely the points cluster about a straight line. Tightly clustered points indicate strong correlation; a diffuse cloud indicates weak correlation. It is essential to look at the scatter before computing any coefficient, because a numerical correlation can be misleading when the true relationship is curved or when a single outlier distorts it.

Correlation captures only the LINEAR part of a relationship. Data lying on a perfect parabola can have a product-moment correlation near zero despite an exact functional relationship, because the linear trend cancels. A scatter diagram exposes such non-linear structure that a coefficient alone would hide, which is why the diagram and the number must always be read together.

The variables are labelled by role where one is thought to explain the other: the explanatory (independent) variable on the horizontal axis and the response (dependent) variable on the vertical axis. This ordering matters for regression in a later section, where the line is fitted to predict the response from the explanatory variable.

Worked example

Describing a scatter

A scatter of hours revised against test mark shows points rising from lower left to upper right, tightly clustered. Describe the correlation and name the explanatory variable.

Direction
Marks rise with hours revised, so the correlation is positive.
Strength
The tight clustering about a line indicates strong correlation.
Roles
Hours revised is the explanatory variable (horizontal axis); test mark is the response (vertical axis).

Result: The data show strong positive correlation, with hours revised as the explanatory variable.

Typical mistakes

Reading a correlation coefficient without inspecting the scatter for curvature or outliers.
Swapping the explanatory and response variables on the axes.

Active revision

Sketch what a scatter diagram would look like for two variables with weak positive correlation, and contrast it with strong positive correlation.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

The product-moment correlation coefficient#

StandardPearson Edexcel 9ST0, Topic 7 (Paper 1)

A scatter diagram with strong positive correlation

Fig. 2Hours revised (x) against test mark (y): eight points with r = 0.983.

The product-moment correlation coefficient (PMCC), denoted

r

, measures the strength and direction of a linear relationship on a scale from

-1

+1

. A value of

+1

means the points lie exactly on a line of positive gradient,

-1

an exact line of negative gradient, and

0

no linear association. It is computed from the summation quantities

r = \dfrac{S_{xy}}{\sqrt{S_{xx}\,S_{yy}}}

The building blocks are

S_{xy} = \sum xy - \dfrac{\sum x \sum y}{n}

S_{xx} = \sum x^2 - \dfrac{(\sum x)^2}{n}

and

S_{yy} = \sum y^2 - \dfrac{(\sum y)^2}{n}

. The numerator

S_{xy}

measures how

x

and

y

vary together (positive when they rise together), and the denominator scales it by the spread of each variable so that

r

is dimensionless and bounded.

For the scatter of hours and marks below, the summations give

S_{xx} = 42

S_{yy} = 43.5

and

S_{xy} = 42

, so

r = \dfrac{42}{\sqrt{42 \times 43.5}} = 0.983

(3 s.f.). This value, close to 1, confirms the strong positive linear relationship visible in the diagram.

Interpretation must be careful and contextual. A high

|r|

indicates a strong LINEAR relationship, not a causal one, and

r

is sensitive to outliers and undefined if either variable is constant. Because

r

is unaffected by which variable is called

x

and by linear rescaling of the data, it is a pure measure of linear association, distinct from the regression line, which does depend on the roles of the variables.

r = \frac{S_{xy}}{\sqrt{S_{xx}\,S_{yy}}}, \quad S_{xy} = \sum xy - \frac{\sum x \sum y}{n}

The product-moment correlation coefficient

Ranges from $-1$ to $+1$ ; $S_{xx}$ and $S_{yy}$ are the corrected sums of squares.

Worked example

Computing the PMCC

Eight students give $\sum x = 36$ , $\sum y = 50$ , $\sum xy = 267$ , $\sum x^2 = 204$ , $\sum y^2 = 356$ ( $n = 8$ ). Find the PMCC.

Corrected sums
$S_{xx} = 204 - \dfrac{36^2}{8} = 42$ ; $S_{yy} = 356 - \dfrac{50^2}{8} = 43.5$ ; $S_{xy} = 267 - \dfrac{36 \times 50}{8} = 42$ .
Apply the formula
$r = \dfrac{42}{\sqrt{42 \times 43.5}} = \dfrac{42}{\sqrt{1827}}$ .
$r = \frac{42}{\sqrt{1827}} = 0.983$
Interpret
$r = 0.983$ (3 s.f.), a strong positive linear correlation between hours revised and test mark.

Result: $r = 0.983$ (3 s.f.), indicating a strong positive linear relationship.

Typical mistakes

Forgetting to take the square root in the denominator, or using $S_{xx}S_{yy}$ un-rooted.
Interpreting a high $r$ as evidence of causation.

Active revision

For a data set with $S_{xx} = 42$ , $S_{yy} = 43.5$ and $S_{xy} = 42$ , find $r$ and interpret it.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Spearman's rank correlation coefficient#

StandardPearson Edexcel 9ST0, Topic 7 (Paper 1)

Ranking table for Spearman's coefficient

Fig. 3Two judges rank six competitors; the rank differences give sum of d^2 = 6, so r_s = 0.829.

Spearman's rank correlation coefficient

r_s

measures the strength of a MONOTONIC relationship — whether

y

consistently increases (or decreases) with

x

, even if not linearly. Each variable is replaced by its ranks, and the coefficient is computed from the differences in rank:

r_s = 1 - \dfrac{6\sum d^2}{n(n^2 - 1)}

, where

d

is the difference in the two ranks for each unit and

n

is the number of units.

Spearman's coefficient is used in place of the PMCC when the data are ranks or ordinal, when the relationship is monotonic but not linear, or when outliers would distort the PMCC — ranking is resistant to extreme values. Like the PMCC it ranges from

-1

+1

, with

+1

for perfect agreement of the rankings and

-1

for perfectly reversed rankings.

The table below shows two judges' rankings of six competitors. The rank differences give

\sum d^2 = 6

, so

r_s = 1 - \dfrac{6 \times 6}{6(6^2 - 1)} = 1 - \dfrac{36}{210} = 0.829

(3 s.f.). This high positive value indicates that the two judges largely agree on the order of merit, though not perfectly.

When the raw data are quantitative, computing both coefficients can be informative: a Spearman value much larger than the PMCC suggests a monotonic but non-linear relationship, while close values suggest the relationship is approximately linear. Choosing the appropriate coefficient, and justifying the choice, is an examined AO2 skill.

r_s = 1 - \frac{6\sum d^2}{n(n^2 - 1)}

Spearman's rank correlation coefficient

$d$ is the difference in ranks for each unit; ranges from $-1$ to $+1$ ; resistant to outliers.

Worked example

Computing Spearman's coefficient

Two judges rank six competitors, giving $\sum d^2 = 6$ . Find $r_s$ and interpret it.

Identify n
There are $n = 6$ competitors, so $n(n^2 - 1) = 6 \times 35 = 210$ .
Apply the formula
$r_s = 1 - \dfrac{6 \times 6}{210} = 1 - \dfrac{36}{210}$ .
$r_s = 1 - \frac{36}{210} = 0.829$
Evaluate and interpret
$r_s = 0.829$ (3 s.f.), a strong positive agreement between the two judges' orderings.

Result: $r_s = 0.829$ (3 s.f.), indicating strong agreement between the judges.

Typical mistakes

Using the raw data instead of the ranks in the formula.
Mishandling tied ranks (assign the mean of the tied positions).

Active revision

Six items are ranked by two methods with rank differences squared summing to 20. Find $r_s$ and interpret it.

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Least-squares regression#

AdvancedPearson Edexcel 9ST0, Topic 7 (Paper 1)

Scatter with the least-squares regression line

Fig. 4The least-squares regression line y = 1.75 + x fitted to the hours-and-marks data.

The least-squares regression line of

y

x

is the straight line

y = a + bx

that minimises the sum of the squared vertical distances (residuals) from the points to the line. Minimising this sum leads to the coefficients

b = \dfrac{S_{xy}}{S_{xx}}

for the gradient and

a = \bar{y} - b\bar{x}

for the intercept, so the line always passes through the mean point

(\bar{x}, \bar{y})

The gradient

b

is the estimated change in the response for a unit increase in the explanatory variable, and this is where the interpretation lives: a gradient of 1 mark per hour means each extra hour of revision is associated with about one more mark on average. The intercept

a

is the predicted response when

x = 0

, which is meaningful only if

x = 0

is within or near the range of the data.

For the hours-and-marks data,

b = \dfrac{S_{xy}}{S_{xx}} = \dfrac{42}{42} = 1

and

a = \bar{y} - b\bar{x} = 6.25 - 1 \times 4.5 = 1.75

, giving the line

y = 1.75 + x

, drawn through the scatter below. Each additional hour of revision is associated with about one extra mark, and a student revising for 5 hours is predicted to score

1.75 + 5 = 6.75

The regression of

y

x

is not the same as the regression of

x

y

: minimising vertical distances gives a different line from minimising horizontal ones, unless the correlation is perfect. The choice of which variable is the response therefore matters, and the line of

y

x

should be used only to predict

y

from

x

b = \frac{S_{xy}}{S_{xx}}, \qquad a = \bar{y} - b\bar{x}, \qquad y = a + bx

Least-squares regression of y on x

Minimises the sum of squared vertical residuals; the line passes through $(\bar{x}, \bar{y})$ .

Worked example

Fitting the least-squares line

For the hours-and-marks data with $S_{xx} = 42$ , $S_{xy} = 42$ , $\bar{x} = 4.5$ and $\bar{y} = 6.25$ , find the regression line and predict the mark for 5 hours.

Gradient
$b = \dfrac{S_{xy}}{S_{xx}} = \dfrac{42}{42} = 1$ .
Intercept
$a = \bar{y} - b\bar{x} = 6.25 - 1(4.5) = 1.75$ , so the line is $y = 1.75 + x$ .
Predict
At $x = 5$ , $\hat{y} = 1.75 + 5 = 6.75$ .
$\hat{y} = 1.75 + 5 = 6.75$
Interpret
Each extra hour is associated with about 1 more mark; 5 hours predicts a mark of about 6.75.

Result: The regression line is $y = 1.75 + x$ , predicting a mark of 6.75 for 5 hours of revision.

Typical mistakes

Swapping $S_{xy}/S_{xx}$ for $S_{xy}/S_{yy}$ when finding the gradient.
Interpreting the intercept when $x = 0$ lies far outside the data range.

Active revision

Given $S_{xx} = 42$ , $S_{xy} = 42$ , $\bar{x} = 4.5$ and $\bar{y} = 6.25$ , find the regression line of $y$ on $x$ .

Active recall

Recall the key points — then reveal.

Sources: Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification (Pearson Edexcel)

Prediction, residuals and correlation versus causation#

AdvancedPearson Edexcel 9ST0, Topic 7 (Paper 1)

A residual is the vertical gap between an observed point and the regression line,

e = y - \hat{y}

, where

\hat{y} = a + bx

is the predicted value. Least squares makes the residuals sum to zero and their squared total as small as possible. A pattern in the residuals — curvature, or a fanning-out spread — is a warning that a straight-line model is inadequate, so residuals are the standard diagnostic for the fit.

Prediction using the line is trustworthy only within the range of the data (interpolation). Extending the line beyond the observed range (extrapolation) assumes the linear pattern continues where there is no evidence for it, and can give absurd predictions — the hours-and-marks line would eventually predict a mark above the maximum available. Any prediction should be accompanied by a comment on whether it is interpolation or extrapolation.

The most important interpretive point is that correlation does not imply causation. A strong

r

between two variables can arise because one causes the other, because both are driven by a third (confounding) variable, or by pure coincidence. Ice-cream sales and drowning incidents correlate because both rise in hot weather, not because one causes the other; asserting causation from correlation alone is a serious and frequently penalised error.

Establishing causation requires more than a correlation: a plausible mechanism, the ruling out of confounders, and ideally a controlled experiment (the subject of the final topic). The honest conclusion from a regression analysis is therefore an association with a stated strength and range of validity, together with an explicit caution against reading it as cause and effect.

e = y - \hat{y}, \qquad \hat{y} = a + bx

Residual

The vertical distance from a point to the line; patterns in the residuals reveal a poor fit.

Worked example

A residual and a caution on extrapolation

For the line $y = 1.75 + x$ , find the residual at the point (4, 6) and comment on predicting the mark for 20 hours of revision.

Predicted value
$\hat{y} = 1.75 + 4 = 5.75$ .
Residual
$e = y - \hat{y} = 6 - 5.75 = 0.25$ .
$e = 6 - 5.75 = 0.25$
Extrapolation
20 hours lies far beyond the data range (1 to 8 hours), so the prediction $1.75 + 20 = 21.75$ is unreliable extrapolation and may exceed the maximum possible mark.

Result: The residual at (4, 6) is +0.25; predicting for 20 hours is unreliable extrapolation.

Typical mistakes

Extrapolating far beyond the data and trusting the prediction.
Concluding that one variable causes another purely from a strong correlation.

Active revision

Using $y = 1.75 + x$ , find the residual for the point (4, 6) and comment on whether predicting the mark for 20 hours is reliable.

Active recall

Recall the key points — then reveal.

Sources: GCE AS and A level subject content (Statistics) (Department for Education / Ofqual)

References & sources

Sources

Pearson Edexcel

Pearson Edexcel Level 3 Advanced GCE in Statistics (9ST0) Specification

Department for Education / Ofqual

GCE AS and A level subject content (Statistics)

Correlation and linear regression

5 sections~12 min reading time3 competenciesLevel Foundation 1 · Standard 2 · Advanced 2

Correlation and linear regression

Key points

Describing a scatter

Key points

Computing the PMCC

Key points

Computing Spearman's coefficient

Key points

Fitting the least-squares line

Key points

A residual and a caution on extrapolation

Sources

Correlation and linear regression

Key points

Describing a scatter

Key points

Computing the PMCC

Key points

Computing Spearman's coefficient

Key points

Fitting the least-squares line

Key points

A residual and a caution on extrapolation

Sources