1 exam-style question plus unlimited generated ones on this topic, at up to four difficulty levels, with full mark schemes. Notes and worksheets are open to everyone. Practice needs a free account.
Or take a quick quiz: 12 questions on this subtopic in AS Maths (Edexcel), marked as you tap.
Identifying outliers with a rule given in the question, deciding whether to remove them, cleaning data (errors, missing values, and ‘n/a’ and ‘tr’ in the large data set), and choosing or criticising a way to present data. Usually short parts linked to box plots or measures of spread.
Key facts
Quartile rule: outlier if below \(Q_1 - k(Q_3 - Q_1)\) or above \(Q_3 + k(Q_3 - Q_1)\)
Mean rule: outlier if outside \(\bar{x} \pm k\sigma\)
Hardly affected by outliers: median, IQR. Strongly affected: mean, standard deviation, range, PMCC
n/a = not available (leave out); tr = trace of rain, less than 0.05 mm
Notes
Identifying outliers
diagram
An outlier is a value that is very different from the rest of the data.
The rule to use is always given in the question. Common rules: an outlier is less than \(Q_1 - k(Q_3 - Q_1)\) or greater than \(Q_3 + k(Q_3 - Q_1)\), often with \(k = 1.5\); or it is more than \(k\) standard deviations from the mean, e.g. outside \(\bar{x} \pm 2\sigma\)
Only the smallest and largest values need checking against the limits.
On a box plot, mark each outlier with a cross; the whisker ends at the most extreme value that is not an outlier.
Only 14.2 is beyond the limit, so it is a cross; the whisker ends at 5.0
Keep or remove?
Remove a value only if it is an error: impossible in context (e.g. negative rainfall), a recording or typing mistake, or not from the population being studied.
A genuine extreme value (e.g. a very wet day during a storm) is real data and should be kept; removing it would misrepresent the population.
Outliers strongly affect the mean, standard deviation, range and PMCC; the median and IQR hardly change.
Cleaning data
Cleaning means dealing with errors, missing values and outliers before analysing the data.
Leave out missing values (n/a in the large data set) and reduce \(n\). A rainfall value of ‘tr’ (a trace, less than 0.05 mm) is often treated as 0 in calculations; say what you have done.
To remove a value \(x_0\) using summary statistics: new \(n = n - 1\), new \(\Sigma x = \Sigma x - x_0\), new \(\Sigma x^2 = \Sigma x^2 - x_0^2\)
Choosing and criticising diagrams
diagram
Histogram: the shape of grouped continuous data. Box plots: comparing distributions and showing outliers. Cumulative frequency: medians, quartiles, percentiles and how many values lie above or below a value. Scatter diagram: the relationship between two variables.
Criticise a diagram by spotting what misleads, e.g. unequal class widths plotted as frequency, or a vertical axis that does not start at zero and exaggerates differences.
Same data (52 and 55): a cut-off axis exaggerates the difference
How to answer each type of question
M = method mark, A = answer mark, B = a mark on its own.
Show that a value is an outlier (quartile rule)
3 to 4 marksGrade D
Find \(Q_1\) and \(Q_3\) using the rule for an ordered list.
Calculate the limit using the rule given.
Compare the value with the limit and state the conclusion.
If asked whether to remove it, use the context: is the value possible?
Example. The daily total rainfall, in mm, for 15 days in a student's sample, in order, is 0, 0, 0.2, 0.4, 0.6, 1.0, 1.2, 1.5, 2.0, 2.4, 3.0, 3.6, 4.1, 5.0, 14.2 An outlier is a value greater than \(Q_3 + 1.5(Q_3 - Q_1)\). (a) Show that 14.2 is an outlier. (b) State, with a reason, whether 14.2 should be removed from the data.
Show the model answerHide the model answer
(a) \(\tfrac{15}{4} = 3.75\), so \(Q_1\) is the 4th value, 0.4; \(\tfrac{45}{4} = 11.25\), so \(Q_3\) is the 12th value, 3.6
(B1)
Limit \(= 3.6 + 1.5 \times (3.6 - 0.4) = 8.4\) (M1);\(14.2 \gt 8.4\), so 14.2 is an outlier
(A1)
(b) No: 14.2 mm is a realistic amount of rain for a very wet day, so it is probably a genuine value
(B1)
Choose a diagram and deal with missing data
2 to 3 marksGrade B
Name a diagram suited to the purpose (comparing, showing shape, showing a relationship).
Give a reason linked to what the diagram shows.
For missing or invalid values, say what you would do and the effect on \(n\)
Example. A student wants to compare the daily mean temperatures at Leeming in 1987 and in 2015, and to show any outliers. (a) Suggest a suitable diagram, giving a reason. (b) Some days have ‘n/a’ recorded for the daily mean temperature. Explain how the student should deal with these.
Show the model answerHide the model answer
(a) Two box plots on the same scale (B1): they show the medians, spreads and any outliers side by side, so the two years are easy to compare
(B1)
(b) Leave these days out, as the data are missing, and reduce the sample size accordingly
(B1)
Mean and standard deviation rule, then clean and recalculate
6 to 7 marksGrade A
Find \(\bar{x}\), \(\sigma\) and the limits \(\bar{x} \pm k\sigma\)
Compare the value with the limit and conclude.
Remove the value from \(n\), \(\Sigma x\) and \(\Sigma x^2\)
Recalculate the mean and standard deviation.
Example. The times, \(x\) minutes, taken by 12 students to solve a puzzle give \(\Sigma x = 504\) and \(\Sigma x^2 = 22\,032\). One of the times is 62 minutes. An outlier is a value more than 2 standard deviations from the mean. (a) Show that 62 is an outlier. (b) The time of 62 minutes was recorded wrongly and is removed. Find the mean and the standard deviation of the remaining times.
Using a different outlier rule from the one given in the question.
Removing an outlier just because it is extreme, when it is a genuine value.
Forgetting to reduce \(n\) after removing a value or leaving out missing data.
Drawing the whisker out to an outlier instead of marking the outlier with a cross.
Saying that removing an outlier changes the median and IQR a lot.
More tips
Memory tricks
Error or extreme? Ask whether the value is possible here. Impossible means an error, so remove it; possible means keep it.
Removing a value: subtract it from \(\Sigma x\), subtract its square from \(\Sigma x^2\), and reduce \(n\) by 1.
Only test the most extreme values: if the largest value is not an outlier, no other value above the median is.
Exam technique
In ‘show that’ outlier questions, calculate the limit, compare it with the value and write a conclusion.
When deciding whether to remove a value, refer to the context: could this value really happen?
Know how the large data set records missing values (n/a) and very small rainfall (tr), and say how you would handle them.
What each grade needs
What you need to be able to do, from the first marks up to the top grade.
Grade E
State what an outlier isAn outlier is a value that is very different from (much larger or smaller than) the rest of the data.
Grade D
Find outlier limits using quartiles and the IQRUse the rule given, e.g. below \(Q_1 - 1.5 \times \text{IQR}\) or above \(Q_3 + 1.5 \times \text{IQR}\)
Grade C
Find outlier limits using mean and standard deviationUse the rule given, e.g. outside \(\bar{x} \pm 2\sigma\)
Grade C
Draw a box plot showing outliersMark outliers with crosses and end the whisker at the most extreme value that is not an outlier.
Grade B
Choose a suitable diagram for the dataMatch the diagram to the data and the purpose, e.g. box plots to compare distributions.
Grade B
Decide whether an outlier should be removedRemove it only if it is an error; keep a genuine extreme value.
Grade A
Clean data and recalculate summary statisticsTake errors and missing values out of \(n\), \(\Sigma x\) and \(\Sigma x^2\), then recalculate.
Grade A*
Evaluate how cleaning affects the conclusionsExplain which statistics change a lot (mean, standard deviation, range, \(r\)) and which barely change (median, IQR).
Sample questions
Written for this site in the style of Edexcel exam questions. They are not taken from real past papers.
Question 1Easy5 marks
The times, in minutes, taken by a random sample of customers to be served at a bank are listed in order. 2, 5, 5, 6, 6, 8, 8, 8, 9, 9, 10, 10, 16 An outlier is a value that is greater than \(Q_3 + 1.5 \times (Q_3 - Q_1)\) or less than \(Q_1 - 1.5 \times (Q_3 - Q_1)\).
(a) Find the lower quartile and the upper quartile.[2]
(b) Show that there is exactly one outlier, and state its value.[3]
A scientist measures the masses, \(x\) grams, of 20 seedlings. The results are summarised by \[\sum x = 356, \qquad \sum x^2 = 6532\]A student calculates the standard deviation as follows: \(\sigma = \sqrt{\frac{\sum x^2}{n}} - \bar{x} = \sqrt{\frac{6532}{20}} - 17.8 = 18.07 - 17.8 = 0.27\) g
(a) Explain the error in the student’s method and find the correct standard deviation, giving your answer to 3 significant figures.[3]
(b) An outlier is defined as a value more than 2 standard deviations from the mean. Show that a seedling of mass 25 g is an outlier.[2]
(c) The scientist removes this seedling from the data. Find the mean and the standard deviation of the remaining 19 masses, giving your answers to 3 significant figures.[3]
(d) Give a reason why the scientist should check this seedling before deciding whether to remove it from the data.[1]
Show the answer and mark scheme
(a)Answer: 3.12 g
B1 for the correct formula is \(\sigma = \sqrt{\frac{\sum x^2}{n} - \bar{x}^2}\): the square of the mean must be subtracted inside the square root
M1 for \(\sqrt{\frac{6532}{20} - 17.8^2} = \sqrt{326.6 - 316.84}\)
A1 for 3.12 (g)
Worked solution: The variance is \(\frac{\sum x^2}{n} - \bar{x}^2\); the student took the square root before subtracting and subtracted \(\bar{x}\) instead of \(\bar{x}^2\). \(\sigma = \sqrt{326.6 - 17.8^2} = \sqrt{9.76} = 3.12\) g.
A1 for 24.05 (or 24.0 or 24.1) and a comparison with 25, with a conclusion
Worked solution: \(\bar{x} + 2\sigma = 17.8 + 2 \times 3.124 = 24.05\). Since \(25 \gt 24.05\), the mass 25 g is an outlier.
(c)Answer: Mean 17.4 g, standard deviation 2.72 g
M1 for \(\sum x = 331\) and \(\sum x^2 = 6532 - 625 = 5907\)
A1 for mean 17.4 (g)
A1 for standard deviation 2.72 (g)
Worked solution: \(\sum x = 356 - 25 = 331\), \(\sum x^2 = 6532 - 25^2 = 5907\). Mean \(= \frac{331}{19} = 17.4\) g. \(\sigma = \sqrt{\frac{5907}{19} - \left(\frac{331}{19}\right)^2} = \sqrt{7.40} = 2.72\) g.
(d)Answer: An outlier may be a genuine value (a naturally large seedling), not an error; it should only be removed if it is a mistake.
B1 for the value may be genuine (not an error in measuring or recording), in which case it should be kept
Worked solution: A seedling of 25 g may be a genuine, unusually large seedling. Outliers should only be removed if they are errors (for example in weighing or recording); removing genuine data would bias the results.
Question 3Hard8 marks
The daily maximum gust, \(g\) knots, is recorded for each of a random sample of 33 days that a student took from the large data set for Hurn in 2015. (The values are illustrative.) The largest value is 37. The values are summarised by \(\Sigma g = 690, \ \Sigma g^2 = 15{,}838\). An outlier is a value more than 2 standard deviations above or below the mean.
(a) Show that 37 is an outlier.[3]
(b) The value 37 is found to be a recording error and is removed. Find the mean and the standard deviation of the remaining values, giving your answers to 2 decimal places.[4]
(c) Comment on the changes to the mean and the standard deviation.[1]
Show the answer and mark scheme
(a)Answer: Mean 20.91, s.d. 6.54, so the upper limit is 33.99, and 37 is greater than this.
M1 for a correct method for the mean and standard deviation
A1 for mean awrt 20.9 and s.d. awrt 6.54 (or 6.64)
A1 for limit awrt 34.0 (or 34.2) and a correct conclusion
(b)Answer: Mean 20.41, standard deviation 5.98 (knots)
M1 for \(\Sigma g = 690 - 37 = 653\) and \(n = 32\)
M1 for \(\Sigma g^2 = 15{,}838 - 37^2 = 14{,}469\)
A1 for mean awrt 20.41
A1 for standard deviation awrt 5.98 (allow 6.07)
(c)Answer: Both decrease; the standard deviation decreases proportionally more, because it is based on squared deviations, so it is strongly affected by an extreme value.
B1 for both decreasing, with the standard deviation affected more (it is more sensitive to extreme values)