School performance tables, and high stakes accountability more generally, incentivise secondary schools to maximise the Attainment 8 scores of their pupils.
If this is achieved by improving teaching and learning then the incentives are working as intended. Having said that, the approach to maintaining standards over time known as comparable outcomes means there are limits on the extent to which results can improve from one year to the next.
However, as we have written previously, there are also incentives to enter pupils for qualifications that are perceived to be higher scoring than others. We’ve written about the equivalence of technical awards and GCSEs, the most notable example being the European Computer Driving Licence (ECDL) which is no longer counted in performance tables.
Even amongst GCSEs there isn’t comparability between subjects as we wrote here some time ago. French, German and Spanish tended to be graded more severely than other subjects, for example.
We’ve not looked at the comparability of different qualifications and subjects for a while, so let’s do so using end of Key Stage 4 data for 2018.
I’m going to compare pupils’ results in each subject to their English language and maths average point score (APS), taking the two scores (using zero if a pupil did not enter) and averaging them. English and maths are used because almost everyone enters both subjects.
Next, taking each pupil’s results that count towards the performance tables we can look at the difference between the points awarded in each subject and their English and maths APS – before working out the average difference for each subject.
The chart below shows technical awards with at least 5,000 state school entrants in 2018. Numbers of entrants are shown in brackets.
On the whole, pupils tend to score more points in BTEC qualifications than they do in GCSE English and maths. For most BTECs, the difference is one point on average, the equivalent of a grade at GCSE. In health and social care, it is a grade and a half.
However, some technical awards are scored more severely than English and maths. These include the Edexcel certificate in digital applications (CiDA) and the LIBF Level 2 Certificate in Financial Education (CeFE).
There is also variation among GCSEs. The chart below shows subjects with at least 20,000 entrants in 2018. Again, I’m comparing the scores in each subject to those in English and maths.
The rank order is very similar to that shown in the first table of the 2016 blogpost on GCSE inter-subject comparability.
Art and design (photography) leads the way. On average, pupils achieve 0.9 more points in this subject than in English and maths.
At the lower end, we again find French, German, Spanish and computing. Pupils even tend to achieve lower outcomes in geography and history than in English and maths, by over a third of a grade.
There is a dilemma here. On the one hand, many would argue that there should be qualifications that pupils who might struggle with academic qualifications can succeed in. On the other hand, others would argue that this creates a perverse incentive to enter other pupils for higher scoring qualifications that may not be in their best interests.
One way to square this circle might be to change the points awarded and bring them more in line with English and maths. We have previously proposed a method of recalculating Attainment 8 (and therefore Progress 8). We plan to update that work shortly.
Want to stay up-to-date with the latest research from FFT Education Datalab? Sign up to Datalab’s mailing list to get notifications about new blogposts, or to receive the team’s half-termly newsletter.
1. For all but a handful of pupils who took AS-Levels early, these scores will be based on GCSEs