Update 20/03/20: this post was written before the Department for Education published further details on how grades will be awarded in 2020.
*Thanks to John Jerrim and Steve Wren for comments and suggestions. You can read more of John’s thoughts on this topic here*
With no GCSEs and A-Levels this year, everyone’s wondering just how pupils will be awarded grades in the summer.
There is no alternative to teachers awarding them. But can there be any quality control in the system?
Ofqual already has the answer – the much unloved approach of comparable outcomes. This assumes that if the prior attainment (at Key Stage 2 for GCSE; GCSE for A-Level) of pupils entering a subject doesn’t change then the grades awarded won’t change either.
Ofqual could produce an indicative range of grades for each school to award in each subject. This would be based on three things. Taking GCSE as an example these would be:
- Last year’s Key Stage 2 to GCSE transition matrices
- The list of pupils entered in a subject, together with their Key Stage 2 results
- Historic Key Stage 2 to GCSE value added data
An example of a transition matrix, for GCSE geography, is shown below.
Using the transition matrices, the probability of each pupil in the list of entrants achieving each of the possible grades can be inferred. For instance, someone at level 4a has a 19% chance of grade 5, a 14% chance of grade 6 and so on.
Once the probability of each grade for each pupil has been calculated, the average probability of each grade for a subject at a school can be calculated. In the example below, the school has 10 pupils entered in geography in 2020.
So based on the average probabilities, the following indicative grade ranges could be awarded to the 10 pupils:
- two grades in the range 7 to 9 (corresponding to an average probability of 20% for these 10 pupils: 9% + 7% + 4%)
- three grades in the range 4 to 6 (corresponding to an average probability of 34% for these 10 pupils)
- five grades in the range U to 3 (corresponding to an average probability of 45% for these 10 pupils)
With larger cohorts, indicative grades could be provided rather than ranges such as 7 to 9.
Ofqual could go further. It could tailor the set of indicative grades according to historical value added data for each subject at each school. So if an additional 10% of pupils at this example school historically achieved a grade 7 or above (grade A or above in legacy GCSEs) in geography then an extra grade in the range 7 to 9 could be awarded.
It would be up to the school to decide which pupils to award these grades to.
Alternatively, the grades could be awarded “blind” as far as the school is concerned. Schools could upload a list in which pupils A to J have been put into ranked order of attainment based on the evidence available to teachers. Grades would then be assigned by Ofqual using the method above. This would help avoid schools making difficult decisions about, for example, which pupil should get the last grade 5 available.
There is also the National Reference Test, which was taken in February. This might suggest an increase in attainment in the cohort nationally. If there is, the transition matrices could be tweaked accordingly.
None of this is perfect. No system of awarding is perfect – even exams can result in some pupils being awarded the “wrong” grade.
There will be winners and losers. This method may not work well for schools with small cohorts. Some groups of pupils may be disadvantaged. Schools entering a subject for the first time will have indicative grades based on the national transition rates. Departments that have genuinely been improving in 2020 may not have that improvement recognized.
But this could be one of the least unfair options given where we are, particularly (as some have suggested) if pupils have the option to resit in November (assuming life is returning to normality by then).
Want to stay up-to-date with the latest research from FFT Education Datalab? Sign up to Datalab’s mailing list to get notifications about new blogposts, or to receive the team’s half-termly newsletter.