Update 20/03/20: this post was written before the Department for Education published further details on how grades will be awarded in 2020.
*Thanks to John Jerrim and Steve Wren for comments and suggestions. You can read more of John’s thoughts on this topic here*
With no GCSEs and A-Levels this year, everyone’s wondering just how pupils will be awarded grades in the summer.
There is no alternative to teachers awarding them. But can there be any quality control in the system?
Ofqual already has the answer – the much unloved approach of comparable outcomes. This assumes that if the prior attainment (at Key Stage 2 for GCSE; GCSE for A-Level) of pupils entering a subject doesn’t change then the grades awarded won’t change either.
Ofqual could produce an indicative range of grades for each school to award in each subject. This would be based on three things. Taking GCSE as an example these would be:
- Last year’s Key Stage 2 to GCSE transition matrices
- The list of pupils entered in a subject, together with their Key Stage 2 results
- Historic Key Stage 2 to GCSE value added data
An example of a transition matrix, for GCSE geography, is shown below.
Using the transition matrices, the probability of each pupil in the list of entrants achieving each of the possible grades can be inferred. For instance, someone at level 4a has a 19% chance of grade 5, a 14% chance of grade 6 and so on.
Once the probability of each grade for each pupil has been calculated, the average probability of each grade for a subject at a school can be calculated. In the example below, the school has 10 pupils entered in geography in 2020.
So based on the average probabilities, the following indicative grade ranges could be awarded to the 10 pupils:
- two grades in the range 7 to 9 (corresponding to an average probability of 20% for these 10 pupils: 9% + 7% + 4%)
- three grades in the range 4 to 6 (corresponding to an average probability of 34% for these 10 pupils)
- five grades in the range U to 3 (corresponding to an average probability of 45% for these 10 pupils)
With larger cohorts, indicative grades could be provided rather than ranges such as 7 to 9.
Ofqual could go further. It could tailor the set of indicative grades according to historical value added data for each subject at each school. So if an additional 10% of pupils at this example school historically achieved a grade 7 or above (grade A or above in legacy GCSEs) in geography then an extra grade in the range 7 to 9 could be awarded.
It would be up to the school to decide which pupils to award these grades to.
Alternatively, the grades could be awarded “blind” as far as the school is concerned. Schools could upload a list in which pupils A to J have been put into ranked order of attainment based on the evidence available to teachers. Grades would then be assigned by Ofqual using the method above. This would help avoid schools making difficult decisions about, for example, which pupil should get the last grade 5 available.
There is also the National Reference Test, which was taken in February. This might suggest an increase in attainment in the cohort nationally. If there is, the transition matrices could be tweaked accordingly.
None of this is perfect. No system of awarding is perfect – even exams can result in some pupils being awarded the “wrong” grade.
There will be winners and losers. This method may not work well for schools with small cohorts. Some groups of pupils may be disadvantaged. Schools entering a subject for the first time will have indicative grades based on the national transition rates. Departments that have genuinely been improving in 2020 may not have that improvement recognized.
But this could be one of the least unfair options given where we are, particularly (as some have suggested) if pupils have the option to resit in November (assuming life is returning to normality by then).
Want to stay up-to-date with the latest research from FFT Education Datalab? Sign up to Datalab’s mailing list to get notifications about new blogposts, or to receive the team’s half-termly newsletter.
Thank you for this. What about students for whom English is an additional language & who either didn’t sit their SATS, or underperformed because their English wasn’t as good? What about schools that have been rebrokered from inadequate?
Hi Belinda. Pupils without KS2 results would be included in the same way- there would be an additional row in the first table to include them. To be clear, I’m not suggesting using KS2 results to allocate grades to pupils, just the set of grades that a school can award to its pupils. Not sure what you mean about the rebrokered schools though- they would be included. All of this is far from ideal though.
What about independent schools? Use Midyis and ALIS instead of KS2?
I assume there are MIDYIS to GCSE matrices so it should work.
You are correct in that all solutions have flaws.
It would make far more sense to delay the start of the next school year until January so that until that time students stay in their current year groups when they return . Exams could then be sat late September with the university year for new entrants starting in January.
The current proposals are knee jerk reactions that are lack due consideration of other options
What about where a department within a school has made significant improvements this year. The historical data would not reflect the quality of Teaching and Learning for this cohort?
Also a school that has been sponsored. How far back would the historical data be considered?
Most departments make incremental improvements but there are some that make dramatic improvements in a single year. They tend to be, but not always, subjects that had poor ‘value added’ previously. We could set all schools with a historic value added below zero to 0 and implement a small uplift for others on a sliding scale to reflect that they may also have improved.
Yes this would bring the national stats out of kilter with previous years – but let’s be honest this year group will always be different and it only seems right to give them whatever benefit of the doubt we reasonably can.
With reference to ‘historical data’, my only concern is where history begins. For example, we are in our third year as a new school with a different DfE number to our ‘previous’ school following a MAT takeover in 2017.
Since that point our IDSR places us in the top 20% of all schools nationally in all but one category. This is a different picture to pre- 2017. Would our ‘historical data’ therefore begin in 2017 and not prior to that and in that way, not do an injustice to our current year 11 cohort?
I’d do it at subject level not school level (most schools have ‘stronger’ and ‘weaker’ departments whatever the overall picture).
In terms of how far you’d go back? Various options:
Best of last 3 years? (This would obviously be erring firmly on the side of positivity)
Last year? (Risks disadvantaging a cohort based on what might have been a freak poor year previously)
Sliding average (eg. 50% of last year; 35% of year before and 15% of year before that)
None are without flaws – no solution to this problem will be flawless though.
The injustice served by removing the opportunity to sit an exam would be compounded even further by what has been outlined here. As has been stated above, the cancellation is a huge over-reaction. A postponement of a definitive decision to allow time to develop plans to restructure the academic year, for this cohort, would have allowed pupils the opportunity to complete their qualifications fairly. The lack of reasoned thinking or rationale behind the cancellation is deplorable. In my context, the students have made incredible progress, they deserve the opportunity to prove this and receive qualifications that reflect it.
My work in relating Cognitive Ability Test Scores (CATs) with SATs suggests that SATs are an unreliable basis for predicting GCSE outcomes for ‘disadvantaged’ pupils.
The data I have acquired confirms that SATs results are inflated when compared to Non-Verbal Reasoning test standard scores, and especially for pupils with lower CATs NVR scores. This has serious implications for secondary schools, especially in relation to setting Attainment 8 and Progress 8 targets, especially for those schools with high numbers of FSM children on roll, because we know from data published by GL Assessment, who provide the Cognitive Ability Tests, that FSM children, on average, have lower cognitive abilities.
For example, I used FoI to obtain the data for the nine CATs Non Verbal Reasoning Test bands used [for admissions purposes] by one large school. These give the number of pupils in each band in brackets. Underneath each band the mean scaled SATs score for reading and for maths are provided, in that order.
NVR Band 1 69-73 (9) corresponds to -2 SD (2nd percentile)
SATs 95, 97
Band 3 82-88 (25) to -1 SD (16th percentile)
NVR Band 1 pupils should presumably be performing at the 2nd SATs percentile. We do not know what SATs score this corresponds to, but it is certainly not 95/97
NVR Band 3 pupils should be performing at the 16th SATs percentile. This cannot be 102/104
This is like taking the bottom of the regression line and moving it up so that -2SD becomes 96 instead of 70, which is statistical nonsense. I hypothesise that this pattern results from primary schools, having a high proportion of low NVR pupils, resorting to cramming and coaching methods to meet the DfE floor target. Such children will have understood little and forgotten most of it by the end of the summer holidays, which is what hundreds of secondary schools report as the reason why they buy the Cognitive Ability Tests (CATs) to reliably inform diagnostic and target setting interventions for their pupils.
So, it emerges from my analysis that SATs scores are systematically inflated for pupils of lower cognitive ability, and the lower the cognitive ability score, the greater the inflation.
Hi I’m just wondering if you could offer me some help/advise I’ve been working towards my GCSE Maths since September 2020 but as I had holiday pre booked wasnt able to be registered for exam until Nov this year.would I be able to get predicted grade too? As I’ve done whole course or would I have to wait sit the exam in the November. Any advice would be appreciated kind regards
We are on an upward trajectory as a school so I hope this would be recognised – we had a big hike last year and were expecting even better this year.
A pupil comes to secondary school from a failing school that did not prepare its pupils for SATs. The secondary school has made huge steps to help that pupil achieve good mock results far higher than the fft prediction. One teacher tells the parent two years in a row that the data on fft did not reflect the pupils levels ,down to how the data is asked for. (3+ fft to 8 on both mock exams) . This suggests there may be a flaw in the way fft collects its data for some subjects. Or how it processes it.
On a personal note, I’m furious! My daughter got a 4b in her Sat’s, so, according to your probabilities she has a minuscule chance of getting a grade 8, and zero hope of a grade 9. Yet, over the last three years, she has devoted herself to school and received several grade 8s and a grade 9 in her latest round of mocks. To have her grades decided by an anonymous suit who has no real idea of her character or progress and is following some stuffy formula is sickening. Teaching is a social vocation, teachers should know their kids and be trusted to make these “difficult” decisions about their grades.
Hi Patrick. Don’t worry- no one is proposing awarding grades to pupils based purely on their KS2 results. Today’s update (https://www.gov.uk/government/publications/coronavirus-covid-19-cancellation-of-gcses-as-and-a-levels-in-2020/coronavirus-covid-19-cancellation-of-gcses-as-and-a-levels-in-2020) states that a range of evidence will be used- Ofqual and awarding bodies are working on the detail. KS2 results will be used to help ensure consistency between schools and with results awarded in previous years, not to make decisions about individuals.
My daughter is predicted 5 for maths but got a 4 in her mock exams, what GCSE grade is she likely to be given ?
What Date are both year 6 (SATs) and year 11 (GCSE) results due out in England? I have children doing both