The gap in attainment between disadvantaged pupils and their peers is rightly given considerable attention by those at all levels of the education system.

In recent years, this gap has narrowed.

Despite this, for the cohort who sat their GCSEs in the summer of 2018, fewer than half – 47.0% – of disadvantaged pupils in mainstream schools achieved a grade 4 or above in English and maths, compared to 72.6% of non-disadvantaged pupils. (Grade 4 or above is equivalent to the old grade C or above.)^[1]

But we think the real disadvantage gap is bigger than this.

School league tables and gaming

With a few exceptions, the pupils who count in a school’s GCSE results, and who get counted in national statistics, are those who remain on-roll when the school census is carried out in January of Year 11.

By and large, if you’re not on the roll of a school at this point, you don’t count in these statistics.

But only counting pupils who remain on the roll of a school towards the end of Year 11 makes for an unfair comparison between schools. Schools with highly inclusive practices are compared to schools that will have lost more of their pupils along the way. Those who leave school rolls are more likely to have low attainment. In a minority of cases, these pupils will have been lost intentionally, as a form of league table gaming, to boost apparent performance. Ofsted’s work on off-rolling bears this out.

Our proposal – and one that received the backing of the Education Select Committee last year – is that school league tables should instead take into account all pupils who have spent time on-roll at a school, in proportion to the amount of time spent on-roll.

We think that would make for a fairer system all round, and one with reduced scope for gaming.

But, what we’re going to call the real disadvantage gap is larger when you do.

Measuring the disadvantage gap

The gap in the percentage of disadvantaged and non-disadvantaged pupils achieving a grade 4 or above in English and maths currently stands at 25.6 percentage points, as mentioned above – 72.6% minus 47.0%.

But reweighting school performance measures so that all pupils are taken into account, in proportion to the amount of time they spend on a school’s roll^[2], only 45.2% of mainstream schools’ disadvantaged pupils achieved a grade 4 or above in English and maths, with a smaller drop, to 72.1%, for non-disadvantaged pupils. The gap therefore goes up to 26.9 percentage points – an increase of 1.3 percentage points.

(Figures for both disadvantaged and non-disadvantaged pupils drop as we are including pupils who have left mainstream education to other destinations where attainment is lower on average.)

While that might see a small difference, improvements that we see in the disadvantage gap are often measured in fractions of a percentage point.

At individual school-level, the effect ranges from the disadvantage gap decreasing by 13.9 percentage points, to increasing by 24.4 percentage points – adding weight to the argument that in-school disadvantage gaps are not particularly meaningful. For the majority of schools the effect would be somewhere between a decrease in the disadvantage gap of 2.5 percentage points and an increase of 5 percentage points.

We see something similar – and, if anything, starker – if we look at other headline school performance measures. As the chart below shows, disadvantage gaps in Attainment 8 and Progress 8 scores grow by even more once all pupils are taken into account, in proportion to the amount of time they have spent at a mainstream school.

As we write in an accompanying blogpost, nearly 24,600 pupils from the cohort who finished secondary education in 2018 left mainstream state education and weren’t seen again, around a third of whom were disadvantaged pupils. Some of these pupils will have left England or, in a small number of sad cases, died, but we estimate that 6,700-9,200 of these pupils remained in England and yet either did not count in school league tables or else took no qualifications.

To arrive at the above figures on the disadvantage gap we’ve taken the very conservative approach of not taking into account pupils who did not take any qualifications, or, if they did, whose results are not attributed to an establishment currently, whether a state school, alternative provision or independent establishment. (Because we can’t know which of this group of pupils has left the country, the alternative would be to ascribe them all GCSE results of zero.) Our calculations show that including these pupils would only increase the size of the disadvantage gap further.

What next?

What can be done about this?

For starters, until school performance measures are reweighted to take into account all pupils, in proportion to the amount of time they have spent on-roll, there will be incentives to carry out league table gaming, which a minority will be inclined to take advantage of. While the increase in the disadvantage gap that would result makes for uncomfortable reading, that alone isn’t a reason not to do it.

We have a number of other proposals, but first read more on the increase in pupil moves off the roll of mainstream schools. (Those short on time can skip ahead to our recommendations here.)

Want to stay up-to-date with the latest research from FFT Education Datalab? Sign up to Datalab’s mailing list to get notifications about new blogposts, or to receive the team’s half-termly newsletter.