We’ve written on a number of occasions about the European Computer Driving Licence, or ECDL.

It was announced in March 2017 that the qualification would no longer count in the school league tables, starting in 2018, after gaming concerns. (Namely, around the qualification’s equivalence in size to a GCSE in league table calculations, despite a small number of teaching hours, and the value of ECDL grades in these calculations being out of kilter with pupils’ results in GCSEs.)

As we observed earlier this year, ECDL entry rates varied enormously between multi-academy trusts (MATs) in 2017.

So we might wonder: now that the qualification no longer counts in school performance measures, how have the Progress 8 scores of MATs changed?

The data

There are two charts below, that show the change in MAT P8 scores between 2017 and 2018 (the first chart) and what those two P8 scores were (the second chart).

Trusts are grouped according to the proportion of their cohort who entered ECDL in 2017 – ranging from those where 20% or less of pupils took the qualification at the top of the chart, to those where more than 80% of pupils took the qualification at the bottom of the chart.

Schools included here are those that featured for a given trust in the 2017 MAT league tables – schools had to have been part of a given trust for at least three years in order to count. ECDL entry rates for these trusts are based on this same population of schools.^[1]

A clear relationship is visible, with those trusts that had higher ECDL entry rates on average being more likely to have experienced falls in their P8 scores between 2017 and 2018. (The correlation coefficient is minus 0.43 – a moderately strong correlation.)

Looked at another way, the table below puts trusts into bands based on their 2017 ECDL entry rates.

Among MATs with ECDL entry rates of 40% or less, on average there was no change in trust P8 score between 2017 and 2018, or a slightly positive change.

But among MATs which entered more than 80% of their pupils for ECDL in 2017, on average there was a drop in their P8 score of 0.13.

At an individual trust level, some of the falls are much greater: -0.38 at the Fylde Coast trust’s three schools that featured in the 2017 MAT league tables; -0.42 at the Priory Federation of Academies Trust’s four schools; and -0.48 at the Inspiration Trust’s three schools included in the analysis. Despite these trusts’ high ECDL entry rates in 2017, decreases of these magnitudes are unlikely to be accounted for by the removal of ECDL from performance measures this year alone – we think perhaps half of the change could be accounted for by this one factor.

(It is worth saying that there’s more variability in value added scores from year-to-year than there is in attainment measures, so we shouldn’t be surprised to see small changes at trust level.)

A couple of other things to say. Firstly, the 2018 data is provisional data that was published last month. Between the publication of this data, and the publication of revised data in January, schools will have had the opportunity to suggest corrections that are required.

And when the 2018 MAT league tables are published in January, as well as the schools included in the analysis here, the figures will take into account a number of schools that have opened or joined a MAT recently, which didn’t count in the 2017 MAT league tables.

But this analysis suggests that trusts with high ECDL entry rates on the whole may see their reported performance drop this year, in some cases by large amounts.

The upshot

While some might be tempted to take satisfaction from a tumble in the relative rankings of other trusts, particularly where performance in previous years has been propped up by ECDL entries, schadenfreude would be the wrong reaction.

ECDL has been in widespread use since 2015, with the number of entries soaring between 2015 and 2017, as the chart below shows. For context, the 150,000 entries in 2017 was more than a quarter of all pupils taking GCSEs that year.

Now that the main headline measure on which secondary school performance is measured is a zero-sum one, once some schools identify that there’s benefit to be achieved from entering certain qualifications or groups of qualifications, it is no surprise that others will follow suit.^[2]

So rather than taking pleasure in the righting of apparent league table wrongs, much better would be to focus on drawing attention to the pressures that lead to gaming behaviour.

Want to stay up-to-date with the latest research from FFT Education Datalab? Follow us on Twitter to get all of our research as it comes out.