A couple of weekends ago I attended the London Festival of Education. The “London Effect” in secondary schools, a topic on which I have blogged previously, was much discussed.

One of the important contributions to the debate, cited by a number of presenters at the festival, was made last year by Simon Burgess from Bristol University. He found that the London Effect could be explained by differences in ethnic composition between schools in the Capital and those elsewhere.

The paper created much heat and a bit of light. It upset some who believed that the London Effect was real and gave ammunition to those who did not. Some, such as Chris Cook of the BBC, argued that the London Effect was still evident when outcomes in academic qualifications (excluding vocational qualifications) were analysed. In fact, Burgess had found such an effect in his research but in a follow-up blog, he asked whether it was “legitimate to decide now, after the fact, that some qualifications count less or not at all in a measure of what schools do?” This was a fair question. Would schools have entered pupils for vocational qualifications if they knew they would not be counted in Performance Tables? At the time the research was published there was no easy answer to this question resulting in an impasse.

As luck would have it, the rules of the School Performance Tables game changed following the Wolf review and the consequences can be seen in the 2014 data. In brief, the changes were that:

• Large numbers of previously eligible qualifications were no longer counted
• No qualification was counted as more than one GCSE
• A maximum of 2 non-GCSE qualifications per pupil were counted
• ‘First entries’ rather than ‘best entries’ were counted in some subjects (English, maths and other EBACC subjects).

This led to some substantial changes in qualifications offered by schools which we will cover in greater detail in a later blog. In short, pupils entered fewer qualifications overall but tended to enter more GCSEs compared to previous national cohorts. So what if Burgess had done his research on 2014 data?

To investigate this, we firstly run a simple value added analysis of KS2 test scores (in English, mathematics and science) on capped ‘best 8’ points plus English and mathematics ‘bonuses’ for pupils attending state-funded mainstream schools. This is the outcome measure is that used in DfE value added calculations. The results for 2013 are shown in Table 1. This largely concurs with Table T3 in Burgess’s paper. Pupils in London schools achieved, on average, almost 12 additional points in the outcome measure. This equates to an additional 2 grades on average across the 10 subjects counted in the measure. However, some groups, such as Bangladeshi, Black Caribbean and Black African, appeared to make less progress in London than in the rest of England

Table 1: Key Stage 2 to Key Stage 4 Value Added, Pupils in State-funded Mainstream Schools 2013

We then repeat the analysis for 2014. This time, however, the outcome measure is based on the new rules for School Performance Tables. Results are shown in Table 2. Compared to 2013, the London effect is larger at 19 points, equivalent to three additional grades on average in the 10 subjects counted in the measure (or 0.17 standard deviations). The average VA score is higher in London for all ethnic groups apart from Black African (the 2^nd largest group in London), including Bangladeshi and Black Caribbean groups which appeared to make less progress in London than elsewhere in 2013.

Table 2: Key Stage 2 to Key Stage 4 Value Added, Pupils in State-funded Mainstream Schools 2014

Of course, I do not control for ethnicity and other factors, such as English as an additional language and free school meal eligibility in Tables 1 and 2. Doing so removes the London Effect in 2013 as Simon Burgess found. But for 2014 a small positive effect for capped points is apparent at almost 4 points (little more than 0.04 of a standard deviation or a quarter of the size of the effect controlling just for prior attainment).

This does seem to suggest that if Burgess updated his research using 2014 data he might conclude that there is a small London effect after all. As I noted in the previous blog I wrote on this subject, schools in London tended to maintain more traditional, GCSE-based curricula and enter pupils for fewer vocational qualifications compared to schools in the rest of England. This meant that they were better placed to respond to the changes in the rules of engagement for Performance Tables so that the performance of London has once again pulled away from the rest of England in 2014. They may also be better placed to respond to accountability changes Post-16 as I will discuss in a future blog.