It is actually quite hard to say anything about the causal impact of grammar schools. There is no such thing as selective and non-selective areas: 1 in 5 grammar school students cross over a local authority border on the way to school. And perhaps as many as 1 in 5 grammar school students were not in a state school at age 10. We can, however, watch what happens to children who attend the same state primary in an unambiguously selective area. It is likely that most able children will sit the 11+ – some will pass and proceed to a grammar school – others will end up in non-selective state or private schools.

Here, we look at children in just over 500 primary schools in grammar school areas that routinely see their pupils pass the 11+. Children who have come from the same primary school will have access to a similar set of grammar schools with similar pass rates, similar practice for 11+ tests within the school, similar classroom experiences and so only differ in the amount of parental support for their learning and private tutoring they receive.

We don’t have these children’s 11+ scores (which in any case are skewed by test preparation investment and missing for non-takers), but we do know how they did in their Key Stage Two tests just before leaving primary school. We look in these primary schools and find the child who scored highest in their Key Stage Two tests and yet did not attend a grammar school, calling them our ‘highest failer’. (We exclude children who go onto private schools since we do not know whether their decision to do so was influenced by their 11+ performance). We then find the child who scored lowest in their Key Stage Two tests and who went on to attend a grammar school, calling them our ‘lowest passer’. It is worth noting that our highest failers are twice as likely to be eligible for free school meals as the lowest passers (6% vs 3%).

What happens to our ‘highest failer’ and ‘lowest passer’ by the age of 16?

The child scoring highest at Key Stage Two who goes onto a non-selective school outperforms their primary school peer who ‘just’ passes their 11+, i.e. scores the lowest Key Stage Two mark of all those in the primary school going onto a grammar. They outperform in both broad measures of attainment – total point score and best eight subjects – and in GCSE English and maths (though not in science). The ‘highest failer’ takes more qualifications at 16 than the ‘lowest passer’, though fewer GCSE examinations. (Note: faded bar charts indicates difference not statistically significant, estimated using primary school fixed effects model.)

The child who ‘just’ gets into grammar school does worse at GCSE than their primary school peer at a state non-selective school

The child who ‘just’ gets into grammar school does worse at GCSE than their primary school peer at a state non-selective school

High Key Stage Two attaining children sometimes fail the 11+ while low Key Stage Two attaining children sometimes pass

Should we take this to mean that secondary modern and grammar schools are equally well-suited for high-attaining pupils? Or that 11+ selection isn’t a problem because everyone does well, regardless of destination? No. Even though the ‘worst passer’ must have passed an 11+ test, their average Key Stage Two scores are lower than the ‘highest failer’. It seems hard to believe, but there are children at grammar schools who only achieved a Level 3 at English or maths. Equally, some children at non-selective state schools in grammar school areas have Key Stage Two scores equal to the smartest pupils at grammar schools. We don’t know why the 11+ is a poor match for Key Stage Two attainment and the local authorities where the discrepancy is particularly high have nothing obvious in common.

Somehow children with very low Key Stage Two scores manage to pass the 11+

Somehow children with very low Key Stage Two scores manage to pass the 11+

Passing the 11+ leads to a very different secondary school experience

How lucky are these ‘lowest passers’ for passing the 11+? We already know they don’t do particularly well at GCSE, compared to their ‘highest failers’ from their primary school class, so it isn’t possible to argue that the 11+ is a better indicator of academic potential than the Key Stage Two tests. We can also show that they tend to find themselves amongst the weakest performers academically at the grammar school they attend. More worryingly, many of them will be labelled as having Special Educational Needs (School Action) by their grammar school, a status commonly given to children on the basis of emotional and behavioural difficulties.

At primary school, about 4% of our ‘highest failers’ and ‘lowest passers’ have the SEN School Action status; by age 16, about 10% of the ‘lowest passers’ at grammar school will have this status compared to 4% of the ‘highest failers’ at a non-selective state school.

The ‘lowest passers’ tend to find themselves struggling against their new grammar school peers

The ‘lowest passers’ tend to find themselves struggling against their new grammar school peers

How might our ‘highest failers’ have performed had they had the opportunity to go to a grammar school? Rather than compare them to the ‘lowest passer’, instead we find the child in their primary school class with who they most closely match on Key Stage Two scores but who attends a grammar. With this matched comparison we see that the grammar school attendee outperforms by about half a GCSE grade in their core subjects. They also appear to take a more traditional curriculum, with a greater number of GCSE subjects and fewer GCSE equivalent qualifications.

We cannot be sure these advantages are entirely down to the grammar school. Although they attended the same primary school and had the same Key Stage Two score, the very fact they passed the 11+ may not have been down to chance. They may have had greater academic potential that we cannot identify in Key Stage Two tests, or more likely came from the kind of family who helped them prepare for the 11+ exam and has equally supported their educational progress since.

The grammar school pupil outperforms a pupil from the same primary school with the same Key Stage Two score who did not attend

The grammar school pupil outperforms a pupil from the same primary school with the same Key Stage Two score who did not attend

This analysis doesn’t say anything about whether selective schooling systems are better for the average child, the low-attaining child, the high-attaining child, or anyone else. It simply points out the shortcomings of the 11+ exam across every highly selective local authority, regardless of whether they use verbal reasoning, non-verbal reasoning, numeracy, English or any other test paper.

There are children who routinely manage to pass the 11+ exam ahead of primary school peers who both score higher at Key Stage Two tests and must have greater academic potential since they go on to achieve more at GCSE. This means the 11+ exam is frequently less successful than Key Stage Two tests in selecting the highest potential children from primary schools. Differences in the social background of these pupils who ‘just pass’ compared to their brighter peers who fail suggest this is not simply due to chance.

This interesting analysis highlights the real challenges posed by trying to measure children’s academic achievement at age 11, let alone their potential. For some children, it appears that the 11+ tests that are used to gain entry into a grammar school do not measure their academic potential as well as Key Stage Two test scores. As a consequence of these measurement problems, undoubtedly many high-ability children miss out on a place at a grammar school. It remains unclear from this analysis however, whether this really matters for their future academic success.

The other issue this work throws up is the problem of the ‘big fish little pond’ effect (Marsh, 1987). This stems from the finding that lower-achieving children in a group of higher-achieving pupils will tend to have lower academic self-esteem, with negative consequences for their subsequent achievement. Children who perhaps are wrongly identified as high-achieving according to the 11+ test (due to the measurement error problem described above) are likely to suffer from the ‘big fish little pond’ effect in a grammar school full of higher achievers. They are then likely to have an even lower achievement as a result.

In a nutshell, measuring achievement at an early age is difficult and our selection systems need to reflect this.

Professor Anna Vignoles, Professor of Education (1938), University of Cambridge