Last year, seven schools were brave enough to measure their progress in English and Maths in a scientifically designed experiment run by FFT’s Education Datalab.

We gave pupils a short set of tests in English and Maths at the start of September, and then again at the end of June. The tests were distributed so that no pupil sat the same test twice. The questions were open-ended and creative, with questions in Maths such as ‘Why do we need negative numbers?’ and in English based on creative writing prompts. The open-ended tests allow us to measure the application of learning and conceptual understanding which forms the basis of the reformed GCSEs.

The tests were judged by subject experts in a Comparative Judgment framework so that every pupil received a performance estimate in September and June which is fair and free from bias. In June, tests from September were mixed with tests from June to ensure that the measurement scale was constant and standards across judging sessions maintained. Judges were blind to which school a pupil came from and which tests were taken in September and which tests were taken in June.

Using a statistical procedure, the independent samples Bayes Factor t-test, we we then able to compare the hypothesis that there has been no improvement in performance over the year, to the hypothesis that there has been an improvement.

Progress in English

In English there was strong evidence that 5 out of the 7 schools had improved their performance over the year.

Figure 1: Progress in English: The dots show the school mean, while the bars show the school standard error

Progress in Maths

In Maths, however, there was only strong evidence of improvement at two out of the seven schools.

Figure 2: Progress in Maths. The dots show the school mean, while the bars show the school standard error

Progress Relative to Each Other

In today’s world of comparable outcomes performance relative to other schools is arguably more important than absolute performance. There was only strong evidence of one school changing performance relative to the other schools, and that school showed both a decline in absolute performance as well as a decline relative to other schools in both English and Maths.

Are you Brave Enough to Join the Experiment?

We have shown that under carefully controlled conditions it is possible to produce objective and standardised measures of progress in English and Maths Year 7.

If you are brave enough to join us and find out if your pupils are making progress, there are still a limited number of places available for this year’s larger scale trial. Contact Laura.james@fft.org.uk by the 7^th September 2015 to register your interest. Participation is free this year, and in return for your participation you will receive detailed individual and department level feedback.