There has been some discussion on blogs and twitter in recent days about whether schools should try calculating Progress 8 on results day. Given that it is the measure used to define the floor standard, it is quite understandable that schools will want to know as soon as possible where they lie in relation to the floor of -0.5 points per subject below expectation. Not only that, internal school analysis of each year’s results informs improvement planning for the next year. Yet the general consensus is that so much is changing you would be best to wait and, indeed, this is the advice we gave earlier this year. Is this really the case?

Yeah. Why not use last year’s estimates to calculate Progress 8?

Surely schools can just grab the 2015 attainment 8 estimates and compare each pupil’s 2016 Attainment 8 score given their Key Stage 2 prior attainment? You could. It might tell you something. But how close will it be to the figure a school will see during the Autumn term when the Performance Tables checking exercise starts?

Let’s go back a year. Let’s imagine schools have just received their 2015 results. What if we grabbed the 2014 attainment 8 estimates and calculated a “results day” P8 measure? How different would it be to the actual P8 measure that would arrive some time later? We show the difference in the chart below.

For schools with lower attaining intakes there is very little difference although in some cases the “results day” measure would have under-stated the actual score but only by a small amount. We know that entries (and as a result Attainment 8) increased for pupils with lower prior attainment.

But the big differences are among schools with higher attaining intakes. Actual scores tended to be better than those that might have been calculated on results day, sometimes by a large margin. One Grammar school changed from a “results day” measure of +0.37 to an actual measure of +0.68.

This is a legacy of the Key Stage 2 boycott of 2010. All pupils affected who were assessed at level 5 by teachers were allocated a Key Stage 2 fine level of 5.5 for the purposes of Progress 8. If they had been tested, they would (mostly) have ended up in lower prior attainment bands. The 2015 estimate for this group certainly cannot be trusted. In fact, it is lower than the estimate for Key Stage 2 fine level 5.4. If it was to be used in the calculation of P8 for 2016, pupils with an average fine level of 5.5 would have estimates that were too low and so the “results day” P8 score would be too high compared to the actual.

Can we really not do anything?

So this year we won’t have the problem of the boycott. However, its legacy blights the published Attainment 8 estimates for 2015.

We could recalculate the 2015 Attainment 8 estimates stripping out the pupils affected by the boycott. Such a table is presented below if you really want to use it. Just bear in mind that it ignores the increase entries this year in subjects which count in the Ebacc buckets of Attainment 8, which will affect pupils with lower prior attainment more than others. As a result, you might calculate a P8 score that is slightly too high. But it might still be a useful starting point, albeit for internal consumption only.

The danger is that schools put a lot of wasted effort into calculating numbers which a) will change and b) may not be particularly helpful anyway. If, for example, you calculate a low P8 score that leaves you close to the floor standard, it won’t provide you with much insight about where to target improvement efforts. Would it not be better to cut out the middleman and just examine departmental performance as you would have done in previous years?

Perhaps the best thing to do is follow the sage advice of headteacher Stephen Tierney: Don’t rush to calculate Progress 8!

Thanks to @DataEducator for his comments on a draft of this post.