Senior Education Lead Colin Logan writes…
Earlier this month I sat with over 100 delegates at SSAT’s data leaders’ conference in London where Tim Leunig, the DfE’s chief policy adviser and analyst, was giving the keynote.
It’s fair to say that civil servants on these occasions rarely shine because they are either too timid or have an insufficient depth of knowledge of their subject to move beyond their immediate brief. But I’ve heard Tim speak before and the effect this time was similar: he had the audience in the palm of his hand.
He not only has a rock-solid understanding of the issues but also communicates a passion for education and its moral imperative.
Grades’ changing values
Although most educationists agree that Progress 8 is, in Peter Kent’s words, “the least bad accountability measure yet”, there are two aspects of it that school leaders are concerned about. The first is a short-term issue: when the legacy GCSEs (with letter grades) are being counted alongside the reformed subjects (with number grades), grades B to F will count for less in performance tables than they do now, whereas an A* will be worth more in terms of points.
When the legacy GCSEs are being counted alongside the reformed subjects, grades B to F will count for less in performance tables than they do now
What’s more, the differential between each grade reduces at the lower end but increases at the top end, thereby apparently favouring schools with more able intakes.
Tim made a clear argument for the adjustment. Essentially, there is a much bigger gap in performance between a grade C and a B than there is between a G and an F. This difference will be removed with the new specification GCSEs, where the difference between grades is intended to be even across the scale.
It sounded a convincing argument to me, although I was left wondering why, having lived with the problem for the last 30 years of GCSEs, it was necessary to make the correction now, with unreformed GCSEs about to disappear, causing unnecessary aggravation and resentment among school leaders. Tim assured the audience, however, that the adjustments would make very little difference in practice to most schools’ overall scores, given that they will all be affected in a similar way.
I was left wondering why, having lived with the problem for the last 30 years of GCSEs, it was necessary to make the correction now, with unreformed GCSEs about to disappear, causing unnecessary aggravation and resentment among school leaders.
What about national progress?
The second and more important concern about Progress 8 results from the fact that it’s a relative measure (in other words one school’s performance depends on where it sits compared with every other) rather than an absolute measure (where schools have to hit a common threshold to be counted in or out).
So there will, inevitably be an equal number of schools above and below the national average. If the new floor standard, therefore, is set at -0.5 (meaning that, on average, each student gains half a grade less per subject than average), doesn’t this mean that there will always be schools falling below the floor, irrespective of the rates of progress of all schools nationally?
What I didn’t know about Progress 8 was that all schools can be above the floor standard. I’m not a mathematician, so please don’t ask me to explain how, but the calculations do not inevitably identify schools below the floor as a result of the law of averages.
The proof of the pudding, however, will be in the eating.
What I didn’t know about Progress 8 was that all schools can be above the floor standard.
Talking of which, all our speakers at the conference were unanimous in advising schools not to try to predict what their Progress 8 scores are going to be in the summer. Any attempts to do so are doomed to failure, because we simply do not know at this stage what progression from KS2 tests to GCSE is going to look like this year.
Nor do we know the extent to which each of the Progress 8 slots will be filled nationally, and how this will be reflected in individual schools as they gradually change their curriculum to a more Attainment 8-friendly pattern. It therefore goes without saying that to try to do so with any other year groups at this stage is equally futile.
Any attempts [to predict Progress 8 scores] are doomed to failure, because we simply do not know at this stage what progression from KS2 tests to GCSE is going to look like this year.
One of our speakers explained that all the current reforms will finally have worked their way through the system by 2030, after which we can expect a period of stability…
Louise Walder on said:
It is important that we make school leaders aware of the potential volatility over the next few years. One of the other speakers pointed out that students in current Years 11 and 10 with the same KS2 score could achieve exactly the same grades when they take their GCSEs, which for the year 11s would result in a positive Progress 8 while for the year 10s a negative value because of the way points are matched to grades.
I do not agree that the system necessarily favours schools with more able intakes, as it is all about relative performance. Students who arrive at a school with a higher KS2 score will be expected to attain higher grades. Having 1.5 points difference between the higher grades actually means that there is a higher points risk for students not attaining the “expected” grade.
If the floor threshold were set at P8 = 0 (ie your students make exactly the average progress) then indeed you would have to have schools below as well as above it. However by setting it at -0.5 (ie your students make half a grade less than average) then theoretically all schools could be above the threshold.