SSAT’s Inspection Tracker
As mentioned in a previous blog on attendance, we have been tracking every ‘need to improve’ comment written by inspection teams during their visits to schools and colleges in 2023/24. At the time of writing, there were almost 4,000 such comments. By the end of the school year there were almost 12,000. With such a rich dataset at our disposal, and more to come from June and July inspections, we wanted to share some further insights.
We wanted specifically to have a look at whether there are any notable trends or microtrends that could alert school leaders to an ‘Oliver Effect’. By comparing the first quarter of 23/24, when the inspectorate was led by Amanda Spielman, and the second and third quarters under Martyn Oliver, could we spot a noticeable shift in what the inspectorate are prioritising? And would such a shift give us any clues as to a direction of travel for the coming academic year?
Trends in overall judgment and progress
Let’s start with overall outcomes. Between September and December 2023, the proportion of comments for schools judged to be outstanding and good was 75% and schools deemed to be requiring improvement or inadequate contributed 25% of all ‘need to improve’ comments. After January, the distribution of ‘need to improve’ comments was 79% for outstanding and good schools and 21% for inadequate and requiring improvement schools.
It is worth noting that our tracking focuses on ‘need to improve’ comments rather than schools. So, this data could indicate one of two things. Either, under Oliver, there have been significantly more schools deemed to be good or better, or that schools judged to be less than good have been receiving significantly fewer ‘need to improve’ comments in their reports. Whichever of these is true, and it could be a combination of both, there appears to be a pivot away from challenging judgments against schools and towards a greater recognition of their strengths.
This insight appears also to be borne out by the data regarding whether schools have improved, remained static, or declined relative to their previous inspection judgment. ‘Need to improve’ comments for improving schools remained static under Spielman and Oliver at 14%. The proportion of comments for declining schools, however, fell from 25% before the Christmas break to 21% after it, as the number of comments for schools retaining their previous grading increased from 61% in late 2023 to 65% in the first half of 2024.
On 2 September 2024 the DfE announced that single-word overall outcome judgments would cease to be used on inspection reports, with immediate effect. Although this was a political decision, not an inspectorial one, it aligns well with the points raised above about an ‘Oliver effect’. The decision to remove overall, single-word grade means that it makes sense to explore more closely the trends in the commentary (rather than judgements) made by inspection teams.
Trends in the Education Inspection Framework
There have also been some interesting trends regarding key elements of the inspection framework that might point to an ‘Oliver Effect’. Across the 23/24 academic year, almost exactly three-quarters of ‘need to improve’ comments concern the quality of education. In the first quarter of the school year (under Spielman), the figure was 75.7% but across 2024 (under Oliver) the proportion has dropped to 74.8%. The drop is much smaller for primaries, from 80.3% to 80% across the two HMCIs. For secondaries, comments related to the quality of education dropped from 61.7% of all ‘need to improve’ comments to 59.4%. Pupil Referral Units have seen the greatest drop-off in the proportion of quality of education comments, from 69.6% under Spielman to 50.9% under Oliver. Special schools have seen the proportion increase by 3%.
‘Need to improve’ comments about leadership and management, and about personal development have also seen a decline. PD is the Cinderella of the framework garnering only 5.1% of comments before Oliver and 4.9% since his arrival. L&M comments have dropped from 9.9% under Spielman to just 9.0% under the new HCMI. And so, you will have rightly concluded, the big increase (from 9.3% to 11.3% of all needs comments) has been for recommendations around behaviour and attitudes. Attendance, perhaps unsurprisingly, has been key to this.
Before January, 4.3% of all ‘need to improve’ comments concerned attendance. Since January, that figure has risen each quarter and, for 2024, attendance has been mentioned in 6.5% of comments. Secondaries have seen an increase in attendance-related needs from 9.3% to 12.3% and PRUs have seen in increase from 18.5% to 20.2%. There is every reason to believe that attendance will continue to loom large(r?) for inspection teams, given the newly statutory footing of attendance guidance and the challenges being encountered nationally by schools. At the same time, it is worth noting that quality of education remains, for now, the key judgment for all sectors.
Other trends and microtrends
Beyond attendance, what other trends and microtrends have we noticed that might interest and assist school leaders? Let’s start with the bigger and more clearly delineated trends. First of these must be formative assessment. Strong under Spielman as a judgment (13.4% of all needs comments), it has grown even stronger under Oliver (14.5%). It is easily the most common of all identified needs for mainstream settings. Curriculum planning and implementation figure more highly for PRUs and special schools but have declined for other schools. This suggests that for mainstream schools there is an expectation that this work will be completed by now. Instead, more focus has been placed on refining the curriculum (up from 7.6% of all comments to 8.5%).
There are other areas where a recent decline in comments might point towards an increasing expectation that the work will have been done already. For example, phonics and reading which, before January, featured in 8.5% of needs comments have only done so in 6.8% of comments since. Support for SEND students as a ‘need to improve’ has also declined under Oliver, from 5.2% of all comments in late 2023 to 4.7% since the start of 2024. A question for school leaders is therefore whether gaps in these areas might now constitute greater risk because they are less common across schools.
Beyond the quality of education and attendance, most ‘need to improve’ comments represent less (often much less) than 2% of all comments. But there are some interesting microtrends within these that are worth watching as we head into 2024/25. Here are some of the most intriguing changes from the periods before and after January for me:
- Writing needs have almost doubled from 1.0% to 1.9% and continue to grow.
- Outcomes and progress references are up from 0.2% to 0.7% under the new HMCI.
- Disadvantaged students feature more often in both attendance and learning comments.
- Behaviour comments are more likely to be about the school policy than the students.
- Knowledge of other faiths/cultures has increased from 0.8% to 1.1% of comments.
- Criticism of governors has fallen from 2.0% of comments to 1.6% and, perhaps related.
- Comments about leaders have increased slightly (capability, planning, monitoring).
- Safeguarding concerns have declined from 1.6% to 1.3% of all comments.
Conclusion: An Oliver effect?
Football fans are increasingly using the term ‘eye test’ to counterbalance the prevailing trend towards data-driven evaluation of teams and individual players. The belief is that sometimes the numbers, however fascinating, might not quite tell the full story. The same might be a wise approach to evaluating whether there has been an ‘Oliver Effect’ in the conduct of inspections. Do the trends and microtrends identified above correlate with what we see and hear of the Chief Inspector? Are they consonant with his approach as a school and trust leader? These are wise questions because data is ever evolving as are the politics in which the head of Ofsted operates (perhaps never more so than during this election year and its emerging legacy).
At SSAT, we will continue to track the ‘need to improve’ comments of every inspection through the coming academic year. In doing so, we will see whether behaviour and attitudes (especially with regards to attendance) continues to nibble away at the primacy of the quality of education for inspection teams. We will see whether formative assessment continues to be the most challenging thing for school leaders and staff to get right. We will see whether outcomes and progress continue their mini revival, whether students from disadvantaged backgrounds remain a focus, and whether less pressure on governance (and more on leaders?) is sustained.
With single-word overall outcome judgments consigned to the dustbin of history, and with further changes likely to emphasise commentary over judgement, the insights our inspection tracker provides can help school leaders. If you would like to know more about our inspection tracker and what it appears to be telling us, join us on Thursday 10 October 2024 for our next ‘Readiness for Inspection’ webinar. At this session, we will show you up to date charts and graphs from the tracker, identify more of these trends and microtrends, and personalise the session to take account of the sector in which attendees work.
