So does data not matter anymore?

It does still matter, but the inspectorate is reducing the quantity, narrowing the focus, and hopefully reducing the associated workload, explains Colin Logan, senior education lead at SSAT

One of the aspects of the new inspection framework that has attracted a good deal of attention from schools has been Ofsted’s assertion that “inspectors will not use schools’ internal data as evidence.” In practice, it intends that inspectors will not look at a school’s internal data, even if it is offered to them. Inspectors have been left in no doubt that agreeing to look at such data (even on the quiet) is a complete non-starter. The ‘no internal data’ rule is seen as a symbol of the principles underpinning the new inspection framework; inspectors will break that rule at their peril.

Even if a school’s published data is lagging behind what school leaders think is its current level of achievement, inspectors have been told to stick to the rule of considering only published, validated data.

So, does this mean that data doesn’t matter anymore? Perhaps unsurprisingly, the answer is no. But there is a definite shift in emphasis, from data being the main source of evidence and the basis for judgement to data being part of a wider discussion with school leaders about what they are doing in response to any data they have collected.

The change is also in response to what one senior Ofsted official referred to as “feeding the data monster”: if schools are making demands on staff over supplying and analysing data that have a detrimental effect on workload, that will be an issue in itself.

(By the way, there is an exception to the ‘no internal data’ rule – data covering behaviour and attendance, where a school’s own in-year figures will be discussed during an inspection.)

The only internal data to be discussed during an inspection will be the school’s own figures on behaviour and attendance

There are also some significant changes to the way in which published data is used prior to an inspection. Inspectors will no longer have access to ASP (Analyse School Performance) which, along with its predecessor, RAISEonline, always played a major role in determining inspection key lines of enquiry and the subsequent judgements.

The IDSR (inspection data summary report) stays, but with some substantial changes to reflect new inspection practice. In the words of another senior Ofsted figure, “the heavy lifting has been done for inspectors so that they can focus on what is meaningful.”

When the 2019 IDSR is released later this term, there will be:

• less data and fewer charts
• more sentences interpreting the data, removing the need for users to work things out for themselves
• a minimised focus on small groups
• more contextual data, including workforce and financial information
• sentences highlighting potential areas for the new Ofsted ‘deep dives’
• attainment and progress trend data
• additional destinations data.

The clear intention here is to stop inspectors from doing their own analyses of data and bring about greater consistency in what is considered to be important.

The reduced emphasis on pupil groups is in recognition of the fact that while data can be important at national level, at school level it can be meaningless, depending on the numbers involved. It can even lead to unrealistic demands being made on schools and divert the focus away from improving teaching and learning. The new IDSR will only focus on areas where there is something noteworthy to say – for example, if the performance of disadvantaged pupils is clearly out of kilter with the rest of the school. Any data in the IDSR that isn’t considered to be significant for inspection purposes will be greyed out – only areas of significance will be highlighted.

While data can be important at national level, at school level it can be meaningless

The data on workforce will include staff absence, vacant posts and staff turnover, potentially leading to a discussion about teacher wellbeing and workload. The financial data will be on published end-of-year balances and per-pupil spend; this is intended to provide information about the context in which leaders are operating rather than making financial management part of the inspection. Among the DfE’s recent announcements on funding was an intention to have this included during an inspection, but there will be no immediate change. Sentences will show which subjects in the school are in the top and bottom 20% for attainment and progress nationally and where entry patterns might be changing. The intention is to provide evidence on subjects that go beyond Ebacc so that these too are subject to the deep dives that are part of the new inspection process. The trend charts will continue to use the now-familiar quintiles for trends over time but an addition this year will be arrows showing if there has been a significant change within a quintile year-on-year that hasn’t quite moved a subject up or down a whole group.

The new-style IDSR will be produced only for 2019 data onwards, so inspections until it appears will be based on the 2018 version. Inspectors will have access to the unvalidated checking data in the autumn term for 2019 results but without any accompanying national figures. Ofsted is hoping to release the new IDSR a couple of weeks earlier than in the past, so it should appear sometime from mid-October onwards.

The IDSR is available to both schools and inspectors. In the case of secondary schools, inspectors will also be provided with a separate data analyst briefing intended to highlight any additional noteworthy trends or patterns – for example, if there are potential concerns about entry patterns, off-rolling or ‘gaming’.

So, to answer the question we started with, data does still matter but there has been a clear shift in emphasis. To quote two principles used in inspector training:

“Data is a signpost for inspection, not the destination.”

“Move beyond the data as quickly as possible and ascertain how well the curriculum is being taught.”