SSAT policy briefing paper – 30 April 2020

Leading teacher assessment and ranking: read the latest policy briefing paper from Tom Middlehurst, Head of Policy and Public Affairs at SSAT. In this update, we took a closer look at how heads, principals, SLT and curriculum leaders might go about the process of teacher assessment and ranking exercises during May, ahead of the submission opening on 1 June.

During the webinar, I was joined by Dylan Wiliam, author of SSAT’s highly regarded Embedding Formative Assessment programme, and former teacher turned football-scout Chris Baker, whose blog many SSAT members have found helpful. You can watch Dylan and Chris’s contribution to the discussion; unfortunately, there was a technical issue with the introductory section of the webinar – what follows is a written outline of what I said about the process and the principles behind it.

SSAT’s guiding principles

• Make decisions in the interests of students, not of the school. We know school performance matters to leaders, governors/trustees, and the community, but this year is about the students who are unable to sit exams in the normal way. This may be easier for year 11s sitting GCSEs, year 12s sitting AS and year 13s sitting A-levels, as performance tables for these cohorts won’t be published. It may be more of a challenge for early entries, who will gain the qualification but whose results won’t count in subsequent years. Leaders will have to make tough choices, but we urge schools to put the needs of students first.
• Teachers know their students best. Use a range of evidence but trust teachers to make the professional judgements. Support the teachers who will need the most guidance – including less experienced staff.
• National data/previous data is a useful sense-check, but isn’t necessary and shouldn’t be the starting point. Transition matrices, prior attainment and previous results could be considered, but ultimately start with the professional judgements of your own teachers – not a depersonalised data set.
• Educational inequalities exist. Centres should be aware of bias but shouldn’t get too hung up on correcting that this year. This might seem counter-intuitive, but you’re being asked what students are most likely to have achieved had they sat the exams. Given educational inequalities exist within the system, these are likely to be reflected in the teacher assessments for 2020. It’s no good thing, but it’s probably accurate.
• If we want to be credible as a system, we need to submit realistic grades. Lots of SSAT members tell us that they don’t think the current exams system is fit for purpose; here’s an amazing opportunity to change it. Imagine if the national standardisation is hardly needed; and that we can show teacher assessment is strongly correlated with expected exam outcomes. That would be a great argument for moving away from national testing. But if a large minority, say even a fifth, of schools inflate their grades and are marked down, it will undermine any argument for teacher assessment for a very long time.

Things not to do

• Inflate grades for the sake of it/to try to the game the system. Obvious, really; but we understand the desire to help your students as much as possible. Submitting inflated grades is not only unethical but undermines trust in the profession.
• Seek a perfect match to national grade distribution or prior attainment. As above, use data sets as a sense-check, but don’t worry too much if your submitted grades don’t exactly match the transition matrices or your previous results. It’s more important you enter an honest grade for each candidate.
• Allow parents and students to influence teachers’ decisions. As SLT, what steps are you taking to protect teachers from harassment or bribes; even the odd email explaining why Nisha performed poorly in January mocks, or that ‘Terrence is having private tuition, perhaps you didn’t know’ – must be discouraged.

Possible process

1. Teachers make teacher assessment of what students would likely have achieved in the summer exams; and how secure they’d be in this grade
2. Simultaneously, create an initial ranking list from data available
3. Include private candidates, if you have sufficient evidence to make a judgement
4. Adjust list as needed based on teacher assessment
5. Go through rank line by line as a whole team; adjust again
6. Sense-check using national and contextual data
7. Final checks
8. Submit from 1 June

Teacher assessed grade

• Remember, and remind teachers, what you are being asked for is the most likely grade that students would have achieved in the summer, had they sat exams. Or, as Dylan Wiliam put it, what is the grade they would have achieved on an average day (not at the best, not at the worst), on an average exam paper (not one that did or didn’t work in their favour). This is therefore not a target grade (which should include challenge), a working-at grade (which is likely to have been last assessed in March), nor even a ‘deserved’ grade (we all know those students whose classwork is better than their exam performance – again, remember, you’re being asked for your judgement on what they would have got in the exam).
• Use whatever evidence you have available to you – classwork, written work, mock exams, prior results, teachers’ knowledge and expertise.
• Be careful about over-relying on assessment outcomes after lockdown started. This might be useful in some cases, especially for private candidates, but remember that times are strange, and that assessments done in normal school circumstances are likely to be more useful.
• To help the ranking exercise, ask teachers at this point to also consider how secure they think students are within that grade.

Ranking

• You will need one list, with each student ranked from top to bottom within each grade, for every qualification you are submitting candidates for. As such, all year groups, including private candidates, need to go into the same list.
• It is likely that a school’s rank order will not be changed, unlike the teacher assessment grades. As such, pay particular attention to the top and bottom students within each grade; these are the ones most likely to move up or down.
• Create the initial ranking from an existing data set; don’t try to do this from scratch.
• Involve all teaching staff in agreeing the final rank order. Agree ground rules for the meeting. Go through line by line; don’t skip ahead. Regardless of personal disagreements about the ranking of certain students, ensure the department ‘own’ the final list as a team.
• Use national distribution data, transition matrices and your school’s previous results as a sense check. It won’t matter if there are some differences; but if last year you got 55% at 5+, but are submitting 90% this year with a similar cohort; something’s probably not right.

We know this process is going to be challenging, and there are lots of questions and uncertainties remaining. SSAT is here if you have any specific queries or need some advice. Good luck.