Reading time: 4 minutes. Related programe: Embedding Formative Assessment
Dylan Wiliam, emeritus professor of educational assessment at UCL, explains the evidence for the cost-effectiveness of formative assessment.
Twenty years ago, Paul Black and I published an article on the impact of classroom assessment practices on student learning. We began by looking at the effects that tests, exams, grades, and other ways of reporting student achievement had on students’ motivation and attitudes to learning. Predictably, we found that such reports often had a demotivating effect, which tended to lower achievement. However, when we looked more deeply, we found that assessment could also enhance learning, especially when it was used by teachers, students, and their peers, to make decisions about how to adjust what was happening in the classroom to better meet students’ learning needs.
To explore the extent to which formative assessment actually worked in the classroom, Paul Black and I, together with Christine Harrison and Clare Lee, worked with maths and science teachers in Kent and Oxfordshire. We found that when teachers were supported in improving their use of formative assessment, their students did better than other students in the same schools, even when performance was measured using national curriculum tests and public examinations.
My partner in subsequent research, Siobhan Leahy, by then headteacher of Edmonton County School in Enfield, developed some materials for supporting monthly meetings of teachers – what we called teacher learning communities (TLCs). Using these materials, teachers could hold each other accountable for trying new ideas, and get support from like-minded colleagues.
She trialled the materials in the school and shared them with other schools in England, who provided feedback. After another year of development work with SSAT, the materials were published as a set of two DVDs, titled Embedding Formative Assessment (EFA). The pack included videos of teachers using various formative assessment techniques, interviews with students, leaders and teachers, and everything a school would need to run 18 monthly TLC meetings.
Since the publication of EFA in 2007, the materials have been used all over the world, in dozens of countries and on every continent (except Antarctica), and teachers have told us that they find the materials useful. However, while the feedback was positive, it was hard to know whether the use of the Embedding Formative Assessment pack had actually raised achievement. Even if student achievement was higher in schools using the materials, it could just be selection bias – the schools where things are going well might have more time to use the materials.
To get a clear indication of the impact of the EFA programme on student achievement, in 2015, SSAT applied to the Education Endowment Foundation (EEF) for a grant to evaluate the effectiveness of the EFA pack with a randomised controlled trial. The National Institute for Economic and Social Research (NIESR) was given the contract to conduct the trial.
A total of 140 secondary schools were recruited, and by the toss of a coin were allocated to either the experimental group (getting the EFA materials) or the control group (who just got the cash equivalent). After the randomisation, it was discovered that 16 of the schools recruited (12 in the control group and 4 in the experimental group) had previously collaborated with SSAT on the Teacher Effectiveness Enhancement Project (TEEP) – a programme developed by SSAT that had several elements in common with the EFA programme. The evaluators therefore decided that the GCSE achievement of students beginning year 10 in September 2015 in the 66 schools that had not been involved in TEEP but were given the EFA materials should be compared to that of students beginning Y10 in the 58 schools in the control group that had not been involved in TEEP. The measure of achievement they used was the government’s Attainment 8 measure in the summer of 2017. The main effect observed was to increase the achievement of the students in the schools given the EFA materials by 0.13 standard deviations.
The EEF estimates that this increase in achievement is equivalent to one extra month’s learning, but this estimate is misleading, because the EEF assumes that one year’s learning is the same for students of all ages. In fact, as students get older, students’ achievement becomes more spread out, because some students learn faster than others, and so the older they are, the more time they have to pull ahead. From the wide range of data sources following research into this factor, it seems not unreasonable to conclude that one year’s progress in Y10 and Y11 is equivalent to 0.3 standard deviations. So, we can make an estimate of the progress we would expect over the two years of key stage 4 for the students in the control group schools. Assuming that they would retain 90% of what they learn in Y10, and taking account of the fact that the GCSEs are taken half-way through the summer term, total progress over KS4 would be 90% of 0.3 (for Year 10) plus five-sixths of 0.3 (for Year 11) or roughly 0.52 standard deviations. The fact that the students in the schools given the EFA materials made 0.13 standard deviations more progress suggests that the rate of learning in the EFA schools was 25% greater, at a cost of around £1.20 per student per year.
Obviously more work needs to be done to refine these estimates. But the significant and substantial increase in the rate of learning, and the very modest cost, suggests that supporting teachers to develop their practice of classroom formative assessment school-based teacher learning communities is perhaps the single most cost-effective way of improving student achievement that we know of.
Until there is evidence that other approaches to improving student achievement might be as effective, classroom formative assessment really does need to be a priority for every teacher and every school.
This is an abbreviated version of a forthcoming article in the Winter 2018 edition of the SSAT Journal.
How SSAT can support you in embedding formative assessment across your school
SSAT have been working with Dylan Wiliam for over 15 years to introduce the Embedding Formative Assessment (EFA) programme into schools, so we know what works. To complement the recently-updated EFA resource, we also offer tailored training and consultancy support to help implement the programme across your school to maximise its impact and make it a sustainable feature.
For more information, visit the Embedding Formative Assessment section of the website.
