Fight Fallacious Targets: How Data Driven Target Setting Harms Achievement

The new year means new classes, new students and for many teachers new targets. Targets that are not based on the judgement of a professional who knows the children in question but are derived solely from data held about them. This post is about how those targets are often set and why determining student targets from data is always a bad idea.

The Fischer Family Trust (FFT) provide the data that many secondary schools use to set targets. The data they give is the GCSE performance of pupils nationally who had the same KS2 attainment, gender and month of birth as each of your students. The pupils in this sample can be drawn from the top 50% (average) the top 20% (high) or the top 5% (very high) of schools for value added. 

null

Imagine a child, Adam, whose target we are going to set. The chart above is a sample distribution of what happened to other boys who had the same prior attainment and month of birth as Adam. If we look at the same from the top 20% of schools, the single most likely grade is D and that, in many schools, would be his target. That’s crazy because while 32% of pupils like him got Ds, 43% got C or better. Many schools will say ‘Ah, but we use FFT + 1 so our targets are appropriate and challenging’. Are they right? No.

Before we consider why not, let’s remind ourselves that the reason we have pupil targets is because we believe some pupils are capable of more than others. This can be obscured by well-meaning statements like ‘we have high expectations for all our pupils’ or ‘we demand excellence from everyone.’ These imply everyone being held to the same high standard when in reality ‘excellence/high’ is defined differently for different pupils and their target is supposed to tell us what that definition is.

Now let’s imagine Adam’s target and those of his peers is set to the FFT mostly likely grade + 1, so for Adam it’s a C. Assuming performance in line with the top 20% for value added that means 60% (32+18+7+3) of pupils are below target. 60% is a big number, the logic of tracking says this is a big problem, SLT will likely demand action, requiring teachers to spend time and effort on this ‘underachievement’. Now consider the 12% of students who got a B or higher. If they are currently achieving a C your tracking system says they are doing well, they’re on target.

If your school uses any form of targets derived from prior attainment some children reaching their potential are deemed to be below it, while others not reaching their potential are deemed to be doing well. Not only do these targets not help, they actively hinder the effort to see every child secure they highest grade of which they are capable.

The idea that FFT is ‘more accurate’ than a teacher’s judgment is based on a flawed understanding of statistics. The validity of statistical predictions depends on the size of the sample about which the predictions are made. Therefore FFT can make accurate predictions about cohorts (n = 150 -250) but when it comes to individuals (ie n = 1) they can’t predict with any accuracy at all.

It’s time to stop using FFT or any algorithm to set targets. Data doesn't know the individuals in your class better than you.

blog comments powered by Disqus