CAUTIONARY NOTE ON TEACHER EVALUATIONS AND PERFORMANCE RELATED PAY
International evidence mixed and there are practical challenges
The School Teachers’ Review Body (STRB), which makes recommendations to Government on teachers’ pay reforms, is calling for greater freedom for schools to set teachers’ pay.In its report published last week it recommends a simpler, more flexible national pay framework for teachers. Taking on board international evidence and views from Government, employers, and teacher and head teacher unions, it proposes:
ending pay increases based on length of service – currently virtually all full time classroom teachers on the main pay scale automatically progress to the next pay point;
linking all teachers’ pay progression to performance, based on annual appraisals – already the case for some teachers who are on a higher pay scale;
abolishing mandatory pay points within the pay scales for classroom teachers to give schools greater freedom on how much teachers are paid. They would remain in place for reference only in the main pay scale to guide career expectations for new teachers entering the profession; and
retaining the higher pay bands for London and fringe areas.
Linking all teachers pay progression to performance is, of course, easier said than done. Evaluating Teacher performance is central to education reforms in the United States. There, they are way ahead of us in seeking to evaluate, fairly, teachers performance. At least 30 states have launched new systems to evaluate teachers using more rigorous criteria about what makes a good teacher, and you would have thought that there was by now considerable consensus on what the criteria should be. But there isn’t. Teacher evaluations have become highly controversial as states introduce a variety of models. There is little consensus on how you reliably measure added-value. Can the quality of a teacher be measured by looking at just a few key skills, such as setting academic goals and running an effective class discussion? Should teacher performance be based simply on test results? (there have been a number of scandals in the States over manipulating test scores) And if so what about areas of a childs education which are not ,or cant be, tested, related, for example, to non-cognitive skills? Or should teachers be evaluated based on a broader range of abilities, including lesson-planning and content knowledge? If it was accepted that one model of measuring value added did so with a considerable degree of accuracy over time and was absolutely fair and not subject to random results then the task would be pretty straightforward. But that is not how things stand. There can be random differences across classrooms in unmeasured factors related to test scores, such as pupils abilities, background factors, and other pupil -level influences and, secondly, what has been described as ‘ idiosyncratic’ unmeasured factors that affect all students in specific classrooms, such as a barking dog on the test day, or a particularly disruptive student in the class on the day. Existing research has consistently found that teacher- and school-level averages of student test score gains can be unstable over time. Studies in the States have found only moderate year-to-year correlations—ranging from 0.2 to 0.6—in the value-added estimates of individual teachers (McCaffrey et al. 2009; Goldhaber and Hansen 2008) or small to medium-sized school grade-level teams (Kane and Staiger 2002b). As a result, there are significant annual changes in teacher rankings based on value-added estimates.
A report for the US Department of Education ‘ Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains’ (July 2011) found that there is ‘evidence that value-added estimates for teacher-level analyses are subject to a considerable degree of random error when based on the amount of data that are typically used in practice for estimation.’ It also said, and this is crucial, that evidence suggests ‘that more than 90 percent of the variation in student gain scores is due to the variation in student-level factors that are not under control of the teacher’
It may be possible under existing models for measurement to differentiate the performance at the top and bottom of the distribution but is it precise or accurate enough to differentiate clearly between the bulk of teachers in the middle of the distribution? There must be some doubt about this even if you factor in ‘observation’ of teachers work. (teachers performance evaluation doesn’t rely entirely on value added measurement) It is worth repeating what the NFER in the UK said in a paper in 1999 when debate on added value was really beginning in earnest here- ‘What value added data cannot do is prove anything. Value added evidence is only part of the story of school effectiveness. The notion of a value added measure which tells you – and everyone else – how well your school or department or class is doing, and is also simple to calculate, understand and use, is a non-starter’.
The Education Select Committee in a report on Teachers this year recommended ‘that the Department develop proposals (based on consultation and a close study of systems abroad) for a pay system which rewards those teachers who add the greatest value to pupil performance. We acknowledge the potential political and practical difficulties in introducing such a system, but the comparative impact of an outstanding teacher is so great that we believe such difficulties must be overcome.’
The Committee was right to flag up the potential practical difficulties and international evidence is not ,as we can see ,exactly clear.
Under the current system for teachers’ pay according to the government:
automatic pay progression means there is a poor link between a teacher’s performance and reward; and schools in some parts of the country struggle to recruit and retain good teachers.
As things stand, the current career structure has three specific pay grades, linked to higher level standards that are designed to keep the best teachers in the classroom. These are the Post Threshold, Advanced Skills Teacher (AST) and Excellent Teacher (ET) grades. The Government believes that Schools should be free to decide how to reward classroom teachers according to the contribution they make. Some schools and local authorities have found the existing AST model does not always fit their requirements. They have innovated and worked round this model or created new roles altogether such as “consultant teacher” supported by a training programme of their own design.
The recommendations provide for ‘ the creation of posts that will pay salaries above the upper pay scale and that will enable some of the very best teachers to remain in the classroom to demonstrate excellent teaching and lead the improvement of teaching skills. We propose to accept this recommendation. It will be for individual schools to decide whether or not to create such posts and whether or not to move existing ASTs and ETs into those posts.’
The new proposals mean, as far as teachers are concerned , that ‘ Pay progression will be decided through your annual appraisal. At the beginning of each academic year, you will discuss with your line manager/head teacher how your performance will be monitored and assessed, and how this may link to any changes in your pay. Your school will need to work out how it wishes to make these changes. You can expect your school’s pay policy to set out how they will work in practice.’
Schools will need to develop systematic and transparent arrangements for both appraisal and pay.
The STRB has recommended that the Department develops guidance or a toolkit to help schools develop systematic and transparent local approaches to pay progression. Subject to consultees’ views,the Government propose to accept this recommendation.