VALUE ADDED MEASUREMENT OF PUPIL PERFORMANCE-UNRESOLVED CONCERNS

VALUE ADDED MEASUREMENT OF SCHOOLS AND TEACHER PERFORMANCE

Not so clear cut, as consensus is hard to find

Comment

The Government is keen on value added measurements for pupils and for this to be  included in performance tables, as is the Specialist Schools and Academies Trust, which prefers Contextual Value Added measurement, designed by Professor Jephson, of York University, to the five good GCSEs in maths and English, used as the key Government performance measure and for initiatives such as the National Challenge.

The attractions of CVA, as opposed to raw test scores are obvious as it is widely recognized among education researchers and practitioners that school rankings based on unadjusted test scores are highly correlated with students’ socioeconomic status. So CVA is thought to give a more complete and fairer picture of a pupils’ achievements. It can also be used as a basis for rewarding teacher and pupil performance and to determine teachers pay as well as for teachers self-improvement or pupil target setting.

Value-added methods refer to efforts to estimate the relative contributions of specific teachers, schools, or programmes to pupil test performance.

In recent years, these methods have attracted considerable attention because of their potential applicability for accountability, teacher pay-for-performance systems, schools and teacher improvement, programme evaluation, and research. But the challenge is that value-added methods involve complex statistical models applied to test data of varying quality and so it is not free from controversy.  Indeed, academics will agree that there are unresolved concerns about the precision and bias of value added results and many academics argue against employing value-added indicators as the sole basis for high-stakes decision.  Interest in value added measurement took off after the well known Sanders and Rivers report was published in 1996. They found that teacher effects, estimated using student test score trajectories, predict student outcomes at least two years into the future. This finding suggested that teachers have persistent effects on their students’ achievement and that the accumulation of these effects could be substantial. The following year, Sanders and his colleagues published another paper claiming that teachers are the most important source of variation in student achievement (Wright, Horn, and Sanders, 1997).

A report Getting Value Out of Value-Added was recently released in the US by National Research Council and National Academy of Education. This documents discussions of a workshop jointly held by the two organizations in 2008 to help policy makers understand the current strengths and limitations of value-added approaches and whether to implement them in their jurisdictions. The workshop explored the advantages and disadvantages of value-added methods and responsible and defensible uses in education settings. While the report does not issue recommendations, it summarizes current research findings and the judgments expressed by workshop participants.

Many workshop participants “observed that value-added approaches have the potential to provide useful information for educational decision-making. At the same time, they noted that there are many technical and practical issues that need to be resolved in order for researchers to feel confident in supporting certain policy uses of value-added results.”

To date, there is little relevant research in education on the incentives created by value-added evaluation systems and the effects on school culture, teacher practice, and student outcomes.

Significantly, there is a range of different types of value added measurement. And the US workshop discussions found, to nobody’s surprise, there is not one dominant Value Added Measurement. No value-added approach (or any test-based indicator, for that matter) addresses all the challenges to identifying effective or ineffective schools or teachers. Each major class of models has shortcomings, and there is no consensus on the best approaches, and little work has been done on synthesizing the best aspects of each approach. There are questions too about the accuracy and stability of value added estimates of schools, teachers, or programme effects. The overall conclusion was that more needs to be learned about how these properties differ, using different value-added techniques and under different conditions.  This is probably worth remembering when the SSAT next publishes its value added measurements for its schools.

http://www.nap.edu/catalog.php?record_id=12820