Are league tables a fair way to compare school effectiveness?February 2017
DR JOHN BOEREBOOM suggests that a school’s effectiveness should be judged on the basis of how much the students learned from the time they entered the school to the time they left rather than simply relying on a traditional ‘snapshot’ measure in the NCEA exams.
Students entering New Zealand secondary schools in year 9 come from a variety of socio-economic, cultural and ability backgrounds. They have been taught in different schools, by teachers of varying effectiveness, in different classrooms using different programmes, and have progressed at different rates. This is reflected in the results of school entrance testing, which shows the wide diversity of each school’s intake in terms of National Standards and curriculum level achievement.
Despite this diversity at the start of secondary schooling, our education system measures the achievement of students in years 11–13 using NCEA as a common yardstick. The annual league tables published by the media rank schools on the basis of their students’ performances in NCEA.
Parents and the public frequently make judgements about school effectiveness based on the ranking of the school.
Parents often vote with their feet by enrolling their children in schools on the basis of ‘league table’ rankings. In some cases this has meant moving house to be in the zone of the preferred school. The school leagues even have an impact on property values and school zones make their appearance in real estate advertisements.
Schools face the temptation of concentrating their efforts on those students considered capable of improving their NCEA scores, while giving less attention to those perceived less likely to improve.
An inevitable result of league tables is that there are winners and losers. If our efforts to meet increasing demands for assessment, accountability, standards monitoring, quality assurance, school effectiveness causes us to lose sight of ensuring that what we offer in school education is accessible to all students it would be counterproductive.
This leads to the question of whether league tables are a fair and valid way to compare schools and judge the effectiveness of individual schools and whether league tables contribute to a desirable outcome for our education system.
The principal argument against league tables is that the performance of a school is determined largely by the pre-existing achievements of the students when they enter it. School intakes differ markedly in this respect and some schools have highly selective entry criteria. Horse-race comparisons of schools are at best misleading and may have detrimental effects on teaching and learning. It is therefore invalid to judge the quality of the education within a school solely in terms of league tables.
At the grassroots level, principals and teachers of schools that rank poorly in the league tables often comment that their students have made tremendous progress during their time at school. Students in such schools may achieve below average NCEA results but they may have progressed more since entering secondary school than the students to whom they are compared.
Which is the best way to measure school performance, the percentage of students getting A, M or E grades in NCEA or the growth and improvement shown by the students during their time at the school?
Should a school’s effectiveness be judged on the basis of how much the students learned from the time they entered the school to the time they left rather than simply relying on a traditional ‘snapshot’ measure in the NCEA exams?
These questions highlight that student performance on assessments can be measured in two very different ways. Achievement describes the summative attainment of students in tests and Achievement Standards. Value-added assessment, in contrast, describes the progress made by students over the school year. In the past, students and schools have traditionally been ranked according to achievement.
Value-added assessment is a way of analysing test data that can measure growth and progress. Starting with a baseline assessment in year 9, we can statistically predict the progress students are likely to make by the time they sit the NCEA exams in year 11. The value-added score measures whether the NCEA performance of a student, subject or school has kept pace, lagged behind or was better than expected when compared with students with similar scores in the baseline test nationally. This lens of measuring student learning provides schools with valuable information to ensure they are meeting the academic needs of groups of students, as well as individual students. See table above.
The Centre for Evaluation and Monitoring at the University of Canterbury provides value-added analysis of NCEA results for New Zealand schools using the MidYIS9 assessment as a baseline in year 9.
The value-added results and comprehensive online feedback show the relative value-added by different subjects in the school and the effectiveness of instruction for different ethnic and ability groups. The results can be used to identify giftedness and special learning needs, highlight effective practice, guide professional development and judge the effectiveness of educational interventions and changes to the school curriculum.
The differences between NCEA results and value-added assessment are:
- measures a student’s achievement at a single point in time
- is highly correlated with a student’s demographics and school decile rating
- compares student performance to a standard
- is used for certification and entry to further study.
- measures a student’s progress across years
- compares student performance with his or her own prior performance
- is critical to ensuring a student’s future academic success.
Both types of assessment are necessary and serve a different purpose. The summative NCEA results provide a snapshot in time used for the certification of students. The value-added results provide rich data and detailed diagnostic information that can be used by schools to inform teaching and learning.
By measuring students’ academic achievement AND growth, schools have a more comprehensive picture of their own effectiveness in raising student achievement. The power of school-based data analysis can be further enhanced by correlating achievement and growth data with the results of student and teacher attitude and engagement surveys.
Whatever approach is used, we need to remember that educational institutions have a responsibility for encouraging children’s learning and development across a much wider range of areas than can reasonably be tested.
You might also like to read:
- Education sector leaders debate latest NZ Initiative research
- Careers NZ now part of Tertiary Education Commission
- Part-time secondary teachers seek equal pay
- Report compares NZ and European qualifications frameworks
- Manurewa Intermediate shines at PM's Education Excellence Awards
- Digital fluency investment marks biggest change to curriculum in a decade