League Tables: Learning from experience

September 2012

 

Facebook       Tweet

If there was a league table for international education systems, New Zealand would likely rank near the top. Will the introduction of primary school league tables see our ranking plummet?

JUDE BARBACK considers the differing international viewpoints, research and experience.

The Ministry of Education’s proposal to release data on primary schools’ performance levels has been met with mixed reactions. The main cause for controversy is the potential for media to compile and disseminate public league tables based on the information given, under the Official Information Act.

A Herald DigiPoll revealed that almost 59 per cent of respondents approve of publishing information on schools’ performance, either by the Ministry, or the media, or both. While the survey didn’t explore the rationale behind the participants’ stances, a common reason people are in support of league tables is the need for democratic openness. Parents, in particular, are likely to value the ability to compare the performance between schools in their area.

League tables can also help provide targets for change. If a school is lagging behind, and everyone knows about it thanks to the published league tables, the lagging school is likely to try harder to raise its game, so the argument goes.

However, there is also strong opposition to the idea. While only 36 per cent of DigiPoll respondents believe comparisons between schools are unfair, it is a very audible minority.

Among them is Paul Drummond from the

New Zealand Principals’ Federation, who says league tables would provide an incomplete picture for parents on which to base their choice.

“Parents have always compared schools so that they can choose the most suitable match for their children, which usually ends up being their neighbourhood school. They access ERO reports, talk to principals, and teachers, and other parents, and visit schools before enrolling their children. League tables will not help them because they will be based solely on immature national standards data in two subjects, which cannot provide a picture of a whole school,” says Drummond.

A large cluster of education academics from universities across New Zealand share Drummond’s stance. Over 100 names were attached to an open letter, which called on the Government to halt plans to compile league tables of schools. The letter outlined how league tables have the potential to cause harm to learners, teachers, schools, and local communities. “Data release in league table form will ... misinform rather than inform parent and community judgments about how well children are learning,” the letter read. It also affirms that national standards are unsuitable for comparing schools because they do not take account of a school’s whole context.

NZEI president, Ian Leckie, agrees. He has written to schools advising them not to release information to the news media under the Official Information Act.

Leckie suggests that any country that used league tables had gone backwards.

International experience

Certainly there is some staunch opposition to league tables in England, when used as a basis for parents’ decisions on where to send their children.

Rankings of schools’ exam results were first published in England in the early 1990s, followed by the publication of key stage test-score ranking at the end of primary school. The rankings appear in the national and local media and are used by schools in their promotional material.

Initially, the simple school averages formed the basis of rankings and unsurprisingly, schools with the brightest and advantaged students tended to emerge at the top of the rankings. In 1995, the government accepted the weaknesses of the system and moved to a ‘value-added’ system, which takes account of the different levels of achievement of students entering their schools. More recently, these have been tweaked again to form a ‘contextual value-added’ system, which takes into account school-level factors such as the previous achievement of students, eligibility for free school meals, lack of spoken English at home – all in an attempt to create a fair ranking system.

However, expert on the subject, Harvey Goldstein, of University of Bristol, argues that the contextual value-added ranking system – or any of the previous systems – is not appropriate for parents making decisions on school choice. Goldstein says if a school-level factor is associated with achievement, this is strictly part of the effect being measured and therefore not something to be adjusted for, making the system inappropriate for choice purposes.

Goldstein also argues that league tables result in parents choosing a school based on future predicted results. For example, parents of 11-year-old students selecting schools on the basis of the published exam results of 16-year-olds in the same year, are basing their choice on results predicted five years into the future.

An article by Goldstein and colleague George Leckie, School league tables: What can they really tell us? states categorically that ‘parents relying on league tables to select a school for their children are using a tool not fit for purpose’.

Their article also touches on the perverse side effects of league tables. Goldstein and Leckie discuss the incentive for schools to concentrate excessively on borderline students at the expense of those likely to achieve high results, and the incentive to discourage students from taking difficult subjects such as foreign languages and sciences, because they fear depressing the proportion achieving passes. In another article published this year, Measuring Success, Goldstein and co-author Beth Foley point to evidence that schools engage in ‘gaming’ to improve their ranking, by manipulating exam entry policy to the detriment of student choice, or even by excluding low achievers.

These concerns echo those raised by teachers in New Zealand over the proposal to move to performance-pay incentives.

In fact, the prevalence of league tables has grown out of the performance-management movement in the private sector. Advancing technology and availability of large administrative databases have also played a part. League tables are now widely used in the public sector, in health, social services, policing, and now education sectors.

Goldstein and Foley give a number of caveats and recommendations for the use of league tables in education, based on international research and pilot studies. Top among them is the suggestion that league tables should not be closely linked with rewards, to discourage ‘gaming’ tactics as described above. This would also serve to make the tables a more objective measure of performance.

Interestingly, the authors suggest the government should consider ways to prevent league tables being exploited by the media. Further, the limitations and strengths, including the degree of statistical uncertainty, should be made clear to the public.

Ultimately, Goldstein and Foley argue that consideration should be given to alternative ways for measuring and comparing educational performance. They suggest performance information should be used for screening device purposes, that are not published or made available beyond those schools involved, but used as part of an improvement programme, so that schools can seek improvement without perverse incentives that may arise from ‘exposing’ them using public rankings.

Russell Wildeman is also sceptical about league tables from his experience with the South African system. He says school league tables ‘could only result in a race to the bottom in an environment that is already too competitive and deeply unequal’.

However, British research, also from the University of Bristol, shows that the performance levels of Welsh schools has fallen in the ten years since league tables have been abolished, particularly for schools in the poorer areas. A study earlier this year showed that the same decrease was not apparent in English schools where the league tables remain, suggesting that the competition provided by the league tables could be a necessary driver for performance.

The Australian government introduced public reporting of national literacy and numeracy results last year and like here, there are both proponents and opponents to the decision. Upon the launch of MySchool, the website used to report Australian schools’ results, teachers threatened to boycott the tests.

Federal president of the Australian Education Union, Angelo Gavrielatos, is strongly opposed. “There’s ample evidence that high-stakes testing and the comparison and ranking of schools leads to deepening inequity and segregation of schooling.”

A scathing piece in Australia’s The Punch, published several years ago when the decision was being made in Australia whether or not to introduce school league tables, suggested that the Australian government should not be looking to countries like the UK and the USA for educational direction, but rather to New Zealand.

“Countries that do not use league tables include Finland and New Zealand – two countries that consistently top international benchmarks for student performance. So why would we want to follow the lead of education systems that Australia clearly outperforms, and ignore the lessons from those education systems that do it better than ours?” reads the article.

If that was the feeling several years ago, it will be interesting to see what light is cast on New Zealand’s education system, both here and internationally, once school league tables are in full swing. Of more significance, will the naming and shaming tactic help drive performance as suggested in some studies? Like so many educational initiatives, only time will tell.