Dr JOHN BOEREBOOM discusses why defining an educational standard is so problematic in both the primary and secondary schooling sectors.
In 1878 the educational fathers of our fledgling nation introduced six educational standards for primary schools. The standards were highly prescriptive and precise. For instance, the prescription for Standard 2 Arithmetic prescribed: “Numeration and notation of not more than six figures; addition of not more than six lines, with six figures in a line; short multiplication and multiplication by factors not greater than 12; subtraction; division by numbers not exceeding 12, by the method of long and short division, mental problems and multiplication tables to 12 times 12.”
All primary school students were to be assessed at the end of each year by a school inspector, often on horseback. The assessment data was centrally controlled and students who did not pass the standard exams were held back. Parents, teachers and the public soon came to judge a school by the ‘percentage of passes’ and a teacher’s status and promotion prospects depended on the pass rates in the annual standards examination.
There was much debate in the 1920s about the impact of a standards-based primary school curriculum and the purpose of annual exams. When analysing the debate you constantly need to check the date because it could have been written yesterday. In 1936 the newly elected Labour Government abolished the proficiency exam in Standard 6 and schools received autonomy to design their own programmes of learning and assessment.
With a sense of déjà vu, standards-based assessment was revisited in secondary schools in the mid-1990s with the introduction of Unit Standards for secondary school qualifications and reintroduction of National Standards for primary schools in 2010. The National Standards describe expectations that students need to meet in reading, writing, and mathematics in the first eight years at school.
At first glance it seems straightforward for the Ministry of Education to prescribe the National Standards for primary education and for NZQA to provide Achievement Standards for secondary school qualifications.
In practice however, the definition of an educational standard is more problematic. Far from being a precisely and narrowly defined pedagogical statement, the standard is a complex social construct that reflects the interaction of a number of factors.
Firstly, there is the statement of the standard, which is by necessity precise and narrow. The standard specification has to be so specific and unambiguous that both learners and assessors can interpret them consistently.
Secondly, there is the level of difficulty and content of the assessment used to judge whether students have met the standard. This needs to be valid and requires interpretation of the standard by the examiner. This requires pre-assessment moderation of the assessment activity and assessment schedules to ensure they reflect the learning outcomes of the standard.
Thirdly, the marker of the assessment needs to interpret the marking schedule consistently with other markers throughout the country. This necessitates moderation between markers to reach a consistent interpretation.
Last but not least, it requires a shared understanding of all teachers and providers of educational resources so they can use the standard as a guide in the design of their teaching programmes.
All of these participants in the assessment process need to be socialised into developing a shared and triangulated understanding of the standard which extends far beyond the written statement of the standard.
In primary schools the introduction of National Standards has not gone smoothly. While some have called for the Government to get rid of them, this may be too drastic a recommendation. If assessment against National Standards remains school-based and student-centred to support teaching and learning they can be useful signposts for measuring student progress as part of a learning progression framework. The scenario to avoid is the potential path to national testing, league tables, measures of school effectiveness and, heaven forbid, performance pay.
In the secondary sector, the introduction of Unit Standards in the nineties resulted in widespread public and academic debate. The subsequent evolution of secondary school assessment provides an interesting illustration of the difficulty of adopting a pure form of standards-based assessment.
Unit Standards for secondary school qualifications were internally assessed and competency based. Each standard included strict performance criteria. To achieve the standards students had to meet all of the performance criteria. Critics felt that while this method of assessment was suitable for vocational and skill-based subjects, it was unsuitable for assessing the higher level skills of academic subjects. The method of grading adversely affected student motivation and did not sufficiently differentiate between students. Teachers also complained of the heavy administrative workload involved in keeping track of numerous standards and criteria.
The move to Achievement Standards introduced external exams and an element of ranking by stating whether students had met the standard with an Achieve, Merit or Excellence grade. Due to the complexity of locating the standard, the initial results showed variation from year to year in the percentage of students who attained the various grades. The standards were elusive to assess consistently.
NZQA addressed this by introducing the Profile of Expected Performance (PEP). The PEP is based on the historical proportions of students achieving N, A, M and E grades for each standard. At the initial stages of marking the profile of grades for a sample is compared to the PEP. If there are variations adjustments are made to the assessment schedule to achieve a better match. This is a form of standards based scaling. It does not accommodate national improvements and longitudinal growth as schools and students improve in their preparation for the NCEA exams.
The introduction of cut scores to regulate the percentage of students in the various grade categories were a further move away from a pure form of standards-based assessment towards a hybrid form of assessment that uses a norm-referenced approach to adjust the results. The setting of cut scores to achieve a PEP is dictated by the distribution of grades and not linked back to the interpretation of the standard.
The manipulation of grades using PEPs and cut scores happen behind a veil of confidentiality agreements at the marking stage and lack transparency for students, teachers and parents who expect that a student’s performance in relation to the criteria is the sole determinant of their grade.
History clearly shows us that summative assessment of learning that is used for certification and national qualifications needs to include some form of norm referencing since student ability and achievement, like all human traits, is normally distributed. This provides the basis for norm-referenced assessment which enables recognition of a wide range of achievement. In the case of assessment for learning however, standards-based assessment is student centred, formative, diagnostic and useful for measuring progress and guide future learning and teaching.
We have a world class Curriculum Framework that is student-centred, engages students and provides freedom for teachers to design learning programmes to meet the needs of their students. The challenge is to choose the right assessment paradigm for the correct end use. The lessons from history are valuable and need to be revisited periodically to ensure that the student remains at the heart of our education system.
Value-added assessment is useful because it supplements the summative nature of standards by introducing a monitoring component which provides a clear indication of student progress and growth.
Dr John Boereboom is the Director of the Centre for Evaluation and Monitoring at the University of Canterbury. He is a past national moderator, examiner and standard writer.