Surviving ERO: how to get the most from your evaluationJune 2017
A visit from the Education Review Office (ERO) can sometimes send schools into a panic. So we asked ERO’s Acting Chief Review Officer NICHOLAS POLE to give some helpful advice and guidance on how ERO works with early learning services and schools. His key tip: don’t be scared – just be prepared.
What does ERO do?
Our role is to independently evaluate and report on the quality of education and care for learners in schools and early learning services. Since ERO was established in 1989, we have moved from being ‘inspectors’, when our job was auditing to ensure compliance – to an organisation that engages in a collaborative evaluation process, with the goal to ensure positive learning outcomes for all of New Zealand’s children and young people.
Each year, we evaluate about 800 schools and 1,300 early learning services. As well as evaluating in mainstream education settings, we also undertake home-school evaluations at the request of the Ministry of Education. Our Māori medium evaluation team, Te Uepū ā-Motu, evaluate kura, kōhanga reo and immersion early childhood services.
Why does ERO evaluate early learning services and schools?
The primary aim of ERO evaluations is to ensure that learners, their parents and whānau, can have confidence in our education system. We want to know that the components that comprise our education system are performing well and are adapting to a rapidly changing world. We are interested in how equitable education outcomes for all learners are being achieved.
We publish all of our evaluation reports online because we want parents, whānau, families and the wider community to know about the quality of teaching and learning at early learning services and schools. Making reports easily accessible invites parents to take an interest and ask questions, which in turn contributes to the educational success of children.
How do I prepare for an ERO visit?
To get the most out of your evaluation, it’s important that centre managers, leaders, staff and boards of trustees have discussions before we visit. These discussions will cover your strategic plans, internal evaluation processes and outcomes, and any issues specific to your context.
During the visit, your early learning service or school and our evaluation team will work together to ensure focused, considered discussions on matters that are important to your situation, and most importantly, to your learners. Your staff, supported by leaders and managers, need to be clear about the direction you are taking and your plans for the future.
While our evaluations are context specific and tailored to your needs, it is safe to assume we will focus on what has been achieved since the last time we evaluated your service or school and the culture of improvement you are building.
What’s available to help me prepare?
For schools, we use the School Evaluation Indicators, which focus on our increasing understanding of how schools can improve, and the vital role of evaluation in that process.
These Indicators guide schools with their internal evaluation. The indicators are key domains that work together to provide focus on accelerating student achievement. The indicators, and the associated resource, Effective School Evaluation, encourage schools to reflect and assess what they are doing well and to identify those areas for improvement.
Because effective internal evaluation is so critical, we have adapted this resource so it can be applied to early learning services as well. All of these resources are available on our website, where you will find a wide range of material to support your practice, and prepare for your evaluation.
We have specific methodologies and indicators for various types of early learning service. In early learning we are interested in the quality of children’s education and care and we evaluate through the lens of the early childhood curriculum, Te Whāriki.
He Pou Tātaki comprises of the framework and indicators we use for our evaluations of early learning services. This resource reflects the changes seen in the early childhood sector, as well as changes to our own practice – where internal and external evaluation meet. Positive learning outcomes for children are at the heart of He Pou Tātaki.
For immersion settings, we have worked closely with Te Kōhanga Reo National Trust, Te Runanga o Ngā Kura Kaupapa Māori o Aotearoa and Ngā Kura ā Iwi o Aotearoa to develop specialised evaluation processes and methodologies that reflect their unique philosophies.
What do I need to do?
We provide plenty of notice of an upcoming evaluation – usually between four and six weeks. At that point we request some information about your compliance with legal obligations, including child and student safety. Early learning services complete a report, while schools are asked to consider our reflection questions.
The reflection questions are intended to provide an overview of your school’s internal evaluation findings. There is no need to write down formal responses to these questions – but be prepared to discuss your reflection with us. We want to use your school’s data, and the findings of your internal evaluation to design an external evaluation that is a good fit for your school’s context. A discussion will take place on the first day of our visit to enable this.
Before we visit, one of our team leaders will contact you to go over the process, focus of the evaluation, and to answer any questions you have.
What’s the process?
During our visit to your early learning service or school, we talk to the board of trustees, senior leaders, teachers and support staff. We may speak to children, parents and whānau and even the wider community too.
It’s natural to be apprehensive at the beginning of an evaluation but it is all about mindset – your ERO evaluation provides an opportunity! It is the chance to have an external lens applied to your internal evaluation, and while it can be challenging, feedback tells us the process is useful and stimulating. Our evaluation conversations can also confirm good practice, and support you to establish priorities and create positive momentum for change.
When we visit your early learning service or school we will be interested in your account of what is happening, for whom, and why. We will initiate conversations that help us gauge what is working well, and where improvements can be made.
We’ll look at documents you provide for evidence that shows the practices in place are working. We want you to talk us through your internal evaluation processes. When done effectively, these processes enable trustees, leaders, teachers, parents, families, whānau and the wider community to better understand:
- how individual learners and groups of learners are performing in relation to valued outcomes
- how improvement actions have impacted on learner outcomes and what difference is being made
- what needs to be changed and what further action needs to be taken
- the patterns and trends in outcomes over time
- what kinds of practices are likely to make the most difference for diverse learners and in what contexts
- the extent to which the improvements achieved are good enough in terms of the school’s vision, values, strategic direction, goals and equity and excellence priorities.
In early learning settings, we look at the service’s curriculum provision, assessment processes, and how well staff are noticing, recognising and responding to children’s learning interests and needs.
Over the past year, our evaluators have been focusing on the achievement of Māori and other learners in need of acceleration by asking a specific evaluative question in primary, intermediate and area schools. We ask how effectively does this school respond to Māori and other children whose learning and achievement need acceleration? This leads to conversations about what achievement looks like in their school and what is being done to address any in-school disparity. We also focus on the organisational conditions that contribute to or inhibit student achievement for all learners.
What happens next?
Soon after our visit, we present our initial findings to you. This generates a discussion about the findings and what they mean. Within 20 working days of the evaluation ending, we will send you a draft evaluation report, which you can feedback on. We then finalise the report and publish it.
Our evaluation process is designed to be as open as possible. There should be no surprises. Our evaluations draw on a wide range of information, gleaned from data and the conversations we initiate. Evaluators want to gather all relevant information and discuss their findings with the school as they emerge during the evaluation.
More indepth information is available at www.ero.govt.nz or by contacting firstname.lastname@example.org