Internal evaluation

Ahakoa he iti kete, he iti nā te aroha.

It is the thought that counts.

Key ideas

Internal evaluation is an integral part of each early learning setting’s curriculum planning and implementation practices. Internal evaluation assists leaders and kaiako to evaluate the quality of their curriculum whāriki. Some settings refer to self review, quality improvement or teaching as inquiry as approaches they take to evaluating curriculum implementation. These approaches can be used to:

  • to contribute to ongoing improvement

  • to help with decision making

  • to increase understanding

  • to advance a principle such as equity

  • for accountability purposes.

The Education Review Office (ERO) defines internal evaluation as the use of robust processes to systematically inquire into and evaluate the effectiveness of policies, programmes, and practices.

Internal evaluation findings are used to inform decision-making, improve the quality of practice, and promote positive outcomes for all children. It is about finding out what is and is not working, and for whom, and then determining what changes are needed to improve quality. Effective internal evaluation is always driven by the motivation to improve.

Internal evaluation involves:

  • asking good questions

  • gathering fit-for-purpose data and information

  • making sense of that information

  • prioritising actions for improvement

  • monitoring and evaluating the impact of specific improvement actions.

Adult and child


Internal evaluations vary greatly in scope, depth, and focus depending on the purpose and the context. An evaluation may be strategic, linked to vision, values, goals, and targets, or it may be a business-as-usual review of, for example, policy or procedures. It could also be a response to an unforeseen (emergent) event or issue.

Guidance on conducting internal evaluations may be found in the ERO publication Effective School Evaluation – How to do and use internal evaluation for improvement. While this is designed for schooling, its content is relevant for the ECE sector (see Further resources: references).

Effective evaluation requires us to think deeply about the data and information we gather and what it means in terms of priorities for action. By asking the right questions of ourselves, we keep the focus on our learners, particularly those for whom current practice is not working. Aspiring for excellence and equitable outcomes is always front and centre.


Self review, quality improvement, and inquiry

Self review

Self review is the deliberate process of preparing, gathering information, making sense of information, and making decisions in order to bring about improvement. It offers opportunities for early childhood education services to evaluate the impact of practice on children’s learning.

Ngā Arohaehae Whai Hua was developed to support ECE settings to engage in self review for the following key purposes:

  • Improvement – self review enables us to improve our practice to ensure it supports children's learning in the best possible ways.

  • Accountability – self review enables us to ensure that we are meeting our legal requirements, responsibilities, and accountabilities. This is called compliance.

Reference:

Ministry of Education. (2006). Ngā Arohaehae Whai Hua/Self review Guidelines for Early Childhood Education. Wellington, New Zealand


Quality improvement

The purpose of quality improvement is to focus attention on and improve the “right things” for children. This process was outlined in The Quality Journey – He Haerenga Whai Hua (Ministry of Education, 1999). It is still a useful resource to support ECE settings to develop a quality improvement system, encouraging them to ask questions such as:

  • Are we doing the right things?

  • Are we getting the right results?

  • How can we do better?

  • What could we be doing differently?

It sets out an action research approach to quality review that involves a cycle of planning, doing, studying, and acting.

Reference:

Ministry of Education. (1999). The Quality Journey – He Haerenga Whai Hua. Wellington, New Zealand: Learning Media.


Teaching as inquiry

The teaching as inquiry cycle has been developed to support kaiako to achieve improved outcomes for all children. It is also an organising framework that kaiako can use to help them learn from their practice and build greater knowledge.

While originally designed to support kaiako in the compulsory sector to implement the New Zealand Curriculum, the process and questions are easily adapted to be relevant for the ECE sector. There are two forms of inquiry, focusing inquiry and teaching inquiry.

In the focusing inquiry, kaiako identify the outcomes they want their children to achieve (Te Whāriki and curriculum and learning priorities) and notice and recognise how children are doing in relation to those outcomes, identifying the next step of learning.

In the teaching inquiry, kaiako select strategies that will support children to achieve these outcomes. This involves asking questions about how well current strategies are working and whether others might be more successful. Kaiako research their own practice and identify (through collaborative discussion and reading) other strategies that may be more effective. They set up processes for capturing evidence about whether the strategies are working for their own students.

Both of these forms incorporate a learning inquiry which takes place both during and after teaching as kaiako observe children’s progress and reflect on what this tells them. Kaiako use this new information to decide what to do next to ensure continued improvement for children and in their own practice.

Although kaiako can work in this way independently, it is more effective when they support one another in their inquiries. We all have basic beliefs and assumptions that guide our thinking and behaviour but of which we may be unaware. We need other people to provide us with different perspectives and to share their ideas, knowledge, and experiences.

This process is driven by the following reflective questions:

  • What is important (and therefore worth spending time on), given where my children are at?

  • What strategies (evidence-based) are most likely to help my children learn this?

  • What has happened as a result of the teaching, and what are the implications for future teaching?

  • How can/will these learning outcomes be sustained?

For further guidance, go to: The New Zealand Curriculum Online: Teaching as inquiry

Stories and examples

Child with snail

Assessment documentation as self review data

Greerton Early Childhood Centre engages in self review through action research. Assessment data they collect every day is used to provoke conversations in the teaching team to take them “beyond the ordinary.” They develop and reflect on learning stories.

The teachers believe self review is understanding what enhances learning in your place. It shifts the focus from individual teachers, towards understanding how the culture of the setting contributes to children’s identities as lifelong learners.

Greerton ECC believes that a self review research question should follow something the teachers do well, exploring this deeply to grow understanding and articulate what makes it work. They provide the following guidance:

  • When deciding on which area to focus, first think about children’s interests. This is where passion will be driven from.

  • Find the thing that captures the team’s imagination and explore this deeply, widely, and enthusiastically. Self review happens best when teams set a research question that is meaningful to them.

  • Gather documentation to track progress.

  • Organise this documentation and additional reflective questions inside self-review folders. Reflective questions (for example, How did it all start? How was this interest, enquiry sustained? What surprised us? How have we changed? How are dispositions embedded into the fabric of the way we teach and learn?) will act like signposts, mapping intent and journey.

  • As questions are answered, more questions will be asked. These are revisited, re-worked, and reflected on, emerging as powerful practice, understood by everyone in your team and in your community.

Reference:

Sands, L. (2011). Self Review – Questioning our Practice through Thoughtful Evaluation. Educational Leadership Project (Ltd).


Child at desk

Review stories

When Ngā Arohaehae Whai Hua/Self review Guidelines for Early Childhood Education was created, a range of ECE settings told their stories of self review.

These stories are found at Ministry of Education: Self review guidelines for ECE. They and include stories from:

  • A rural Playcentre

  • A Rudolf Steiner kindergarten

  • A home-based service

  • A Tokelauan education and care service

  • A kindergarten


Adult and child

Centres of innovation – quality outcomes through action research

The Early Childhood Centres of Innovation (COI) programme ran from 2003–2009. Each ECE setting completed two rounds of action research around innovative teaching and learning in ECE settings.

Through this process, positive benefits were reported for kaiako, parents, whānau, and children. Kaiako engaged in critical thinking that challenged their assumptions about teaching and learning. As a result, their planning, formative assessment processes, and use of ICT within the learning and teaching programme were of a higher quality. On completion, kaiako were motivated to continue researching their professional practice.

Kaiako developed stronger learning–focused relationships with parents. Parents also felt more valued and had a greater desire to be involved in the centre programme.

Inspirational COI reports full of practical ideas for practice can be found at Education Counts: COI (Centres of Innovation).


Two children drawing

Effective self review

In 2009, ERO identified some common features of practice in the ECE settings where self review was well understood and implemented. Each of these features of practice and contributing factors is discussed, with specific examples in Implementing Self Review in Early Childhood Services (Education Review Office, 2009).

For these settings:

  • Self review was integral to the operation of the service and focused strongly on improvement.

  • Well-established procedures guided self review.

  • Reviews were both planned and spontaneous.

ERO also identified factors common to ECE settings where self review was well understood and implemented. These included:

  • strong leadership in promoting self review

  • professional development to support self review

  • stable and collaborative staff

  • sound systems for self review, and the use of relevant resources.

Reference: Education Review Office. (2009). Implementing Self Review in Early Childhood Services. Wellington, New Zealand.

Reflective questions

Kaiako can use these questions in team discussion to guide them through the process of internal evaluation and inquiry.

  • Whose knowledge is valued and reflected in our service’s curriculum?

  • How well does our curriculum respond to the strengths, interests, and needs of our infants, toddlers, and young children?

  • To what extent does our service whāriki support diverse learning needs so all children can be confident and competent learners?

  • How competent and capable are we at evaluating our teaching practices and the impact of these on outcomes for children? How do we know?

  • To what extent do our collaborative inquiry and evaluation processes focus on the impact of our bicultural curriculum whāriki for all children?

  • How do our collaborative inquiry and evaluation processes lead to improved practices?

    • How are they “enabling” and based on principles of advocacy, equity, and social justice?

The following questions have been taken from ERO’s report Working with Te Whāriki (2013).

  • To what extent is Te Whāriki referenced in our statement of philosophy?

    • Which aspects are included?

    • Why were these aspects included?

  • What do we know about how well our philosophy is enacted in practice?

  • How explicit is Te Whāriki in our service’s curriculum?

    • Which aspects are implicit and which are explicit?
  • To what extent is our service’s curriculum based on all of the aspects of Te Whāriki?

    • What aspects of Te Whāriki are included? For example, principles, strands, goals, and learning outcomes?

    • Does our service give greater emphasis to some aspects of Te Whāriki than others? Why?

    • What might this mean for children’s learning?

  • How is Te Whāriki visible in our service’s curriculum?

    • Which aspects are visible?

    • Are we just using the language of Te Whāriki, or do we have a deeper understanding of what the principles and strands mean for curriculum in our service?

  • What informs and guides our bicultural curriculum?

    • How do Te Whāriki, Ka Hikitia, and Tātaiako inform our bicultural practice?

    • What do we know about the impact of our bicultural curriculum on Māori children? On all children in our service?

  • What framework(s) do we use to evaluate the quality and effectiveness of our service’s curriculum?

    • What aspects of Te Whāriki might help us to undertake such evaluation?

Implications for leadership

Regulatory requirements for internal evaluation

Early childhood services are required to undertake review and evaluation as part of their licensing requirements (GMA6). The criteria to assess the governance, management, and administration standard specifies that an ongoing process of self review helps the service maintain and improve the quality of its education and care. The licensing criteria require early childhood services to document:

  • a process for reviewing and evaluating their operation

  • a schedule showing timelines for planned review of different areas of operation

  • recorded outcomes from the review process.

For more information, go to the regulatory framework.

ERO’s evaluation indicators

ERO’s evaluation indicators highlight the importance of leading internal evaluation (self review). The indicators of effective practice include the following:

  • Leaders are focused on improving the quality of education and care through ongoing systematic internal evaluation.

  • Internal evaluation is valued, championed, and effectively led.

  • Internal evaluation includes the gathering and analysis of useful information from a range of sources.

  • Leaders ensure teachers have time to critically reflect on their practice.

  • Leaders use evidence to reflect on and improve practice.

  • Good use is made of resources and research to support internal evaluation.

  • Leaders access professional learning and development to increase their individual and collective capability in internal evaluation.

  • Leaders ensure that all members of the service have opportunities to be involved in internal evaluation.

  • Evaluation is documented and leaders ensure that outcomes are shared with those involved in the service.

Connections to the principles

Empowerment – Whakamana

Inquiry and evaluation processes include children’s perspectives and views. Children’s agency and empowerment is promoted.


Family and Community – Whānau Tangata

Inquiry and evaluation processes take account of the views and perspectives of parents, whānau and community.


Holistic Development – Kotahitanga

Inquiry and evaluation processes are undertaken in a way that is mindful of all dimensions of children’s learning and development.


Relationships – Ngā Hononga

Relationships are integral to all Inquiry and evaluation processes. Relational trust supports effective inquiry and evaluation.

Further resources

Cardno, C. (2008). Action research in early childhood centres: Balancing research and professional development goals. New Zealand Research in Early Childhood Education (Vol. 11).

Dalli, C, E J White, J Rockel, I Duhn, with E Buchanan, S Davidson, S Ganly, L Kus, & B Wang. (2011). Quality early childhood education for under two year olds: What should it look like? A literature review. The Quality Debate: Past and present discourses. pp 25–50.

Cullen, J. (2008). Outcomes of Early Childhood Education: Do we know, can we tell, and does it matter? NZARE Annual Conference, Palmerston North, 2008.

Education Review Office. (2009). Implementing Self Review in Early Childhood Services. Wellington, New Zealand: Author.

Grey, A. E. (2006). Self-review: How far have we journeyed? Early Education 40, pp. 33-35.

Grey, A. E. (2010). Self-review as practical philosophy: A case study in early childhood education in Aotearoa New Zealand. Auckland: Auckland University of Technology.

Sands, L. (2011). Self Review – Questioning our Practice through Thoughtful Evaluation. Educational Leadership Project (Ltd).

Wansbrough, D. (2004). Management Matters. Self review – the hot topic. Early Education 35, pp. 23–30.

White, E. J. (2007). Systematic inquiry as evaluative activity: Self review in early childhood education. Early Education 42, pp. 26–29.

References

Education Review Office. (2013). He Pou Tātaki: How ERO reviews early childhood services. Wellington, New Zealand.

Education Review Office. (2015). Effective School Evaluation – How to do and use internal evaluation for improvement. Wellington, New Zealand.