Ahakoa he iti kete, he iti nā te aroha.
It is the thought that counts.
Internal evaluation is an integral part of each early learning setting’s curriculum planning and implementation practices. Internal evaluation assists leaders and kaiako to evaluate the quality of their curriculum whāriki. Some settings refer to self review, quality improvement or teaching as inquiry as approaches they take to evaluating curriculum implementation. These approaches can be used:
to contribute to ongoing improvement
to help with decision making
to increase understanding
to advance a principle such as equity
for accountability purposes.
The Education Review Office (ERO) defines internal evaluation as the use of robust processes to systematically inquire into and evaluate the effectiveness of policies, programmes, and practices.
Internal evaluation findings are used to inform decision-making, improve the quality of practice, and promote positive outcomes for all children. It is about finding out what is and is not working, and for whom, and then determining what changes are needed to improve quality. Effective internal evaluation is always driven by the motivation to improve.
Internal evaluation involves:
asking good questions
gathering fit-for-purpose data and information
making sense of that information
prioritising actions for improvement
monitoring and evaluating the impact of specific improvement actions.
Internal evaluations vary greatly in scope, depth, and focus depending on the purpose and the context. An evaluation may be strategic, linked to vision, values, goals, and targets, or it may be a business-as-usual review of, for example, policy or procedures. It could also be a response to an unforeseen (emergent) event or issue.
Guidance on conducting internal evaluations may be found in the ERO publication Effective Internal Evaluation for Improvement (see further resources: references).
Effective evaluation requires us to think deeply about the data and information we gather and what it means in terms of priorities for action. By asking the right questions of ourselves, we keep the focus on our learners, particularly those for whom current practice is not working. Aspiring for excellence and equitable outcomes is always front and centre.
Self review
Self review is the deliberate process of preparing, gathering information, making sense of information, and making decisions in order to bring about improvement. It offers opportunities for early childhood education services to evaluate the impact of practice on children’s learning.
Ngā Arohaehae Whai Hua was developed to support ECE settings to engage in self review for the following key purposes:
Improvement – self review enables us to improve our practice to ensure it supports children's learning in the best possible ways.
Accountability – self review enables us to ensure that we are meeting our legal requirements, responsibilities, and accountabilities. This is called compliance.
Reference:
Quality improvement
The purpose of quality improvement is to focus attention on and improve the “right things” for children. This process was outlined in The Quality Journey – He Haerenga Whai Hua (Ministry of Education, 1999). It is still a useful resource to support ECE settings to develop a quality improvement system, encouraging them to ask questions such as:
Are we doing the right things?
Are we getting the right results?
How can we do better?
What could we be doing differently?
It sets out an action research approach to quality review that involves a cycle of planning, doing, studying, and acting.
Reference:
Teaching as inquiry
The teaching as inquiry cycle has been developed to support kaiako to achieve improved outcomes for all children. It is also an organising framework that kaiako can use to help them learn from their practice and build greater knowledge.
While originally designed to support kaiako in the compulsory sector to implement the New Zealand Curriculum, the process and questions are easily adapted to be relevant for the ECE sector. There are two forms of inquiry, focusing inquiry and teaching inquiry.
In the focusing inquiry, kaiako identify the outcomes they want their children to achieve (Te Whāriki and curriculum and learning priorities) and notice and recognise how children are doing in relation to those outcomes, identifying the next step of learning.
In the teaching inquiry, kaiako select strategies that will support children to achieve these outcomes. This involves asking questions about how well current strategies are working and whether others might be more successful. Kaiako research their own practice and identify (through collaborative discussion and reading) other strategies that may be more effective. They set up processes for capturing evidence about whether the strategies are working for their own students.
Both of these forms incorporate a learning inquiry which takes place both during and after teaching as kaiako observe children’s progress and reflect on what this tells them. Kaiako use this new information to decide what to do next to ensure continued improvement for children and in their own practice.
Although kaiako can work in this way independently, it is more effective when they support one another in their inquiries. We all have basic beliefs and assumptions that guide our thinking and behaviour but of which we may be unaware. We need other people to provide us with different perspectives and to share their ideas, knowledge, and experiences.
This process is driven by the following reflective questions:
What is important (and therefore worth spending time on), given where my children are at?
What strategies (evidence-based) are most likely to help my children learn this?
What has happened as a result of the teaching, and what are the implications for future teaching?
How can/will these learning outcomes be sustained?
For further guidance, go to: The New Zealand Curriculum Online: Teaching as inquiry
Using internal evaluation to put the assessment back into learning stories
Sue, the professional leader of Campus Creche in Hamilton, observed that while the centre’s learning stories were engaging, they were not doing the whole job of assessment. The stories were very general, which made pinpointing the learning, and how to progress this for individual children, much harder to identify. The stories had become an “add on” rather than a means of assessment for curriculum planning.
Sharing her thoughts with kaiako led to the entire team considering the:
As a result of this investigation, the team identified that some changes to practice needed to occur. The centre has now invested in a part-time assessment mentor who works with individual kaiako during their non-contact time. The mentor supports kaiako to:
To help them accelerate change, Sue sought advice on the process of internal evaluation from an outsider. This highlighted that their process was missing strong, clear, and agreed indicators of what they wanted to achieve. Once they had these and knew what they were aiming for, kaiako motivation to engage in the change process increased noticeably.
Positive outcomes are evident, and Sue believes that the addition of a mentor helped kaiako foreground assessment in their learning stories in the following ways:
Assessment documentation as self review data
Greerton Early Childhood Centre engages in self review through action research. Assessment data they collect every day is used to provoke conversations in the teaching team to take them “beyond the ordinary.” They develop and reflect on learning stories.
The teachers believe self review is understanding what enhances learning in your place. It shifts the focus from individual teachers, towards understanding how the culture of the setting contributes to children’s identities as lifelong learners.
Greerton ECC believes that a self review research question should follow something the teachers do well, exploring this deeply to grow understanding and articulate what makes it work. They provide the following guidance:
When deciding on which area to focus, first think about children’s interests. This is where passion will be driven from.
Find the thing that captures the team’s imagination and explore this deeply, widely, and enthusiastically. Self review happens best when teams set a research question that is meaningful to them.
Gather documentation to track progress.
Organise this documentation and additional reflective questions inside self-review folders. Reflective questions (for example, How did it all start? How was this interest, enquiry sustained? What surprised us? How have we changed? How are dispositions embedded into the fabric of the way we teach and learn?) will act like signposts, mapping intent and journey.
As questions are answered, more questions will be asked. These are revisited, re-worked, and reflected on, emerging as powerful practice, understood by everyone in your team and in your community.
Reference:
Review stories
When Ngā Arohaehae Whai Hua/Self review Guidelines for Early Childhood Education was created, a range of ECE settings told their stories of self review.
These stories are found at Ministry of Education: Self review guidelines for ECE. They and include stories from:
A rural Playcentre
A Rudolf Steiner kindergarten
A home-based service
A Tokelauan education and care service
A kindergarten
Centres of innovation – quality outcomes through action research
The Early Childhood Centres of Innovation (COI) programme ran from 2003–2009. Each ECE setting completed two rounds of action research around innovative teaching and learning in ECE settings.
Through this process, positive benefits were reported for kaiako, parents, whānau, and children. Kaiako engaged in critical thinking that challenged their assumptions about teaching and learning. As a result, their planning, formative assessment processes, and use of ICT within the learning and teaching programme were of a higher quality. On completion, kaiako were motivated to continue researching their professional practice.
Kaiako developed stronger learning–focused relationships with parents. Parents also felt more valued and had a greater desire to be involved in the centre programme.
Inspirational COI reports full of practical ideas for practice can be found at Education Counts: COI (Centres of Innovation).
Effective self review
In 2009, ERO identified some common features of practice in the ECE settings where self review was well understood and implemented. Each of these features of practice and contributing factors is discussed, with specific examples in Implementing Self Review in Early Childhood Services (Education Review Office, 2009).
For these settings:
Self review was integral to the operation of the service and focused strongly on improvement.
Well-established procedures guided self review.
Reviews were both planned and spontaneous.
ERO also identified factors common to ECE settings where self review was well understood and implemented. These included:
strong leadership in promoting self review
professional development to support self review
stable and collaborative staff
sound systems for self review, and the use of relevant resources.
Kaiako can use these questions in team discussion to guide them through the process of internal evaluation and inquiry.
Whose knowledge is valued and reflected in our service’s curriculum?
How well does our curriculum respond to the strengths, interests, and needs of our infants, toddlers, and young children?
To what extent does our service whāriki support diverse learning needs so all children can be confident and competent learners?
How competent and capable are we at evaluating our teaching practices and the impact of these on outcomes for children? How do we know?
To what extent do our collaborative inquiry and evaluation processes focus on the impact of our bicultural curriculum whāriki for all children?
How do our collaborative inquiry and evaluation processes lead to improved practices?
The following questions have been taken from ERO’s report Working with Te Whāriki (2013).
To what extent is Te Whāriki referenced in our statement of philosophy?
Which aspects are included?
Why were these aspects included?
What do we know about how well our philosophy is enacted in practice?
How explicit is Te Whāriki in our service’s curriculum?
To what extent is our service’s curriculum based on all of the aspects of Te Whāriki?
What aspects of Te Whāriki are included? For example, principles, strands, goals, and learning outcomes?
Does our service give greater emphasis to some aspects of Te Whāriki than others? Why?
What might this mean for children’s learning?
How is Te Whāriki visible in our service’s curriculum?
Which aspects are visible?
Are we just using the language of Te Whāriki, or do we have a deeper understanding of what the principles and strands mean for curriculum in our service?
What informs and guides our bicultural curriculum?
How do Te Whāriki, Ka Hikitia, and Tātaiako inform our bicultural practice?
What do we know about the impact of our bicultural curriculum on Māori children? On all children in our service?
What framework(s) do we use to evaluate the quality and effectiveness of our service’s curriculum?
Regulatory requirements for internal evaluation
Early childhood services are required to undertake review and evaluation as part of their licensing requirements (GMA6). The criteria to assess the governance, management, and administration standard specifies that an ongoing process of self review helps the service maintain and improve the quality of its education and care. The licensing criteria require early childhood services to document:
a process for reviewing and evaluating their operation
a schedule showing timelines for planned review of different areas of operation
recorded outcomes from the review process.
For more information, go to the regulatory framework.
ERO’s evaluation indicators
ERO’s evaluation indicators highlight the importance of leading internal evaluation (self review). The indicators of effective practice include the following:
Leaders are focused on improving the quality of education and care through ongoing systematic internal evaluation.
Internal evaluation is valued, championed, and effectively led.
Internal evaluation includes the gathering and analysis of useful information from a range of sources.
Leaders ensure teachers have time to critically reflect on their practice.
Leaders use evidence to reflect on and improve practice.
Good use is made of resources and research to support internal evaluation.
Leaders access professional learning and development to increase their individual and collective capability in internal evaluation.
Leaders ensure that all members of the service have opportunities to be involved in internal evaluation.
Evaluation is documented and leaders ensure that outcomes are shared with those involved in the service.
Empowerment – Whakamana
Inquiry and evaluation processes include children’s perspectives and views. Children’s agency and empowerment is promoted.
Family and Community – Whānau Tangata
Inquiry and evaluation processes take account of the views and perspectives of parents, whānau and community.
Holistic Development – Kotahitanga
Inquiry and evaluation processes are undertaken in a way that is mindful of all dimensions of children’s learning and development.
Relationships – Ngā Hononga
Relationships are integral to all Inquiry and evaluation processes. Relational trust supports effective inquiry and evaluation.
Cardno, C. (2008). Action research in early childhood centres: Balancing research and professional development goals. New Zealand Research in Early Childhood Education (Vol. 11).
Dalli, C, E J White, J Rockel, I Duhn, with E Buchanan, S Davidson, S Ganly, L Kus, & B Wang. (2011). Quality early childhood education for under two year olds: What should it look like? A literature review. The Quality Debate: Past and present discourses. pp 25–50.
Cullen, J. (2008). Outcomes of Early Childhood Education: Do we know, can we tell, and does it matter? NZARE Annual Conference, Palmerston North, 2008.
Grey, A. E. (2006). Self-review: How far have we journeyed? Early Education 40, pp. 33-35.
Grey, A. E. (2010). Self-review as practical philosophy: A case study in early childhood education in Aotearoa New Zealand. Auckland: Auckland University of Technology.
Wansbrough, D. (2004). Management Matters. Self review – the hot topic. Early Education 35, pp. 23–30.
White, E. J. (2007). Systematic inquiry as evaluative activity: Self review in early childhood education. Early Education 42, pp. 26–29.
References
