Blog: Evaluating Transformative Change: Insights from the Indonesia Green Principle Awards Workshop

For an in-person evaluation workshop in Indonesia, Suci Lestari Yuana used the Formative Evaluation guidelines as a basis and transformed the MIRO canvas for identifying Transformative Outcomes into an offline exercise. A simplified version of the framework helped workshop participants to reflect on the impact of their circular economy initiatives. 

In the pursuit of sustainable development, transformative change plays a vital role in challenging existing systems, questioning assumptions, and embracing innovative approaches. Recently, we organized an evaluation workshop utilizing the Transformative Innovation Policy (TIP) tools to assess the impact of the Indonesia Green Principle Awards (IGPA) program on transformative change in Indonesian schools. This blog explores our experience, modifications made to the tools, participant engagement, and the potential for creating a community of practice dedicated to transformative change in our region.

Our project 

The Circular School Program Partnership, an initiative that brings forth the  Indonesia Green Principal Award (https://igpa.or.id/), is a trimester program organized by the Centre for World Trade Studies (CWTS) Universitas Gadjah Mada and JBI Education Consulting. It aims to promote circular economy principles in education at the elementary, junior high, and high school levels. The program addresses the pressing environmental challenges and economic development needs in Indonesia. With a focus on transforming mindsets from a linear to a circular economy, school principals are invited as agents of change to drive sustainable practices within their schools. The program emphasises collaboration on the circular economy between academia, industry, government, and society, fostering networks and synergies to achieve a circular society. Implemented gradually for almost two years, the IGPA program combines competition and mentoring to engage participants actively. By raising awareness and promoting circular economy principles, the program strives to create a network of dedicated school principals working towards transformative change for a sustainable future.

Evaluation Tools and Modifications:

After running for almost two years, we decided to organize an evaluation workshop on 27th May 2023 to learn the impact of IGPA and to identify our room for improvement. Prior to this workshop, we have sent invitations to IGPA 1,2, and 3 alumni as well as teachers from private and public schools. We had 43 participants registered for this workshop, including 25 school principals from private schools and 18 teachers from Jakarta, West Java, Central Java, and East Java public schools. Out of the participants, 24 were women. Among them, 25 had previously attended the IGPA workshop, while 18 were new to the program.

To ensure accessibility and active participation, we decided to modify the original TIP tools. Inspired by the TIP guidelines on formative evaluation, we transformed the online tool Miro app tools into a more tangible format, using physical materials such as sticky notes and pens. This adjustment aimed to enhance participant engagement and provide them with a visual representation of their evaluations. We draw inspiration from the multi-level perspective on transition which refers to a framework for understanding and analyzing complex societal

transitions, particularly in the context of sustainable development and environmental changes. The MLP theory on transformative changes provides a dynamic framework for understanding how societal systems transition from one state to another. It emphasizes the importance of regime crises, niche innovations, and interactions between different levels of society and innovation. This theory recognizes that transformative changes are complex and non-linear processes that involve multiple actors, contexts, and feedback loops. In this workshop, we formulate the transformative outcomes of circular school initiatives into three stages: niche building, niche mainstreaming, and unlocking regimes. 

By using the metaphor of “demolishing the wall,”  participants were able to visualize their progress and better understand the trajectory of their transformative initiatives. The metaphor of “demolishing a wall” is often used in education policy research to symbolize breaking down barriers, overcoming obstacles, and eliminating obstacles that hinder progress or understanding. It’s a powerful image that conveys the idea of tearing down something that separates people or ideas, allowing for greater communication, collaboration, and connection.

Picture 1: The examples of results from implementing the evaluation tool which is translated into building niches, mainstreaming niches, and unlocking regimes

Participant Engagement and Reflections:

During the workshop, participants were encouraged to reflect on their circular economy initiatives and evaluate their impact using the modified tools. This process yielded valuable insights, allowing participants to better understand the scale and trajectory of their transformative initiatives. It provided them with a structured framework to assess the effectiveness of their programs and identify areas for improvement and integration. Through this reflection, participants were able to engage in critical discussions about the outcomes of their efforts.

Picture 2: Participants are reflecting their circular economy initiatives

One of the notable outcomes of the evaluation workshop was the exchange of knowledge and experiences between participants who had previously attended IGPA workshops and those who were new to the program. This exchange fostered a collaborative learning environment, enabling participants to share insights, lessons learned, and best practices. Interestingly, as participants discussed their circular economy initiatives, some newcomers realized that they had been unknowingly practising circular economy principles. For example, before joining IGPA, some schools have already enacted policies and daily practice to separate trash into three categories—organic, non-organic, and recyclable—representing the traditional 3R (Reduce, Reuse, Recycle) in the circular economy principles. Other participants shared how they further the principle by creating art projects from recycled waste or plans to reuse water from ablution to water plants. This revelation highlighted the power of the evaluation tools in facilitating knowledge exchange and promoting a deeper understanding of transformative change.

“After participating in IGPA, I realized that my previous knowledge about circular economy was limited to the basic concept of the 3R. I had never heard about the broader concept of the circular economy before. Through IGPA, I was exposed to a wealth of information and resources that expanded my understanding of the topic. I learned that the circular economy encompasses much more than just waste management and recycling. Participating in IGPA has not only broadened my knowledge but also inspired me to advocate for the adoption of circular economy practices in my school and community. I now understand the importance of spreading awareness about circular economy principles and working towards their implementation” – Arfi Destianti (Principle of Yayasan Semut Beriring, West Java).”

The evaluation tools allowed for the exchange of knowledge and experiences between IGPA alumni and new participants, as well as among the alumni themselves. This knowledge exchange enabled participants to question their assumptions, broaden their perspectives, and challenge their existing practices.

“To be honest, I didn’t fully understand it because I have never participated in IGPA. However, the term ‘circular economy’ is frequently heard as school principals often mention it. After participating in this evaluation workshop, I came to realize that our schools have been practicing the circular economy all along, even though we never consciously recognized it as such. It was eye-opening to discover that the initiatives we have implemented align with the principles of the circular economy. We have been actively collecting and repurposing waste materials, finding creative ways to give new life to discarded items, and promoting sustainable practices within our school community. It’s amazing to see how our efforts, which we considered to be separate and unrelated actions, fit into the broader concept of circular economy” – Septimbrawati (Teacher from SD Muhammadiyah Sukorejo, Kendal, East Java) 

Interestingly, in the transformative change question where participants were asked to categorise how far their school has progressed in adapting the circular economy concept, there was a significant difference in response between members who have joined the IGPA and those who haven’t. IGPA alumni exhibit more confidence, having reached the mainstreaming niche and unlocking regime than participants who have not joined the IGPA program. This provides us with an insight into how the IGPA program might play a factor in educating and motivating the principals to adopt the transformative agenda of the circular economy  into their daily school activities. It means, even though participants have practised some circular programs prior IGPA workshop, but the knowledge and experiences they gained in the IGPA workshop allowed them to link it to a holistic vision of transformation. 

Though non-IGPA alumni have also displayed circular economy principles in their school, they seem less confident and often use wishful statements that ‘hope’ and ‘want’ their school to implement more recycling activities, as compared to IGPA alumni who confidently listed their past activities and progress. 

Furthermore, the lively interaction between IGPA alumni shows a healthy rivalry where everyone is encouraged to think and make more transformative changes in their school. This is also an onus to IGPA’s annual activities, which award the best principal that creates the most transformation in their school. 

Facilitator’s Experience and Future Improvements:

As a facilitator, the workshop provided a rich learning experience. Modifying the evaluation tools and using physical materials resulted in a higher level of engagement and interaction among participants. However, we identified areas for improvement in future exercises. For instance, allocating more time for participants to familiarize themselves with the evaluation

framework before the workshop would ensure a higher level of engagement. Additionally, language barriers emerged as a challenge, and we recognized the need for co-translating workshops to ensure that the evaluation tools are easily understandable in the local language. Addressing these limitations will further enhance the effectiveness of the evaluation process and promote inclusivity. The lab resources, such as the TIP guidelines on formative evaluation, provided a solid foundation for designing the workshop and implementing the evaluation tools effectively.

Moving forward, our commitment lies in nurturing a community of practice for transformative change in our region. The evaluation workshop was just one step in this journey. We are actively implementing the TIP tools in our own research and initiatives and gradually introducing them to our peers. Collaborative research projects, workshops, seminars, and knowledge-sharing initiatives will form the foundation of our community-building efforts. By engaging with the TIP community and leveraging existing networks, we aim to inspire and empower others to drive transformative change in their respective domains.

Shortly after our workshop, we saw principals discussing among themselves projects they plan to do and lessons learned from their current circular activities. Some exchanged numbers and made further plans to meet and share their report, particularly with the Head of the Principal Association of a private school organization in Indonesia, who attended the workshop and was an IGPA alumni. 

We believe that the IGPA program serves as a platform for schools to gain knowledge and incorporate it into their school policies. However, after the IGPA program, participants have ample opportunities to engage in further discussion among participants about their school policies, exchange knowledge, and create a collaborative program to maximize the implementation of the circular economy principles in their respective schools. 

In conclusion, the evaluation workshop utilizing the TIP tools provided valuable insights into the impact of the IGPA program and its contribution to transformative change in Indonesian schools. The modifications to the tools, participant engagement, and knowledge exchange exemplified the potential for creating a community of practice dedicated to transformative change. By continuously refining our evaluation processes, engaging in critical reflections, and fostering a culture of collaboration, we can collectively work towards a more sustainable and inclusive futures. For example, after our workshop some participants in East Java collaborate to organize another circular economy workshop and invite teachers who were not part of IGPA community. This exhibit the collaborative outcome from the IGPA program.  

As we navigate the challenges and opportunities presented by transformative change, it is essential to harness the power of evaluation and foster a community of practice that shares knowledge, challenges assumptions, and drives innovation. The journey towards transformative change requires collective efforts, and by nurturing a community that embraces collaboration, we can make a meaningful impact in our region and beyond.

Research Team: 

1. Suci Lestari Yuana, Lecturer at the Department of International Relations, Universitas Gadjah Mada, Yogyakarta, Indonesia 
2. Maharani Hapsari, Assistant Professor at the Department of International Relations Universitas Gadjah Mada, Yogyakarta, Indonesia
3. Milda Longgeita Br Pinem,Assistant Professor at the Department of Social Development and Welfare  at Universitas Gadjah Mada, Yogyakarta, Indonesia
4. Wendi Wiliyanto, Undergraduate student at Department of International Relations, Universitas Gadjah Mada, Yogyakarta, Indonesia
5. Tane Andrea Hadiyantono, Postgraduate student at the Department of International Relations, Universitas Gadjah Mada, Yogyakarta, Indonesia
6. Nindita Nilasari, Staff of Institute of International Studies at Universitas Gadjah Mada, Yogyakarta, Indonesia 
7. Maria Josefina Figueora, Associate Professor at the Copenhagen Business School, Denmark 

This article is part of a research project titled “Mundane Circular Economy Policy: Mainstreaming CE through the Agency of Schools.” This project is funded by FISIPOL UGM 2023 under the Research, Publication, and Community Service Grants 2023 managed by the Institute of International Studies, Department of International Relations, Universitas Gadjah Mada, Indonesia.