In 2022 I attended a First Nation's panel discussion on the decolonisation of galleries, libraries, archives and museums. One of the panellists, Wiradjuri Man and Archival De-colonist Nathan “Mudyi” Sentance, reflected on people’s hesitancy to criticise such institutions because of the value they bring to society. He argued instead that it is especially important to criticise such institutions because they have the capacity to and should be doing better. Philanthropy is no different. It is presumed by virtue of its title to be contributing to social good, but that doesn’t mean it shouldn’t be open to self reflection, evaluation, and mechanisms of transparency and accountability; after all it is community money that philanthropists steward.

Commonly, funders delegate the responsibility of proving impact to grantees, asking “How did you use the dollars we granted and how were people’s lives (or society) made better as a result?” While we agree, it’s important for grant partners to apply rigour to measuring, evaluating, and understanding the impact of their work in community and the ripples of change it creates, we are also interested in turning that mirror on ourselves. We want to understand our impact beyond ‘dollars granted’ and look at how we as a funder can create enabling conditions for good work to happen through how we grant, how we show up as partners, and how we share what we are learning openly.

Through this partnership we agreed that good measurement systems are ones that are fit for purpose and adopt an approach that is aligned with the organisational culture and needs.

For the Foundation, we want a MEL system that…

• Won’t create a burden for our partners in terms of heavy evaluation requirements

• Can be embedded in how we do things and implemented through existing rhythms and routines

• Sees us taking a humble approach, not over claiming our impact but focused on capturing the ripples and influence of our collective work

• Helps us to remain clear, strategic, and intentional

• Provides avenues for us to be transparent and accountable 

• Is based on building a culture of learning and valuing the process

Our SIML Framework has a few key components:

• An overarching Theory of Change for the Foundation

• ‘Mini’ Theories of Change for key focus areas e.g. Spark Grants, Strategic Partnerships

• Impact Log: app. and web-based log for tracking instances of impact and process learnings against predetermined domains on the fly

• SIML handbook - a ‘how-to’ guide for operationalising the SIML across the foundation, includes key tools like partnership surveys, Most Significant Change interview guides, and rubrics to understand systems shifts around key impact areas identified e.g. shared language

• SIML Advisory Committee to apply a strategic lens and help us translate data and evidence into learnings and actions

We’ve been operating as a Private Ancillary Fund for 20 years, but over the past several years with a growth in corpus and shift from a ‘desktop’ to staffed foundation, we’ve continued to become more strategic about where and how we fund. 

Developing our Theory of Change as a team saw us coming together to define the Foundation’s goals under our current strategy, the different pathways to reach those goals and where the Foundation’s functional areas sit within this strategy. With our end goals in place we could then ask ourselves, what enablers will be required and what are the foundational activities we need to undertake to give us the best chance of making this happen. 

 … then we tested it with our partners. 

We asked, 

• Can you see your work in this? 

• Does this align with your experience of partnering with us? 

• Is there a role or function you see us playing that we’ve missed?

This step was invaluable, it uncovered areas we’d missed, terms we needed to define, and some miracle leaps between actions and outcomes.   

There’s no point in a Theory of Change that sits in a DropBox folder or Google Drive, only dusted off when it’s time to onboard a new team member or pulled out to include in a fancy presentation (or online article wink). 

We are actively using our TOC to…

• Understand where we are and aren’t seeing evidence of impact - this helps us know when we’re on track and where we might need to refocus efforts, or what’s just going to take a little more time 

• Redesign our granting processes and programs based on evidence and feedback to ensure alignment to specific outcome pathways

• Create transparency with our partners and wider community by openly sharing what informs our decisions

• Share the journey with others interested in understanding the impact of their philanthropy beyond dollars granted

• Know when it’s time to revisit our longer term strategy

Each of our grant streams has an individual plan for capturing impact aligned to the grant type, timeframe, intentions and where it aligns to our TOC. Most importantly we want to support our grant partners to have a plan from the start of the grant that supports them to identify and capture data that is relevant to them, their organisation, and the future of their work. Secondary to this is how our partners can contribute to how we understand our impact through their work and how, through our processes and ways of partnering, we enable or hinder that. 

This is captured through a combination of the following tools:

• Partnership and principles surveys - how well do we live up to our principles and how do we show up as partners, what does this enable or unlock?

• Most Significant change storytelling data collection tool 

• Initial meeting to identify where and how their work aligns to our TOC 

• App. based impact log where they can log against TOC domains they feel are most aligned to their work

All this helps us to understand our impact beyond the dollars granted, things like: what doors were unlocked, what new capability was built, what connections were woven, and how did new parts of a community or system come together? What new knowledge or understanding has been contributed to, how has the narrative shifted? - And how do we do better next time?

Social impact measurement and learning in the philanthropic context is new to us, and by no means do we think we’ve nailed the 'meaningful impact evaluation' equation. What we do think is that we have is a pretty cool system for measuring impact across different domains and in different ways to capture the rich complexity of the spaces we work in and some incredibly generous partners that are on the journey with us.

We’re sharing about this in the hopes we open some conversations with others that may be grappling with where to start when it comes to understanding their impact while avoiding the same pitfalls we hoped to. We’ll be sharing more as we deepen our impact evaluation practice, both in the form of impact and learning reports but also learnings about the implementation of the framework itself.