Systems Thinking for Evaluation: How to Know If Your Change Efforts Are Working

You built a plan. You started doing the work. But how do you know if it’s actually working?

Chapter 12 of Systems Thinking for Social Change by David Peter Stroh tackles evaluation. Not the boring, fill-out-a-form kind. The kind that actually tells you whether your change efforts are making things better or just moving numbers around on a spreadsheet.

This matters because evaluating social change is hard. The problems are complex. Funders want results fast. Organizations set goals that are too ambitious just to win grants. And nobody wants to spend their limited budget on evaluation when there’s real work to do.

But without honest evaluation, you’re flying blind. Systems thinking gives you a better way to check your compass.

Five Systemic Guidelines for Better Evaluation

Stroh lays out five ways systems thinking improves the evaluation process. Each one pushes back against how most organizations currently do it.

1. Set Realistic Goals

This sounds obvious. It’s not.

Funders want big results because they want their money to matter. Grantees promise big results because they want the funding. Policy makers create unfunded mandates. Everyone ends up in a cycle of overpromising and underdelivering.

A systemic theory of change helps here. It shows you the scope of what you’re trying to do, the order things need to happen in, and the time delays built into the system. That makes it easier to set goals you can actually hit.

Stroh recommends no more than three goals at any one time. And when new priorities show up (they always do), you need to consciously drop something else. Taking on new work without letting go of old work just makes everyone stressed and reactive. That stress creates its own vicious cycle.

2. Define Clear Indicators and Metrics

Here’s where it gets interesting. Systems thinking says you need both quantitative and qualitative measures.

Quantitative is the stuff you can count. Test scores, dollars spent, people housed. Qualitative is the stuff that’s harder to measure but just as important. The level of collaboration between service providers. Whether people share a common language. Whether stakeholders are actually committed to shared goals.

Stroh also pushes for ratio-based metrics instead of just raw numbers. It’s not enough to know a jobs program costs fifty thousand dollars, provides a hundred hours of training, and places 70 percent of graduates in living wage jobs. You also want to know:

• Productivity: How many quality jobs per dollar spent?
• Effectiveness: How many quality jobs per training hour?
• Efficiency: How many training hours per dollar?

These ratios tell you about leverage. Some actions give you way more return on investment than others. Ratios help you find those actions.

For example, when Stroh and a colleague were asked to propose metrics for ending homelessness, they suggested tracking things like: the monthly decline in homeless people (people housed minus people becoming homeless), prevention effectiveness (new homeless people divided by people at risk), and the financial dividend from job creation (rent and taxes generated minus training costs).

3. Think Differently About Short and Long Term

Remember from earlier chapters: systems tend to show better-before-worse behavior. A quick fix looks great in the short run but makes things worse later. A real solution might look slow at first but builds lasting change.

So how do you tell the difference between a quick fix and a genuine small success? Stroh gives six tests:

• Remember that short-term improvement doesn’t guarantee long-term improvement.
• Check whether the action addresses root causes or just symptoms.
• Ask what theory of change informed this action.
• Make sure time delays were considered when choosing what to do.
• Look for early wins that build system-wide capacity (relationships, insight, organizational infrastructure) rather than ones that just reduce a symptom like low test scores.
• If possible, build a simulation to test short-term versus long-term consequences.

The key insight: early signs of real progress often look like relationship building and shared understanding. Not like dramatic improvements in outcome metrics. If you’re only measuring outcomes in year one, you’ll miss the foundation being laid.

4. Look for Consequences Along Multiple Dimensions

Conventional evaluation asks: did we hit our target? Systemic evaluation asks a much broader set of questions.

First, look for unintended consequences. Not just the bad ones. Some unintended consequences are actually positive, and you want to build on those.

Second, look at resources created, not just resources spent. Did the intervention generate new capacity that didn’t exist before?

Stroh shares a great example from Togo, West Africa. Engineers built wells in rural villages to replace water holes infected with guinea worm. Villagers were thrilled. Two years later, the wells were abandoned because the pumps broke and nobody knew how to fix them.

So the engineers went back. This time, they taught villagers to maintain their own wells. They helped them set up systems for collecting repair money and stocking parts in nearby towns. The result? Not only did the wells stay working, but the villages used their new organizational skills to start their own development projects for additional food and income.

The intended consequence was clean water. The unintended consequence was community self-sufficiency. Both matter for evaluation.

5. Commit to Continuous Learning

Systems are complex and they evolve in ways you can’t fully predict. Evaluation can’t be a one-time thing. It has to be ongoing.

Three things help here:

Longer-term funding commitments. When funders commit for the long haul, everyone can be more honest about failures. Nobody feels the need to hide bad results or inflate good ones just to keep the money flowing.

Ongoing stakeholder engagement. Keep involving existing stakeholders and bring in new ones. This provides the feedback you need to adapt, and it builds ownership among more people over time.

Refine your theory of change. Use what you learn from experiments and stakeholder feedback to update your systems analysis. The more accurate your understanding, the more effective your next steps will be.

Tracking Success Amplification

If you’re using a Success Amplification approach (building on what’s already working), there’s a specific growth pattern to watch for. It follows an S-curve with three phases.

Phase 1: Slow growth. Progress isn’t visible yet. This is where you build the foundation. Common ground. Relationships across service providers, institutions, and beneficiaries. Organizational capacity.

Most funders expect linear growth. They want to see measurable results right away. But organic growth doesn’t work that way. In this first phase, the right things to measure are: Do stakeholders share a common vision? Are service providers collaborating? Are organizations investing in their own infrastructure?

Stroh points out that underinvesting in nonprofit infrastructure (computers, information systems, marketing, personnel development) significantly reduces impact. Keeping overhead artificially low doesn’t help anyone in the long run.

Phase 2: Steep growth. Now outcomes become visible. This is where it makes sense to measure results. Compare outcomes to outputs and inputs. Calculate the return on investment. Show the benefits of saved and more productive lives versus the cost of doing nothing.

Phase 3: Maturation. Growth slows down and levels off. The temptation here is complacency. “We did it.” But the real task in this phase is innovation. Find new engines of growth instead of pushing harder on the old ones. Measure whether the organization is planting seeds for a new approach, not just resting on past success.

Tracking Goal Achievement

If you’re using a Goal Achievement approach (closing the gap between where you are and where you want to be), the trajectory has two phases.

Phase 1: Correction. Similar to the foundation-building phase above. Establish common ground, build relationships, develop organizational capacity. But there’s extra emphasis on understanding current reality. You need to clearly identify the root causes of why you haven’t been able to reach your goal despite your best efforts. If things aren’t progressing, go back and recalibrate your understanding.

Phase 2: Reinforcement. As you start making progress, keep the momentum going. Make sure goals increase along with performance. Identify new growing actions. Redirect dividends from initial improvements to fund the next round of actions.

The Bottom Line

Systemic evaluation is bigger than just measuring outcomes. It includes:

• Setting realistic goals (no more than three at a time)
• Tracking both quantitative and qualitative indicators
• Thinking differently about short-term versus long-term results
• Looking for unintended consequences, both positive and negative
• Treating evaluation as continuous learning, not a final judgment
• Knowing what phase of growth you’re in and measuring the right things for that phase

The point isn’t to prove your program works. The point is to learn what’s actually happening in the system so you can adjust course and do better over time.

That’s a very different mindset from “did we hit our numbers this quarter.” And it’s the mindset that actually leads to lasting social change.

This post is part of a series retelling “Systems Thinking for Social Change” by David Peter Stroh (ISBN: 978-1-60358-580-4). Read the full series for a complete walkthrough of applied systems thinking.

Previous: Systems Thinking for Strategic Planning | Next: Becoming a Systems Thinker