Understanding the Basics of Evaluation
Overview
Resources in this section provide an overview of some of the common terms used in evaluation, as well as overviews of traditional evaluation methods. When reviewing these, it is useful to also keep in mind two particular issues not always addressed in evaluation resources. One has to do with setting reasonable expectations for progress, acknowledging the depth and interwoven nature of structural racism. The other has to do with trying to predict which changes a group observes in the short term are likely to predict positive outcomes in the long-term, given the likelihood of resistance and retrenchment.
Both of these issues are particularly challenging because racism has never been eliminated in the United States, so as a society we do not know exactly what it will take to meet many racial equity goals, and we do not know exactly what short-term outcomes will lead to longer term ones.
In addition, there are many different factors that influence racial equity goals, not all of which will be under the control of a group wanting to evaluate its work. For example, there is often pressure to assess whether or not a specific set of activities caused changes in community conditions (for example, if organizing parents to advocate for improved schooling caused test scores of children in those schools to change). It is usually not technically possible to answer this type of question fully. Instead, it is more often the case that an evaluation can answer two questions: 1) if test scores changed; and 2) if the strategy to organize parents succeeded in creating better or more advocacy for school improvement. But without very sophisticated and expensive techniques (like random assignment), most evaluations cannot prove if organizing caused improved test scores. It is possible to design an evaluation that will look at test scores over time, and that will look at how well organizing went and what it accomplished. Then, it is reasonable to say whether or not organizing “contributed” to changes in the schools, but not whether they “caused” them. Getting clear on these points – that is, what is reasonable to measure, and how to make a data driven case for the contribution of one’s work to a larger goal – is a skill set at the heart of useful and equitable evaluations.
And, it is important to remember that evaluation should not drive choices about what to do. There is always a way to learn from efforts to achieve racial equity work, and a creative way to document progress, challenges and successes.
Resources
- Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources
- Shared Measures of Success
- How to Develop the Right Research Questions for Program Evaluation
- The Step-by-Step Guide to Evaluation: How to Become Savvy Evaluation Customers
