This 5-page article on Action Evaluation from the Beyond Intractability website provides a knowledge base and tools regarding social conflict resolution and community reconciliation. This article assists participants, funders, and facilitators in reaching consensus about what they seek to accomplish (people's goals), why (their values and beliefs), and how (suggested account strategies). The article addresses the process in three stages: establishing the baseline, formative monitoring, and summative evaluation.
Theory
The Annotated Guide to Tools & Resources provides tools, frameworks, and resources to help you develop and implement your evaluation. It’s a repository of useful, practical materials that can help you create an evaluation plan; design your evaluation approach; develop or adapt tools and instruments; and otherwise move your evaluation forward.
This Guide was originally assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts. We continue to add resources. Your suggestions are welcome!
Do you have a useful tool or resource to add? Contact animatingdemocracy@artsusa.org.
Publication Date: May 16, 2012
Resource Format: book / article
This piece sheds light on the philanthropic sector’s efforts to improve measurement and evaluation (M&E), specifically within the context of foundations involved in social change work. The broad observations of the six contributing authors are that purpose; cost-benefit ratio; culture, context and capacity; unit of analysis; timing; feedback; and transparency matter to measurement and evaluation. Luis A.
Based on several years of field research in communities around the U.S., this six-page brief presents an insightful framework for better capturing and measuring arts, culture, and creative expression at the neighborhood level.
Resource Format: book / article, practical tool
Written for Animating Democracy's Arts and Civic Engagement Impact Initiative Working Group, this 14- page paper presents a conceptual framework (or logic model) for arts-based engagement. It offers a discussion of the components of the framework, and a list of questions to guide research explorations.
The site offers an overview of Urban Institute's Arts and Culture Indicators Project, defines cultural vitality, briefly addresses cultural vitality indicators, offers case study examples, and gives abstracts and links to high-quality relevant research studies. The site is a beautiful resource for arts and civic engagement practitioners and policymakers. Graphically strong with top notch research, it is a model for the field.
Resource Format: book / article
This 43-page report is a literature review commissioned by the Arts Council of England to support its two-year social inclusion research program. It examines the impact of arts in addressing social exclusion with the purpose of informing the design of the research and placing it in a policy context. The first section attempts to define social exclusion, offers ideas and obstacles to measuring it, and relates it to the arts.
Resource Format: book / article, case study
This 544-page book by sociology professor Robert D. Putnam makes the case that the social capital of American citizens is declining because of increasingly diminished connections with their communities. He supports this claim with extensive interviews and research, much of which is arranged in charts and graphics.
Resource Format: practical tool, website
The California Endowment believes that evaluation represents an important opportunity to assess achievements, generate knowledge and cultivate learning. It works to support grantees, evaluators and others interested in learning more about culturally competent health evaluation. Its website has a wealth of resources, reports, and articles useful to both funders and practitioners. Of particular relevance are items related to storytelling approaches to program evaluation, multicultural concerns in evaluation, and understanding impact of advocacy efforts.
Resource Format: book / article
Written for Animating Democracy's Arts and Civic Engagement Impact Initiative, this 69-page paper speaks directly to arts and civic engagement work, surveys current research, and makes recommendations for future practice. The paper has three sections. The first attempts to define and differentiate civic terms: civic engagement, social capital, public sphere, community engagement, community and civic capacity, arts, culture, humanities, social inclusion, cultural citizenship, and the cultural public sphere.
Resource Format: book / article, case study
Drawing from Craig McGarvey’s seven years of professional experience as Program Director in Civic Culture at The James Irvine Foundation, a California-wide philanthropy, Civic Participation and the Promise of Democracy explores civic engagement strategies (faith-based and community organizing, popular education, collective learning) through providing an overview of their use in California. During McGarvey’s time at Irvine’s Civic Culture, the program supported Californians who were working to build a durable pluralism from the State’s unprecedented demographic diversity.
Publication Date: December 31, 2010
Resource Format: book / article
The authors cite nonprofits collaborating in an education system to introduce the concept of collective impact, defined as, “the commitment of a group of important actors from different sectors to a common agenda for solving a specific social problem.” Collective impact is unlike collaboration efforts and partnerships in that its “initiatives involve a centralized infrastructure, a dedicated staff, and a structured process that leads to a common agenda, shared measurement, continuous communication, and mutually reinforcing activities among all participants.” In short, large-scale, complex
Resource Format: book / article, case study
Critical Perspectives is a collection of reflective, critical, and creative essays about the role of arts and humanities in civic dialogue. Twelve essays focus on three very different projects that employed the unique capacities of theater, visual art, and historic preservation to reach people and stimulate them to talk together in new ways about issues that matter in their communities.
This 104 page report (with eight chapters and seven appendices), based on extensive community research, discusses cultural vitality and gives extensive information on indicators. This piece -- primarily for evaluators, researchers, and policymakers -- is written in clear, user-friendly language. The fundamental goal of the Urban Institute's Arts and Culture Indicators Project (ACIP) is to help policymakers make better decisions for neighborhoods and cities.
Publication Date: April 1, 2012
Resource Format: book / article
Data Visions, by data artist Aaron Koblin, calls for artists and computer-scientists alike to tap into the growing field of data visualization. Through his own works, Koblin demonstrates the power of numbers to tell a story. This new language transcends boundaries and creates a space where quantitative research can become accessible through its more simplistic visual representation. As Koblin suggests, new types of creativity are waiting to be uncovered by anyone who takes this new technology and uses it to re-think old rules.
Resource Format: practical tool
This checklist is a guide to incorporating democratic processes in an evaluation to assure valid conclusions where there are conflicting views. A concise checklist aims to ensure that the evaluation process, and thus conclusions, are inclusive and unbiased. The checklist is built around three principles: 1) Inclusion, 2) Dialogue, and 3) Deliberation. The Inclusion principle implies that all relevant interests, values, and views of major stakeholders are taken into account when designing and conducting the evaluation.
Resource Format: book / article, practical tool
This short book defines developmental evaluation and addresses some myths. Offering an alternative approach to linear logic of traditional methods, developmental evaluation supports the process of innovation within an organization and its activities. The book outlines developmental evaluation's uses in innovation and developing project ideas and defends the approach's credibility. It is an appropriate resource for organizations that work in an innovative manner and thus seems tailored to arts organizations.
Publication Date: May 22, 2013
Resource Format: case study
Continuum: Capacity (social capital, skills, practice, resources) | Systems, policies, places (legislation, issue/problem resolution)
Developmental Evaluation: An Overview (J.W. McConnell Family Foundation Powerpoint) presented at the Animating Democracy/Americans for the Arts' Funder Exchange on Evaluating Arts & Social Impact at the Nathan Cummings Foundation on May 22, 2013.
Publication Date: December 9, 2019
Resource Format: practical tool
Diving into Racial Equity written by Vanessa Whang, examines the MAP Fund’s deep examination of one of its foundational priorities—racial equity in arts and culture grantmaking—and ongoing efforts to change practices toward this goal. The MAP Fund supports original live performance projects across the U.S. that embody a spirit of deep inquiry, particularly works created by artists who question, disrupt, complicate, and challenge inherited notions of social and cultural hierarchy.
The initial scoping phase, so important for providing a sound and lasting framework for the data and evidence reviews, has now ended and the hard work has begun. Technical work on the modelling is under way, including reviewing over 50 different datasets in the UK that could be useful for the model. The systematic review of the research – a wide-ranging hunt for any study in our sector that addresses engagement – has unearthed over 10,000 studies. This allows us to begin the process of mapping the research onto systems dynamics model of engagement, putting evidence-base into action.
Resource Format: book / article, case study
Abstract: This paper tells two stories of how evaluation helped artists know what difference they made; their narratives help make evaluation concepts accessible and show how evaluation can be doable and even enjoyable! Most community-based arts practitioners feel overwhelmed by what it might take to implement credible evaluation. They’re pressed to define what is meant by “civic” or “social” impact, whose standards to apply, what evidence to look for, and what to document and track.
Publication Date: July 31, 2012
Resource Format: book / article
The authors suggest that traditional evaluation approaches (formative and summative) fail to meet the complex needs of social sector innovators. Instead, grantmakers should approach evaluation differently, specifically involving the use of developmental evaluation (attributed to Michael Quinn Patton). Through a review of literature, interviews, and case studies, this piece assists with putting developmental evaluation into practice. At the heart of this call for new evaluation approaches, is the encouragement of social innovation and change.
This website is a good resource for individuals and community groups working for change in their communities, specifically with issues relating to race and racial equity. The site is written for users who are new to evaluation -or perhaps those who give funds related to racial equity, but who are not yet clear on how to evaluate it. The site discusses how to apply a “racialized” perspective to evaluation, meaning using the ideas of racism, oppression, privilege and access to power as a lens through which evaluation questions are developed and results are analyzed.
Publication Date: July 1, 2014
Resource Format: book / article
Commissioned by the Working Group on Philanthropy for Social Justice and Peace (PSJP), Framing the Discourse, Advancing the Work, by Moukhtar Kocache, speaks to funders as an exploration of philanthropy at the nexus of social justice, peace, and arts and culture. It presents a broad overview of the relationship between progressive social change work and the arts and explores the role of philanthropy in supporting this work. It sets out recommendations for how philanthropy might further its engagement with work at this nexus.
To get insight into a complex community, problem or process of change, sometimes you need to look beyond conventional research or evaluation methods. Ethnography is a powerful way to step inside the culture of an organization or community, hear ongoing feedback from multiple points of view, and understand a program's real impact. This guide describes ethnography's benefits and pitfalls, and how the method is used to document, evaluate and improve approaches to youth engagement, HIV education and neighborhood policing.
Publication Date: June 30, 2014
Resource Format: practical tool
Created by the Georgia Council for the Arts, this guide provides a useful entry for local and state arts agencies and arts organizations newly interested in evaluation. The guide is designed not only as an introduction to evaluation, but to demonstrate how evaluation can be used as an internal learning tool within an organization to help manage programs, spend funds wisely, and assist constituents and potential investors to better understand their work.