Why
evaluation?
In program evaluation, you are doing something very like what an investment counselor does in appraising a portfolio of investments, in other words, apportionment evaluation.
—Michael Scriven, paraphrasing Nick Smith,
as quoted by the Oral History Project Team (2005)
Evaluation—data, metrics, KPIs, assessment, evidence, impact—influences which groups end up with access to resources that they can use to shape the world. But not in the “objective,” “meritocratic,” “data-driven” way that neoliberal discourse and the nonprofit/ nongovernmental industrial complex claim.
Instead, philanthropy and European/ Euro-settler governments like that of the USA (the owning and ruling class) spend money on evaluation to measure the value of their investments in organizations and programs attempting to, or expected to, solve problems caused by the very economic system that produces the capital class’s wealth. Capital compares the “value” generated among the organizations and programs that it puts small amounts of money into as grants and contracts using the same logic that it uses to compare financial returns on the exploitative and extractive industries in which it invests its remaining money to sustain itself, often in perpetuity.
Evaluation directly affects the survival of exploited, excluded, and infantilized groups through the stories that it tells about why most people are poor, why some people are rich, the relationship between the two, and whether—as well as how—this relationship could or should change. The capital class uses evaluation to measure organizations’ and programs’ value based on what it values and what it believes about how the world works and how it should work. This includes ideas about who has knowledge, how it is produced, and what it should be used for.
Governments and philanthropy embed these values and beliefs within the interventions, research methods, standards of evidence, and avenues for dissemination that they require of grant and contract recipients as well as what they prescribe in the requests for proposals that they issue for evaluation. Methods come and go, and they will continue to grow and change. But the fundamental accountability structure in which capital defines success and dictates the terms—including of “qualitative,” “participatory,” “culturally responsive,” “decolonized,'“ and “critical” methods—has not changed.
Why “the next day”?
No one will be able to argue that they were
entitled to join The May 12 Group because
it’s called something generic. And so
the idea was you got invited to
The May 12 Group, and
if you weren’t invited, then
you weren’t in, and so
there was no official stuff.
— Dr. Michael Scriven, as quoted by
the American Evaluation Association’s
Oral History Project Team (2015)
It was the late 1960s, on the heels of the Civil Rights Movement and at the dawn of the Black Power Movement. Protests against racialized and gendered capitalism and imperialism raged around the world. A group of educational researchers in the USA began convening. The researchers made their living conducting large-scale evaluations of Great Society programs, particularly War on Poverty initiatives like Head Start. These initiatives were informed by Daniel Patrick Moynihan’s 1965 report The Negro Family: A Case for National Action, which “diagnosed” African American families as mired in a “tangle of pathology.”
In 2004, one of these educational researchers, Dr. Yvonne Lincoln, retrospectively noted how “in the late 1960s and early 1970s…virtually everyone who was writing about evaluation could be invited and accommodated comfortably.” The group they formed called themselves ”The May 12 Group,” after the day they first met, overtly and intentionally to restrict access.
Lady Bird Johnson attends the ceremony for National Head Start Day, June 30, 1965. Children's Bureau Centennial, CC BY 2.0, via Wikimedia Commons
Eartha Kitt performs at the Lincoln Memorial June 19, 1968 on the Solidarity Day of the Poor People’s March on Washington. Photographer unknown, via Flickr
Spring 1961 issue of the Journal of Negro Education
While we have yet to find a roster, no one documented as a member of The May 12 Group by the American Journal of Evaluation’s Oral History Project Team has identified as, or has mentioned anyone else in the group who identifies as, a member of the global majority (groups racially otherized by colonial powers). Drs. Stafford Hood, Rodney Hopson, Pamela Frazier-Anderson, and Tamara Bertrand Jones have documented African American educational researchers as having earned their doctorates in the 1930s and 1940s with some of the same professors who trained the educational researchers who formed The May 12 Group 30 years later. These African American educational researchers had been practicing evaluation and publishing about it in the only journals that would accept their work during the era of legal segregation and Jim Crow. For example, the Journal of Negro Education has featured research by scholars focused on the education of Black people since 1932.
The exclusionary logic of individual merit and deservingness underlying The May 12 Group’s invitation-only membership, and the understanding of knowledge and evaluation that it reflects and reinforces, extends far beyond educational research and predates formalization of “evaluation” by more than three decades—in fact, by more than three centuries. It can be tied to the eugenics movement, scientific management, positivism, and scientism, all of which are linked with (neo)liberalism, fascism, Indian Boarding Schools, imperial expansion, and colonial expeditions. What makes The May 12 Group noteworthy, though, is 1) the political moment during which it arose; and 2) the influence it continues to have on the way that nearly everyone in the world thinks about “evaluation,” whether or not they experience and recognize it as a distinct practice, discipline, or profession.
Screenshot from Why is Evaluation So White?, presented by Vidhya Shanker and Carolina De La Rosa Mateo on behalf of the MN IBPOC in Evaluation Community of Praxis on May 13, 2020 as part of the Center for Evaluation Innovation’s Evaluation Roundtable
The May 12 Group’s use of the words “entitled” and “official stuff” at a time when oppressed groups were demanding recognition and representation is telling, especially considering that its members’ livelihood came from initiatives explicitly referred to as “entitlement programs” and related to equal opportunity policies. Moreover, The May 12 Group went on to merge with another group to become, in 1986, the American Evaluation Association (AEA)—the USA’s only professional association for evaluators today. AEA has subsequently exalted The May 12 Group members as evaluation’s canon. The African American evaluators mentioned earlier, who brought to their work a commitment to justice in the education of Black people, were not members of The May 12 Group and are thus absent from evaluation’s canon, as is the way that they approached evaluation.
Evaluation’s canon matters when we consider how the field is moving toward professionalization. When the deficit-based “pipeline” program it instituted in the early 2000s started to bear fruit a decade later, AEA’s membership responded to its increasing precarity by working more explicitly to reduce supply and increase demand for evaluation services by professionalizing. The understanding of knowledge and knowledge production that The May 12 Group embodied, which has become synonymous with “evaluation,” is embedded in the competencies that AEA arrived at for purposes of credentialing professional evaluators. These competencies omit understanding of oppression or justice, and fail to specify their epistemological or ethical orientation.
Because AEA’s strategy for advancing evaluation as a profession includes helping establish “voluntary organizations for professional evaluators” around the world, The May 12 Group’s understanding of knowledge and evaluation has become hegemonic not just within the USA but also internationally. Of course sovereign governments and local communities increasingly resist the displacement of their collective, ancestral, and experiential knowledge systems and ways of knowing. But epistemicide continues to occur through the nonprofit/ nongovernmental industrial complex and development aid apparatus that many depend on.
Protesters on Lake Street near the Minneapolis Police Department’s 3rd Precinct station raise a poster quoting Martin Luther King: “Riots are the language of the unheard.” Photograph by Fibonacci Blue, CC BY 2.0
In the spring of 2020, during the COVID lockdown, the Center for Evaluation Innovation invited the Minnesota Indigenous, Black, People of Color in Evaluation Community of Praxis to present a webinar on our efforts to resist this hegemony wielded in, through, and around evaluation. The webinar covered evaluation’s ongoing pattern of excluding, erasing, silencing, and punishing voices that question and critique its fundamental assumptions about knowledge and power. Having started with The May 12 Group’s calculated name, we invited attendees to weave together a nurturing ecosystem to reverse, repair, redress, and regenerate from evaluation’s historical and ongoing patterns of oppression. By coincidence (or divine intervention?), the webinar took place on May 13. So we joked that we could call our counter knowledge economy “The May 13 Group.”
Less than two weeks after the Center for Evaluation Innovation’s webinar, the world mourned the state-sponsored murder of George Floyd and watched Minneapolis rise up in protest. Many rose up as well in solidarity. This confluence of chronic structural violence and acute physical violence helped illustrate for many evaluators how racialized and gendered capitalism operates through the seemingly neutral institutions and structures whose work they routinely evaluate without ever questioning. It also illustrated the rationale underlying direct action organizing and other alternatives to the reformist reforms and individual-level interventions that most evaluators encountered through their training or work.
Underwater view from entangled water lily roots toward the sunlight. Photograph by Reto Gerber. Free for use by Pixabay
To close: It’s not just that May 13 is literally the next day after May 12. Maybe more importantly, “the next day for evaluation” reverberates with efforts increasing around the world to transition from an extractive economy to “the next economy.” The May 13 Group deepens such efforts by rooting them in a critical analysis of, and resistance against, racialized and gendered capitalism, coloniality, and imperialism and expands them by applying them to evaluation.
Representing a just transition from The May 12 Group’s legacy to a solidarity economy, “the next day for evaluation” requires that we attend to the death of reductive knowledge paradigms as well as the neoliberal political ideologies that they support and are supported by. From the decay of these extractive structures and institutions of knowledge production, we can build a nurturing foundation—as in forest ecosystems—for planting new and growing age-old ways of knowing that intertwine inseparably with feeling and doing.
We remember that evaluators from the global majority are documented for at least a century as having drawn from their oppositional experience and consciousness to organize the purposes, processes, and products of knowledge work collectively, collaboratively, and cooperatively. And so, the next day for evaluation is not an entirely new day. But The May 13 Group is transparent in being oriented toward, and energized by, epistemic healing and wholeness.
Contact us
Learn more or join in co-creating this work-in-progress