As the public focus around diversity and inclusion grows, organizations become increasingly interested in measuring their D&I efforts to determine whether these are actually effecting change towards improved diversity, equity, and inclusion. Several articles have explored different reasons as to why diversity and inclusion programs may fail. But what about evaluations? Evaluations can provide essential data about the progress of your diversity and inclusion program. They are also a valuable tool that will inform any adjustments, should you need to course correct. Yet bias often shows up in evaluations, and when it comes to evaluating racial equity, this unaddressed bias can jeopardize the success of your efforts of improving diversity, equity, and inclusion in the workplace.

This month, YW Boston’s Vice President of Programs Kemarah Sika facilitated a roundtable discussion on outcomes measurement related to racial equity in collaboration with the Greater Boston Evaluation Network. During the presentation, titled “Racial Equity and Evaluation,” Kemarah shared insights on how YW Boston’s Dialogues on Race and Ethnicity program approaches evaluations when partnering with organizations looking to advance diversity, equity, and inclusion. At YW Boston, we have focused on designing processes and developing evaluation tools that allow us to effectively measure individual and organizational readiness for D&I work, as well as tracking progress and areas of improvement. We worked with the Northeastern University Public Evaluation Lab (NU-PEL) for six months to broaden our evaluation tools and processes in order to help organizations looking for improved results.

Below are a few takeaways from “Racial Equity and Evaluation” that can help you identify and disrupt bias in evaluations.

1. Acknowledge that bias is always present

The first steps towards addressing any challenge involve fully acknowledging the problem at hand and exploring possible root causes. People are biased, and as biased individuals, we can reproduce our biases in everything from self-assessment and decision making, to the tools and technology that we use. Most importantly, bias extends to third-party tools and secondary data. Prejudice and racism are institutionalized, so it is important to recognize that bias will be present within data—such as demographic data—and processes, regardless of where and how data was collected. When examining evaluations for bias, try pushing beyond quantitative data as it does not always tell the full story. Your quantitative and qualitative data may be communicating two different stories, so it’s important to gather more perspectives in order to gain deeper meaning. One common hurdle in evaluations is the inability to disaggregate. Disaggregation is critical to identifying singular data on specific identities such as race, gender identity, class, abilities, as well as intersections of identities.

2. Examine what is being measured

In the words of Marc Miringoff, “we measure what we value.” Our values are shaped by a number of internal and external aspects. Our environments, experiences, and institutions will impact our ideas about what is important and what should be measured. Therefore, institutionalized bias, prejudice, and racism will impact choices about the data we gather, the findings we prioritize and the meaning we ascribe to them. One way to mitigate racism within evaluations is to examine who is performing them. Evaluators have a lot of power, so it’s important to ask questions early on. Are your evaluators diverse? Do they have an understanding of racism, power, and privilege? What data collection methods are being prioritized? Is your default indicator a white male?

3. Consider who determines outcomes

It’s equally important to consider who’s involved in the post-evaluation process. After assessments are complete, someone will have to interpret the data and decide how to move forward. Kemarah refers to the work of Sally Leiderman who suggests shifting power dynamics from institutions and “experts” to communities and individuals most affected by the research. Kemarah suggests that one of the most important questions evaluators can ask themselves is, “Who is not included?” Explore what success means and who is defining it. Define your theory of change and what it would take to achieve your outcomes. Identify your timetables and gather input from those who will be involved in making them happen. Additionally, think about what a negative or positive outcome might mean and who will frame those results.

Kemarah emphasized the importance of preserving an equity lens throughout, even after evaluations have concluded. Think about how you’ll be sharing your results and be intentional when it comes to making sure participants know what is being measured and why. This is an important component of our Dialogues on Race and Ethnicity program’s post-evaluation process and action planning, as we prepare to support our partners’ journey to organizational change. Click here to learn more about Dialogues’ evidence-based approach to helping organizations build more inclusive environments.

______

About YW Boston

As the first YWCA in the nation, YW Boston has been at the forefront of advancing equity for over 150 years. Through our three programs—Dialogues, LeadBoston, and our youth programming—as well as our advocacy work, we help individuals and organizations create more inclusive environments where women, people of color, and especially women of color can succeed.