Donna M Mertens Keynote Independent Consultant Minnesota Evaluation Studies Institute MESI Spring Training Minneapolis MN March 2015 Why this topic Why now Social Justice Amidst Standards and ID: 163723
Download Presentation The PPT/PDF document "Social Justice Amidst Standards and Acco..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Social Justice Amidst Standards and Accountability
Donna M. Mertens, Keynote
Independent Consultant
Minnesota Evaluation Studies Institute, MESI Spring Training
Minneapolis MN
March 2015Slide2
Why this topic? Why now?
Social Justice
Amidst
Standards and Accountability
3/11/2015
Mertens Keynote MN MESI March 2015
2Slide3
2015 International Year of Evaluation
EvalPartners, UNICEF
, UN Women, and the International Organization for Cooperation in
EvaluationPeer 2 Peer Initiative – gender and equity focused evaluation
3/11/2015
Mertens Keynote MN MESI March 2015
3Slide4
Need for Social Justice Link to
Evaluation:
Pervasiveness of racism and other isms
3/11/2015
Mertens Keynote MN MESI March 2015
4Slide5
Need for a Social Justice Lens
Marie Battiste (2000):
“…society is sorely in need of what Aboriginal knowledge has to offer”
Chilisa (2005)
“it is an issue of life and death”
Brookes (2006)
Failure to include racism as a potential contributor to disparities
3/11/2015
Mertens Keynote MN MESI March 2015
5Slide6
My hypothesis:
If we begin by prioritizing social justice and human rights
And we appropriately involve community members in the evaluation process,
Then we will increase the probability of social transformation as a result of our evaluation.
3/11/2015
Mertens Keynote MN MESI March 2015
6Slide7
What does our evidence lack in credibility from a social justice perspective?
3/11/2015
Mertens Keynote MN MESI March 2015
7Slide8
Social
Justice
Standards
Accountability
3/11/2015
Mertens Keynote MN MESI March 2015
8Slide9
Accountability Question
Did “it” work?
Should we keep spending our money on “it”?
3/11/2015
Mertens Keynote MN MESI March 2015
9Slide10
Social Justice Questions
Was “it” the right thing?
Was “it” chosen and/or developed and implemented in culturally responsive ways?
Were contextual issues of culture, race/ethnicity, gender, disability, deafness, religion, language, immigrant or refugee status, age or other dimensions of diversity used as a basis for discrimination and oppression addressed?
How were issues of power addressed?Do we want to continue to spend money on things that don’t work?
3/11/2015
Mertens Keynote MN MESI March 2015
10Slide11
Resources for Evaluators
AEA Guiding Principles
AEA Public Statement on Cultural Competency
Joint Committee Program Evaluation StandardsEvaluators who have dedicated their lives towards furthering social justice
Communities who want social justice
3/11/2015
Mertens Keynote MN MESI March 2015
11Slide12
Good news; bad news for social justice in evaluation
1.2% (n=22) in Harner’s (2014) survey of evaluators identified their theoretical frame as Social Justice
69% (n=819) of 1,187 evaluators either strongly or somewhat agreed with this statement:
Evaluation should
focus on bringing about social justice.
3/11/2015
Mertens Keynote MN MESI March 2015
12Slide13
AEA Guiding Principles (2004): Systematic Inquiry, Competence, Integrity/Honesty, Respect for People, and Responsibilities for General and Public Welfare
To
ensure recognition, accurate interpretation and respect for diversity, evaluators should ensure that the members of the evaluation team collectively demonstrate cultural competence. Cultural competence would be reflected in evaluators seeking awareness of their own culturally-based assumptions, their understanding of the worldviews of culturally-different participants and stakeholders in the evaluation, and the use of appropriate evaluation strategies and skills in working with culturally different groups. Diversity may be in terms of race, ethnicity, gender, religion, socio-economics, or other factors pertinent to the evaluation
context. (Competence B2)
3/11/2015
Mertens Keynote MN MESI March 2015
13Slide14
AEA Guiding Principle: Cultural CompetenceLinkage with Transformative
Axiology and Epistemology
To
ensure recognition,
accurate interpretation, and respect
for diversity, evaluators should ensure that the members of
the evaluation
team collectively
demonstrate
cultural competence (American Evaluation Association, Public Statement on Cultural Competence in Evaluation, 2011)
3/11/2015
Mertens Keynote MN MESI March 2015
14Slide15
What is cultural competence?
Cultural
competence is not a state
at which one arrives; rather, it is a
process of learning, unlearning, and relearning. It is
a sensibility cultivated throughout a lifetime. Cultural
competence
requires awareness
of self, reflection on one’s
own cultural
position, awareness of others
’ positions
, and the ability to
interact genuinely
and respectfully with others.
Culturally competent evaluators refrain from assuming they fully understand the perspectives of stakeholders whose backgrounds differ from their own.Cultural competence is context dependent. (AEA 2011)3/11/2015Mertens Keynote MN MESI March 201515Slide16
Evaluation and Validity and Cultural Competence
accurately and respectfully reflect the life experiences and perspectives of program participants in their evaluations.
establish relationships that support trustworthy communication among all participants in the evaluation process.
draw upon culturally relevant, and in some cases culturally specific, theory in the design of the evaluation and the interpretation of findings.
select and implement design options and measurement strategies in ways that are compatible with the cultural context of the study. (AEA 2004, p. 6)
3/11/2015
Mertens Keynote MN MESI March 2015
16Slide17
Recognize the dynamics of power
Cultural groupings are ascribed differential status and power, with some holding privilege that they may not be aware of and some being relegated to the status of “other.” For example, language dialect and accent can be used to determine the status, privilege, and access to resources of groups. (AEA, 2004, p. 7)
3/11/2015
Mertens Keynote MN MESI March 2015
17Slide18
Standards for Good Evaluation (Yarborough et al. 2011)
Utility: evaluator credibility, attention to stakeholders, negotiated purposes, explicit purposes
Feasibility: management, practical, contextual viability (recognize, monitor, and balance the cultural and political interests and needs of individuals and groups)
Propriety: responsive and inclusive, formal agreements, human rights and respect, clarity and fairness, transparency and disclosure.
Accuracy: valid, reliable, explicit program and context descriptions, sound designs and analysis
Evaluation accountability: meta-evaluation
3/11/2015
Mertens Keynote MN MESI March 2015
18Slide19
Accountability and Social Justice
Code word for RCT?
Accountable to whom?
Ways to strengthen arguments about accountability are not limited to RCTsTheoretical streams in evaluation provide options for demonstrating accountability
3/11/2015
Mertens Keynote MN MESI March 2015
19Slide20
Evaluation Theory and Social Justice – Alkin’s Tree: Methods, Use & Values Branches
Mertens Keynote MN MESI March 2015
20
Values
Methods
Use
3/11/2015Slide21
Mertens & Wilson 2012: Methods, Use, Values, & Social Justice Branches
Mertens Keynote MN MESI March 2015
21
Methods
Use
Values
Social Justice
3/11/2015Slide22
Metaphor: Tree or Water?
Mertens Keynote MN MESI March 2015
22
3/11/2015Slide23
Paradigms & Branches
Paradigm
Branch
Post-positivist
Methods
Constructivist
Values
Transformative
Social Justice
Pragmatic
Use
Mertens Keynote MN MESI March 2015
23
3/11/2015Slide24
Transformative Theories
Feminist theories
Critical Race Theories
Critical Theories
Human Rights Theories
Disability Rights Theories
Transformative Participatory Action theories
Indigenous theories
Deafness rights theories
Queer theories
3/11/2015
Mertens Keynote MN MESI March 2015
24Slide25
Transformative Paradigm
Philosophical
Assumptions
Axiology
Respect for cultural norms; support for human rights and social justice; reciprocity
Ontology
Issues of power & critical interrogation of multiple realities: social, political, cultural, economic, race/ethnic, gender, age, religion and disability values to unmask those that sustain an oppressive status quo
Epistemology
Issues of power & Interactive link; knowledge is socially and historically located; trusting relationship.
Methodology
Qualitative (dialogic)/ Quantitative / Mixed Methods; Context
Mertens Keynote MN MESI March 2015
25
3/11/2015Slide26
Court Access Project
Start from community
Advisory board formed
Begin understandings from the vantage point of the “least privileged”
Work with an awareness of diversity in the community
Provide respectful support for engagementUse a cyclical approach to data collection and use
3/11/2015
Mertens Keynote MN MESI March 2015
26Slide27
Example: HIV/AIDS Prevention in Botswana
Botswana youth:
addressing power
inequities in the
fight against
HIV/AIDS using a
transformative lens
Mertens Keynote MN MESI March 2015
27
3/11/2015Slide28
Transformative Cyclical Mixed Methods Design
Mertens Keynote MN MESI March 2015
28
Post
tests:
Quant
Qual;
Behavior
& Policy
Change;
Transfer
To other
contexts
RCT
3/11/2015Slide29
Indigenous Contributions
Social
justice as a frame for
evaluationEngagement with indigenous communities
3/11/2015
Mertens Keynote MN MESI March 2015
29Slide30
Indigenous Paradigm
Chilisa (2012)
Wilson (2008)
Linda T. Smith (1999)Marie Battiste (2000)
LaFrance & Crazy Horse (2009)
Relational AxiologyRelational OntologyRelational Epistemology
Relational Methodology
(Chilisa, 2012)
3/11/2015
Mertens Keynote MN MESI March 2015
30Slide31
Ethics and Indigeneity
Connections between people, past, present, and future, all living and nonliving things (ubuntu)
Respect and reciprocity: listens, pays attention, acknowledged, and crates space for the voices and knowledge systems of Indigenous people
Contribute to a better futureSpirituality
3/11/2015
Mertens Keynote MN MESI March 2015
31Slide32
Questions
What is credible evidence?
What criteria establish credibility of evidence?
What does our evidence lack in credibility?
What sources give us insight into credibility?What is the place of social justice in credibility?How do voices of marginalized communities enhance our understandings of credibility?
Why is it important for researchers to understand marginalized communities’ perspectives of social justice?
Mertens Keynote MN MESI March 2015
3/11/2015
32Slide33
Methodological Questions
What data collection methods can the evaluator use to become better acquainted with the various stakeholder groups?
How can the evaluator design the study so that the community members are included in decision making?
How could the use of mixed methods contribute to the quality of the study?
How can the evaluator design the methods to increase the probability of furthering social justice?
3/11/2015
Mertens Keynote MN MESI March 2015
33Slide34
Water as Metaphor
Eventually, all things merge into one, and a river runs through it. The river was cut by the world's great flood and runs over rocks from the basement of time. -
Norman Fitzroy Maclean
When you put your hand in a flowing stream, you touch the last that has gone before and the first of what is still to come. - Leonardo da Vinci
My soul is full of longing For the secret of the Sea, And the heart of the great ocean Sends a thrilling pulse through me – Henry Wadsworth Longfellow
http://www.finestquotes.com/select_quote-category-Water-page-1.htm#ixzz2968zeXR9
Mertens Keynote MN MESI March 2015
34
3/11/2015Slide35
Resources
Mertens, D. M. & Wilson, A. (2012). Program Evaluation Theory and Practice: A Comprehensive Guide. NY: Guilford
.
Mertens, D. M. (
2015).
Research and evaluation in education and psychology: Integrating diversity with qual, quant and mixed methods. 4th ed. Thousand Oaks, CA: Sage
.
Mertens, D. M. (2009).
Transformative research & evaluation
. NY:
Guilford.
Mertens, D. M. & Ginsberg, P. (2009).(Eds.) Handbook of Social Research Ethics. Thousand Oaks, CA: Sage.
American Evaluation Association (2011). Public Statement on Cultural Competence in Evaluation. AEA.
Mertens Keynote MN MESI March 2015
35
3/11/2015Slide36
Contact information
Donna M.
Mertens, PhD
Independent Consultant
Donna.Mertens@Gallaudet.edu
Mertens Keynote MN MESI March 2015
36
3/11/2015