Evaluating Candidates and Identifying the Short List - PowerPoint Presentation

Slide1

Evaluating Candidates and Identifying the Short List

Sherri Irvin

Presidential Research Professor

of Philosophy and WGSSlide2

Screening: systematic with clear criteria

Template for each applicant with clearly defined criteria as specified in the job adRequired qualifications (first screening)

Preferred qualifications (second screening)Slide3

Obstacles to identifying excellent candidates

Implicit bias: tendency to underrate the credentials of women, candidates of color, people with disabilities, and other members of underrepresented groupsRater drift

: tendency for evaluators’ standards to shift over time, so similar credentials are rated differently Overemphasis on “fit”: tendency to discount the achievements of people whose methods, topics, or social identities are marginalized in the fieldMatthew Effect: tendency of further advantages to be heaped on those who have experienced early advantages, thereby inflating their credentialsSlide4

Recent research shows that race, gender & related factors play a major role in hiringSlide5

Experiment on race and

hiring

Bertrand, Marianne, and Sendhil Mullainathan. “Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination.” The American Economic Review

94, no. 4 (2004): 991-1013.

Resumes with

white-sounding names received 50

%

more callbacks than

those

with African-American-sounding names.Slide6

Experiment on motherhood/gender and

hiring

Correll, Shelley J., and Stephen Benard. “Getting a job: Is there a motherhood penalty?” American Journal of Sociology 112, no. 5 (2007): 1297-1339.

Resumes of mothers received significantly lower scores for competence, organizational commitment and lower salary and hiring recommendations (1.8x less).

Resumes of non-mothers received 2.1x the callbacks of mothers.Slide7

Effect of screened auditions

on success of female

musicians Goldin, Claudia, and Cecilia Rouse. Orchestrating impartiality: The impact

of “blind”

auditions on female musicians. No. w5903. National Bureau of Economic Research, 1997.

U

se

of screens during auditions accounts for 1/3 of the increase in the number of female musicians in orchestras.Slide8

Experiment on gender

and hiring in academic p

sychology

Steinpreis

, Rhea E., Katie A. Anders, and Dawn

Ritzke

.

“The

impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study

.”

Sex Roles

41, no. 7-8 (1999): 509-528.

Men received

a

positive hiring

recommendation >70% of the

time;

women received

a positive hiring recommendation only 55% of the time.

Gender of reviewer did not matter. Women and men discriminated equally

regarding

gender of candidate.Slide9

Experiment on hiring of student lab managers in university science labs

Moss-

Racusin, Corinne A., John F. Dovidio, Victoria L. Brescoll, Mark J. Graham, and Jo Handelsman. "Science faculty’s subtle gender biases favor male students." Proceedings of the National Academy of Sciences 109, no. 41 (2012): 16474-16479.

The female student was rated as less competent and less

hireable

. Faculty offered less mentoring and

proposed a 14% lower salary.

Gender and age

of reviewer did not matter. Women and men discriminated equally

regarding

gender of candidate.Slide10

Experiment on race and gender in finalist pools for academic jobs

Johnson, Stefanie K., David R.

Hekman, and Elsa T. Chan. “If there’s only one woman in your candidate pool, there’s statistically no chance she’ll be hired.” Harvard Business Review April 26, 2016.

“When there was only one woman or minority candidate in a pool of four finalists, their odds of being hired were statistically zero.

”

The odds increased dramatically with two women or two minority candidates

.Slide11

Implicit bias is exacerbated by

Evaluator factors:cognitive load (too much on your mind)

stresshurryfatigue, hunger, thirstbelief that one is unbiased (probably false)Evaluation task factors:vague criterialack of structurelack of accountabilitySlide12

StrategiesSlide13

Strategies: evaluator conditions

Improve evaluator conditionsLighten workloadProvide drinks and snacks

Make sure dossiers are easy to access and reviewSlide14

Strategies: counteracting bias

Have a member of the diversity committee (DC) available as a consultant on the searchDC member can flag dossiers with potential bias triggers for careful attention

Social identity (gender, race, LGBTQ identity, disability) where disclosed in the dossierMarginalized methods or topicsRationale: implicit bias sometimes functions by leading us not to notice achievementSlide15

Strategies: counteracting bias

Consider anonymizing

materials where feasibleThis must be done carefully (pronouns in letters, etc.)Can ask candidates to anonymize some materialsDC member can assist or verifyNot a cure-all: nature of research or organization membership sometimes suggests social identityBe aware that submitted materials may be affected by biasLetters of recommendationStudent evaluations of teachingSlide16

Strategies: accountability

Create clear criteria and well-structured ratings templates

Formalize your assessment where possible (e.g., with a points system; if you deviate, be clear about why)Justify short list by appeal to criteriaThis strategy helps with implicit bias, rater drift & overemphasis on “fit”Slide17

Getting to the final listSlide18

Initial interviews

Consider forgoing initial (long-list) interviewsInterview situation is not representative of actual job tasksSubmitted materials are most reliable source of info

Vividness of interview swamps more reliable infoTime and energy is better spent looking more carefully at candidate materialsSlide19

Initial interviews

If you decide to interview:Provide questions in advanceAsk same questions of all candidates

Provide consistent interviewing conditionsInterviewer behavior affects performanceRank-order candidates prior to interviewIf you change your ranking, be clear about whyWhat new info did you get that is relevant to criteria?Slide20

Final thoughts about the Matthew Effect

Access to opportunities is facilitated by privilege (race, gender, economic class, etc.)These opportunities tend to snowball

careful mentorship in undergrad  top-ranked graduate school  invitation to co-author  cushy post-doc  three publicationsSlide21

Experiment on which student queries receive responses

Milkman, Katherine L.,

Modupe Akinola, and Dolly Chugh. "What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations." Journal of Applied Psychology 100, no. 6 (2015): 1678.

Queries from women and students of color are far more likely to be ignored, and requests for meetings more likely denied. The effect is worse in disciplines where pay is higher.

Discrimination was just as bad in fields with higher ratios of women and faculty of color.Slide22

Final thoughts about the Matthew Effect

As you design criteria, look for ways of assessing merit that do not simply codify the results of privilegeOne publication that challenges traditional assumptions, published during a 4/4 teaching load, v. three conventional publications published during a teaching-free post-doc…