Mixed Race Studies

Reliability Concerns in Measuring Respondent Skin Tone by Interviewer Observation

Public Opinion Quarterly
Volume 80, Issue 2 (2016)
DOI: 10.1093/poq/nfw015
pages 534-541

Lance Hannon, Professor
Department of Sociology & Criminology
Villanova University, Villanova, Pennsylvania

Robert DeFina, Professor
Department of Sociology & Criminology
Villanova University, Villanova, Pennsylvania

The current study assesses the intercoder reliability of one of the most important skin tone measurement instruments—the Massey–Martin scale. This scale is used in several high-profile social surveys, but has not yet been psychometrically evaluated. The current evaluation is only possible because, for the first time, the General Social Survey’s 2010–2014 panel used the instrument to guide interviewers’ skin tone observation of the same respondents in two different years (2012 and 2014). Despite the widespread use of the Massey–Martin scale to investigate potential effects of skin tone on social attitudes and outcomes, the data suggest that the measure has low intercoder reliability. Implications for researchers and survey practitioners are discussed.

Read or purchase the article here.

Bias in the Flesh: Skin Complexion and Stereotype Consistency in Political Campaigns

Public Opinion Quarterly
First published online: 2015-12-17
DOI: 10.1093/poq/nfv046

Solomon Messing, Director of Data Labs
Pew Research Center, Washington, D.C.

Maria Jabon, Senior Software Engineer
LinkedIn, Mountain View, California

Ethan Plaut, Postdoctoral Fellow
Stanford University, Stanford, California

Researchers manipulated the tone of President Obama’s skin to measure viewers’ stereotypes. (Courtesy of Solomon Messing / Political Communication Lab, Stanford University)

There is strong evidence linking skin complexion to negative stereotypes and adverse real-world outcomes. We extend these findings to political ad campaigns, in which skin complexion can be easily manipulated in ways that are difficult to detect. Devising a method to measure how dark a candidate appears in an image, this paper examines how complexion varied with ad content during the 2008 presidential election campaign (study 1). Findings show that darker images were more frequent in negative ads—especially those linking Obama to crime—which aired more frequently as Election Day approached. We then conduct an experiment to document how these darker images can activate stereotypes, and show that a subtle darkness manipulation is sufficient to activate the most negative stereotypes about Blacks—even when the candidate is a famous counter-stereotypical exemplar—Barack Obama (study 2). Further evidence of an evaluative penalty for darker skin comes from an observational study measuring affective responses to depictions of Obama with varying skin complexion, presented via the Affect Misattribution Procedure in the 2008 American National Election Study (study 3). This study demonstrates that darker images are used in a way that complements ad content, and shows that doing so can negatively affect how individuals evaluate candidates and think about politics.

Read the entire article here.