Paul Sackett, Ph.D., chair of HumRRO’s Board of Trustees and the Beverly and Richard Fink Distinguished Professor of Psychology and Liberal Arts at the University of Minnesota, and Jeffrey Dahlke, Ph.D., research scientist at HumRRO, were part of the international research team that won this year’s Jeanneret Award for Excellence in the Study of Individual or Group Assessment.
The Jeanneret Award, sponsored by the Society for Industrial and Organizational Psychology (SIOP), recognizes “the best refereed journal article or other publication that furthers public and professional understanding of individual or group assessment in the field of I-O psychology.” The winning team also included lead author Filip Lievens, Ph.D., from Singapore Management University, Janneke Oostrom, Ph.D., from Vrije University Amsterdam, and Britt De Soete, Ph.D., from Ghent University.
Along with a focus on assessment excellence in general, a key goal of the Jeanneret Award is to recognize assessments that foster a diverse workforce. Consistent with this aim, the team’s research takes an important step toward addressing one of the most complex and persistent challenges faced by assessment practitioners: reducing subgroup score differences when developing valid high-stakes hiring and promotion tests.
Published in the Journal of Applied Psychology, “Constructed Response Formats and Their Effects on Minority—Majority Differences and Validity,” focuses on the impact of situational judgment test (SJT) response option formats on subgroup mean score differences between native and immigrant individuals in Belgium. In their first study examining job seekers, the paper’s authors found that written constructed responses were able to reduce subgroup score differences relative to multiple-choice responses.
In their second study examining job incumbents, the authors found that audiovisual constructed responses captured with a webcam produced even smaller subgroup differences than written constructed responses, without impacting the criterion-related validity of SJT scores. As their samples included immigrants and actual job seekers, the work also contributes to a broader effort to increase the realism and diversity of participants in psychological research.
“Beyond the attention psychologists already devote to developing rich SJT stimuli,” Dahlke noted, “this research demonstrates that thoughtful efforts to capture and score the rich information in examinees’ open-ended responses could help us to better balance the twin goals of enhancing diversity in organizations and maintaining a high-performing workforce.”
Because SJTs are grounded in contextually rich, realistic workplace scenarios, they are often an ideal choice for organizations seeking valid assessments that applicants also tend to find very engaging and job relevant. The research acknowledged by the Jeanneret Award complements work conducted by HumRRO staff that also has led to practical refinements to the basic SJT methodology. We have distilled this expertise, honed from decades of experience developing SJTs for a wide array of public and private sector clients, into a set of best practice recommendations. Though SJTs tend to be implemented primarily in talent selection contexts, we also have created a roadmap detailing how SJTs could be fruitfully applied in the education arena.
