Using personal statements in college admissions: An investigation of gender bias and the effects of increased structure. Issue 1 (26th January 2022)

• Record Type: Journal Article
• Authors: Niessen, A. Susan M.; Neumann, Marvin
• Is Part Of: International journal of testing. Volume 22:Issue 1(2022)
• Page Start: 5
• Page End: 20
• Publication Date: 2022-01-26
• Subjects: College admissions -- gender -- personal statements -- reliability -- validity; Psychological tests -- Periodicals; Educational tests and measurements -- Periodicals; 150.28705
• DOI: 10.1080/15305058.2021.2019749
• Languages: English
• ISSNs: 1530-5058

Abstract

Abstract: Personal statements are among the most commonly used instruments in college admissions procedures. Yet, little research on their reliability, validity, and fairness exists. The first aim of this paper was to investigate hypotheses about adverse impact and underprediction for female applicants, which could result from lower tendencies to use agentic language compared to male applicants. Second, we examined if rating personal statements in a more structured manner would increase reliability and validity. Using personal statements (250 words) from a large cohort of applicants to an undergraduate psychology program at a Dutch University, we found no evidence for adverse impact for female applicants or more agentic language use by male applicants, and no relationship between agentic language use and personal statement ratings. In contrast, we found that personal statements of female applicants were rated slightly more positively than those of males. Exploratory analyses suggest that female applicants' better writing skills might explain this difference. A more structured approach to rating personal statements yielded higher, but still only 'moderate' inter-rater reliability, and virtually identical, negligible predictive validity for first year GPA and dropout.