Individual differences at play: An investigation into measuring Big Five personality facets with game‐based assessments. (5th December 2021)

• Record Type: Journal Article
• Authors: Wu, Felix Y.; Mulfinger, Evan; Alexander, Leo; Sinclair, Andrea L.; McCloy, Rodney A.; Oswald, Frederick L.
• Is Part Of: International journal of selection and assessment. Volume 30:Number 1(2022)
• Page Start: 62
• Page End: 81
• Publication Date: 2021-12-05
• Subjects: big data -- cognitve ability -- conscientiousness -- game‐based assessment -- machine learning; Employee selection -- Periodicals; Personality and occupation -- Periodicals; Employment interviewing -- Periodicals; Prediction of occupational success -- Periodicals; Employees -- Rating of -- Periodicals; 658.3112
• DOI: 10.1111/ijsa.12360
• Languages: English
• ISSNs: 0965-075X

Abstract

Abstract: Using game‐based assessments (GBAs) to assess and select job applicants presents the dual challenges of measuring intended job‐relevant constructs while analyzing GBA data that contain more predictors than observations. Exploring those challenges, we analyzed two GBAs that were designed to measure conscientiousness facets (i.e., achievement striving, self‐discipline, and cautiousness). Scores on traditional measures of personality and cognitive ability were modeled using either a restricted set of GBA predictors using cross‐validated ordinary least squares (OLS) regression or by the fuller set ( p = 248) using random forests regression. Overall, the prediction of personality was near‐zero; but the latter approach explained 14%–30% of the variance in predicting cognitive ability. Our findings warn of GBAs potentially measuring unintended constructs rather than their intended constructs. Practitioner points: Game‐based assessment (GBA) vendors often claim to measure individual differences, but related GBA research is often lacking. The current study developed two GBAs intended to measure conscientiousness and its facets. Results demonstrated that our GBAs were measuring cognitive ability instead of conscientiousness, despite deliberate design attempts to avoid the former. Results suggested the usefulness of machine learning to understand GBA data and the constructs underlying them.