Intermittent Faking of Personality Profiles in High-Stakes Assessments

What is it about?

In high stakes assessments of personality and similar attributes, test takers may engage in impression management (aka faking). We propose to consider responses of every test taker as a potential mixture of ‘real’ (or retrieved) answers to questions, and ‘ideal’ answers intended to create a desired impression, with each type of response characterized by its own distribution and factor structure. Depending on the particular mix of response types in the test taker profile, grades of membership in the ‘real’ and ‘ideal’ profiles are defined. This approach overcomes the limitation of existing psychometric models that assume faking behavior to be consistent across test items.

Featured Image

Why is it important?

Personality and similar attributes are almost exclusively assessed using self-reports. When assessments are taken in a high-stakes situation, for example as part of a job application process, applicants may be motivated to manipulate their responses to create the best impression (aka ‘fake good’). If not accounted for in a data analysis, these manipulations are a threat to validity. In this paper, we argue that to provide such a data-analytic control in real-life settings, we need to overcome the limiting assumption that faking behavior is consistent across test items. We propose considering ‘intermittent’ faking, whereby the same test taker may present the ‘real’ self on some questions while presenting an ‘ideal’ candidate on other questions. Depending on the particular mix of these two response types, grades of membership in the ‘real’ and ‘ideal’ profiles are defined for each test taker. We formalize the proposed response mechanism as a Faking-as-Grade-of-Membership (F-GoM) model, which can be implemented with standard software packages. For each test taker, the approach estimates (1) the true scores underpinning the ‘real’ profile, (2) the tendency to select desirable responses underpinning the ‘ideal’ profile, and (3) the grade of membership in one or the other profile. The benefits of a F-GoM analysis are demonstrated using operational recruitment data consisting of scale scores on a comprehensive personality questionnaire. High convergent correlations with objective measures indicate that the approach is effective in identifying responses as ‘real’ or ‘ideal’.