Competing explanations for inconsistent responding to a mixed-worded self-esteem scale: cognitive abilities or personality?

Abstract

In survey scale design, a mixed-worded format intends to ensure attentiveness as respondents need to take into account the wording direction when answering an item. However, some respondents tend to deliver inconsistent responses (i.e., agreeing or disagreeing with both positively and negatively-worded items), posing a validity concern. Two potential directions driving inconsistent responding have each individually been put forward: lack of cognitive abilities to effectively deal with the mixed-wording and sheer carelessness while responding. Using a factor mixture approach, we investigated inconsistent responding on Rosenberg’s self-esteem scale as a function of both cognitive ability and personality. Among n = 4938 Grade 5 students from the German National Educational Panel Study (NEPS), 11 % were classified as inconsistent respondents and class memberships were further related to four cognitive abilities (cognitive reasoning, cognitive speed, reading comprehension, reading speed) and five personality traits (conscientiousness, neuroticism, extraversion, agreeableness, openness). Model comparison results indicated that both ability and personality predictors matter with a more prominent role for ability (especially reading comprehension). We discuss the implications of these findings for mixed-wording’s suitability for scale construction and different populations and how researchers can deal with suspected inconsistent respondents in their survey data.