The Real Criticisms of the Big Five Personality Test

The Big Five model is the personality framework with the strongest empirical backing in psychology today. That doesn’t spare it from criticism, and some of the criticism is serious. Knowing it doesn’t invalidate the model, but it does help you read any result with better judgment.

The objections fall into five areas: validity outside Western cultures, the lack of an explanatory theory, the number of factors, measurement (short versions and faking), and the interpretation of extreme scores.

This article is for information only and is no substitute for an assessment by a qualified professional.

Cultural bias toward Western, industrialized populations

Much of psychology has been built on samples that are Western, educated, industrialized, rich, and democratic, which Henrich, Heine, and Norenzayan (2010) summed up with the acronym WEIRD. The Big Five is no exception. The fact that its structure appears in dozens of countries doesn’t prove it is universal if those countries share university students, schooling, and familiarity with questionnaires.

The Tsimane case

The most cited evidence is the study by Gurven and colleagues (2013) with the Tsimane, a horticulturalist and forager people of the Bolivian Amazon. With a large sample and both self- and peer ratings, the five-factor structure did not replicate. Two main factors emerged instead: one of prosociality, the tendency to cooperate, and one of industriousness, the disposition to work. The authors suggest that in small societies where life depends on a few roles, traits organize around what those tasks demand.

The problem of measuring far from home

Laajaj and colleagues (2019) went a step further with data from 94,751 people in 23 low- and middle-income countries, surveyed face to face. The usual Big Five items did a poor job of measuring the traits they were meant to measure: low validity and structures that looked nothing like the expected one. By contrast, in online surveys of self-selected respondents from those same countries, validity was much higher.

Among the causes they point to: acquiescence (the tendency to say “yes” to almost everything, more common with less schooling), the influence of the interviewer, and limited familiarity with this kind of question. So the criticism isn’t only that traits differ in other cultures; it’s that sometimes the instrument fails to measure them.

Openness, the least stable factor

Of the five, openness to experience is the one that travels worst between languages. In some European lexical studies the fifth factor shows up as intellect, in others as imagination, and in others as unconventionality. That variation in content explains why it is sometimes called Intellect and sometimes Openness, a debate we cover in whether the Big Five and the Five-Factor Model are the same thing.

The lack of an explanatory theory

The classic theoretical objection was made by Jack Block (1995) and Dan McAdams (1992). Block argued that the Five-Factor Model has methodological and conceptual problems and that, strictly speaking, it is not a theory. McAdams accepted that it is useful for organizing trait descriptions, but pointed out that it says little about the causes of personality, about how it is organized within each individual, and about how it develops over a lifetime.

The root of the problem is the method. The model came out of the lexical hypothesis: researchers analyzed which words in a language tend to be applied together to the same people. That yields a good classification, but a classification explains nothing on its own. The fact that “orderly” and “punctual” go together doesn’t say why.

The criticism is still useful, though it is no longer as accurate as it was in the nineties. Costa and McCrae replied to Block that same year, and since then explanatory theories have been built on the model, such as McCrae and Costa’s Five-Factor Theory and DeYoung’s Cybernetic Big Five Theory (2015), which we review in the Big Five personality trait theory. Neither is settled, but it is no longer correct to say the Big Five lacks any attempt at explanation.

Five factors or six? The HEXACO model

Another line of criticism doesn’t question the method but the result. Michael Ashton and Kibeom Lee, analyzing lexical studies in several languages, often found six factors, not five. Their HEXACO model (Ashton and Lee, 2007) includes:

• Honesty-Humility, a new factor: sincerity, fairness, modesty, and greed avoidance.
• Emotionality, similar to neuroticism but with more weight on fear and sentimentality, and without irritability.
• Extraversion, Agreeableness (which absorbs irritability and patience), Conscientiousness, and Openness.

The difference has practical consequences. Honesty-Humility predicts behaviors such as manipulation, exploiting others, or counterproductive behavior at work better than any of the five classic factors, which capture it only partially through agreeableness.

Other authors, from the opposite direction, argue that five factors are too few to capture relevant nuances and that the useful information lies in the facets, the thirty finer-grained components measured by the NEO-PI-R. It’s a debate similar to the one around other personality models, each with its own limits.

Limits of the measurement tools

Short versions

Many online tests, including the quick mode of ours, use very few items per trait. The methodological objections to these versions are well known:

• Lower reliability than full inventories, which use many more questions per trait.
• No information about facets: with two items you can’t distinguish, for example, assertiveness from excitement seeking within extraversion.
• Weaker control of response biases, such as acquiescence, when there aren’t enough reverse-keyed items to offset it.
• Less precise estimates of the latent trait, which isn’t measured directly and has to be inferred from several answers.

Faking in hiring contexts

Self-report questionnaires are transparent: it’s easy to guess which answer looks better. In a hiring process, that matters. The meta-analysis by Birkeland and colleagues (2006), covering 33 studies, compared job applicants with people answering with nothing at stake. Applicants scored higher on conscientiousness (d = 0.45) and emotional stability (d = 0.44), and considerably less so on extraversion and openness. In some jobs, such as sales, the pattern shifted toward the traits most valued in that role. That’s why professional inventories include social desirability or distortion scales, and why a profile obtained during hiring is interpreted more cautiously than one obtained in research.

More isn’t always better: the extremes

A naive reading of the Big Five assumes that for “desirable” traits, more is always better. The data don’t fully support that. Le and colleagues (2011) found curvilinear relationships between some traits and job performance: for conscientiousness and emotional stability, performance rose with the trait up to a point and then stopped improving or declined, especially in low-complexity jobs.

A concrete example: someone with very high conscientiousness may spend so long reviewing a report that they hand it in late, or struggle when the job calls for improvisation. That isn’t the same as perfectionism, which is a distinct construct with its own research, but it illustrates the idea that each pole has costs and benefits depending on the context.

With these limits in mind, you can take our Big Five test and draw your own conclusions about what your result reflects.

Sources

• Ashton, M. C. and Lee, K. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11(2), 150-166.
• Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T. and Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317-335. doi:10.1111/j.1468-2389.2006.00354.x
• Block, J. (1995). A contrarian view of the five-factor approach to personality description. Psychological Bulletin, 117(2), 187-215.
• DeYoung, C. G. (2015). Cybernetic Big Five Theory. Journal of Research in Personality, 56, 33-58.
• Gurven, M., von Rueden, C., Massenkoff, M., Kaplan, H. and Lero Vie, M. (2013). How universal is the Big Five? Testing the five-factor model of personality variation among forager-farmers in the Bolivian Amazon. Journal of Personality and Social Psychology, 104(2), 354-370.
• Henrich, J., Heine, S. J. and Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61-83.
• Laajaj, R., Macours, K., Pinzon Hernandez, D. A., Arias, O., Gosling, S. D., Potter, J., Rubio-Codina, M. and Vakis, R. (2019). Challenges to capture the big five personality traits in non-WEIRD populations. Science Advances, 5(7), eaaw5226. doi:10.1126/sciadv.aaw5226
• Le, H., Oh, I.-S., Robbins, S. B., Ilies, R., Holland, E. and Westrick, P. (2011). Too much of a good thing: Curvilinear relationships between personality traits and job performance. Journal of Applied Psychology, 96(1), 113-133.
• McAdams, D. P. (1992). The five-factor model in personality: A critical appraisal. Journal of Personality, 60(2), 329-361.