A personal view on personality tests

So. New job, new personality test. An opportunity to add another 4-colour quandrant-based set of labels to my collection, which already includes HBDI, Belbin team roles, MBTI, Insights, Via Strengths Institute, and a view others lost to the mists of time.

I have a lot of mixed feelings about these personality tests. I really buy into the idea that the self-reflection and self-awareness that they’re designed to encourage are worthwhile things to pursue, especially for people who are following leadership tracks. And as a way of provoking discussions around team dynamics sounds like a great idea too.

Earlier in my life I embraced these eagerly. Desperately seeking feedback and guidance, I marvelled at the way that a 50-question multiple-choice questionnaire could be so incredibly accurate about me and scoured the reports for further details about the nature of my personality, inclinations and drivers, which so inescapably shaped the person I was both at home and in work.

But as I grew both personally and professionally, and had the chance to work with people who could help me explore my real strengths and address my real failings in a more meaningful way, I started to really understand not just the flaws but the hidden dangers in these flat narratives.

So, here I’m going to break down my issues with personality tests, and follow up with a few pointers on how managers can actively and helpfully use these personality tests. The worries I have with them are:

• They’re often presented as having scientific basis which is unjustified.
• They use “corporate astrology” to present themselves as more accurate than they are.
• They perpetuate and encourage fixed-mindset thinking.
• They rely too much on self-analysis.
• I have concerns about the lack of engagement with significant diversity and inclusivity points.

The scientific basis of personality tests.

It’s important to recognise that on the whole, these personality tests are run by for-profit enterprises and the research into them has been carried out by that enterprise. I always research the academic basis of the personality tests I take, and would recommend you do the same.

The bottom line is this: you’ll be hard-pressed to find a peer-reviewed, statistically valid piece of research that proves the accuracy or value of these tests. My biggest annoyance is the way in which language is used in a way that feels deliberately misleading; marketing material will tout the tests’ “reliability”, but fail to spell out the difference between reliability and validity. A test is reliable if it consistently gets the same result, regardless of whether the result is correct or not. Even when tests are described as “valid”, they’re often rather circumspect about what their criteria for validity are.

The most popular of these tests are based on thinking from a century ago. As a brief idea, Insights is based on MBTI (also known as “16 personalities”), which was developed and popularised between 1919 and 1945. DiSC is based on a theory from 1928. It goes without saying that our understanding of the field of psychology - which was virtually brand-new at the start of the 20th century - has moved on significantly from there, yet these personality tests do not seem to have kept pace with the rest of the field. (It's also worth mentioning that even modern psychology has a problem with proof and validity - in 2018, it was found that over 1 in 3 social and behavioural papers could not be reliably reproduced).

Corporate astrology

But why worry about validity - you can see for yourself how valid the results are and take them or leave them, surely? Even this requires a lot more caution and careful analysis.

The first time you read through a personality report, it’s easy to feel impressed with how “valid” they are. I carried out a more cautious assessment of my Insights profile and highlighted anything that I - as an experienced manager - would take as basic good practice in managing humans. Statements like “works best when given a clear goal to go after and space to pursue it in their own way” or “responds well to encouragement” or “may become frustrated when surrounded by people who are not competent in their roles”. It amounted to nearly half the report, and a great deal more was also pretty generic but could be considered as not-universal (things like “does not enjoy constantly changing environments”).

So, the overall approach is “wow this really gets me” but I think a justified cynical reading of that could be: “of course it does; you’re human”.

A common cognitive bias is confirmation bias. We are primed to read the report and see what we agree with; in doing so, we pay too much attention to what fits our expectations and too little to that which does not. It takes a lot of discipline to read through the report and to really notice and highlight those statements which you feel don’t apply to you. In the Insights report, this amounted to about ⅓ of the report, which I had almost completely missed the first read through. But, highlighted and colour-coded, it was pretty clear to see - there were a lot more misses in there than I’d realised. In fact, the number of statements which I felt both accurately described me and were non-trivially true for most or all people rapidly became a small minority of the overall report.

This is just the way that astrology works: say something vague and generally true, and allow the reader to fill in the gaps with their own assumptions.

They perpetuate and encourage fixed-mindset thinking

Ok so what, so far so harmless right? Wrong.

My insights profile would have me believe that my personality is fixed, immutable, and controls my drivers and style. This is well-intentioned, I am sure! But it’s exactly the kind of fixed mindset that Carol Dweck warns against: that our strengths and weaknesses are fixed parts of our personality, not a function of our choices and where we put effort and focus. In all fairness, the DiSC report fared a little better on that front.

Nonetheless, it’s a dimension which carries significant risk. Only once in my history of being personality-tested did it come with coaching sessions to truly explore how my values, behaviour and choices interacted with and were impacted by those elements of my personality which are more inherent. For a less experienced practitioner, it would be easy for negative self-beliefs and narratives to be reinforced and compounded by the personality report.

They rely too much on self-analysis

The personality tests I’ve done have all relied on multiple-choice questionnaires. They’ve all actively encouraged me to not over-think my answers, but to go with my instinctive response. In other words, they’re actively trying to assess my self-narrative, rather than my actual behaviours in a situation. Some people would say that those instinctive responses are a genuine insight into a person’s chosen behaviours but I couldn’t find any evidence that backs up that claim; the nearest is the debate about implicit association tests, which still attracts a lot of debate.

So, we’re left with a view of someone’s personal narrative about themselves. This is going to be coloured by environment, culture, society and upbringing. It’s certainly not going to be entirely accurate; for some, it may be helpful to reflect on this, for others, it may only reinforce the most negative or inaccurate beliefs they have of themselves.

Coaching is sometimes described as “holding up a mirror” to the coaching subject, to guide them through self-reflection. Personality tests that reflect back your own self-belief can be a helpful thing as part of a guided exercise with an experienced coach, but I’ve very rarely seen significant coaching accompany these exercises.

They do not account for DEI issues and topics.

I was reflecting on questions like “I am very outspoken with my opinions” and wondered how I should, could, possibly answer that in a way that is neutral to my lived experience of existing almost my entire educational and professional life as a gender minority, a woman who’s always had to walk the impossibly fine line between being too quiet or passive vs avoiding being labelled as brash, aggressive, pushy or bossy.

The cultural biases in supposedly-neutral tests like IQ tests and broadly-used psychological diagnostic tests are, increasingly, under scrutiny. It’s unthinkable that these tests, with far less academic backing, research and rigour, fare any better - and are likely to be significantly worse. Where testing has been done, it’s invariably within a cohort in which white, educated males are significantly over-represented. This may not seem like a problem if (as has always been my experience) the tests are also applied to groups who are likewise predominantly white, educated males as well. But when the assessments come back with messages which are subtly or not-so-subtly coded with workplace expectations, the impact on inclusive environments must be scrutinised. Even moreso when you consider the prevailing attitudes to race, gender and even eugenics when these tests were first devised.

I’ve also heard criticism of much of the question wording from autistic professionals. Virtually all of the questions are highly subjective and require interpretation - how do you define “much of the time” vs “sometimes”, or “a little” vs “a lot”? An example question is “I am always positive” with the usual 5-point scoring system (strongly disagree/disagree/neutral/agree/strongly agree) - but taken with a literal reading, nobody is always positive, so a literal interpretation would only allow disagree or strongly disagree, even if the person considers themselves generally positive.

Strongly disagree?

I love a bit of armchair psychology and I generally like the ways in which different models give us a chance to reflect on and conceptualise behaviour. In the Crossick household we have a mixture of parenting and leadership books that lean into this; My Hidden Chimp and The Curse of the Good Girl sit alongside Primal Leadership, Drive and Five Dysfunctions of a Team. Martin Seligman's Optimistic Child gives me another personality quiz to feed the kids, and I worry as much about where my kids sit on the three axes of optimism as much as I question the academic foundations of what this really means for their long-term mental health and happiness.

But what strikes me is that all my most helpful models for understanding humans don’t come from individualised personality tests, but more generic texts and models. Rather than trying to determine which leadership profile in Primal Leadership is most applicable to me, I just learnt the models and self-reflected on the bits that were most resonant. I learnt about overarching ideas of motivation, and assume that in general pretty much everyone will be a bit more engaged if they have the opportunity to work with autonomy. And I call people out in specific ways when they show pessimistic behaviours, rather than trying to work out whether they have done enough to achieve the label of “pessimist”. I perform a mental checklist against the PERMA acronym when diving into demotivation, whether it’s my own or someone else’s. And I grant people a little extra grace when their chimp has got control or someone has moved their cheese.

Ultimately my suspicion is that personality test based training courses are a relatively easy and repeatable solution to training people in “soft skills” which is compounded by a pretty hefty “wow” factor and some very sophisticated pseudo-scientific marketing material. Leadership training is hard, professional coaching is expensive, and there’s so much to learn. But the longer I’m in this leadership game, the less value I’ve found from personality tests, while also discovering more and more depth and nuance to the generic leadership models and ideas that I’ve learnt.

So, here’s my call to action: Don’t read too much into your personality tests. They are less valid and less accurate than they appear. Instead, go start learning about leadership more generally… And then just keep going, because there’s always another thing to learn.