Anemone's Research Emporium

The Empathy Quotient (EQ) - a critique

Anemone Cerridwen
July 20, 2008.

Baron-Cohen and Wheelwright (2004) have created their own empathy scale for measuring empathy in autistic people, rather than use a previously constructed scale. I am not familiar with most of the scales they reject, but I do object to their reasons for rejecting the Interpersonal Reactivity Index (IRI - Davis, 1980). The IRI consists of four subscales, each with its own score (there is no overall score): the Fantasy scale (FS - imagining yourself in fictional situations), Perspective Taking scale (PT - an intellectual approach to empathy), Empathic Concern (EC - an emotional approach to empathy) and Personal Distress (PD - anxiety that might get in the way of behaving empathically). Baron-Cohen and Wheelwright do not like the Fantasy and Personal Distress scales, but do not indicate any problems with the Perspective Taking and Empathic Concern scales. Given that the four scales are scored separately, Baron-Cohen and Wheelwright could simply have ignored the FS and PD scores, and used the PT and EC scores in their research. Instead they have chosen to construct their own scale.

There is nothing wrong with constructing your own scale if you end up producing a superior scale. But Baron-Cohen and Wheelwright have not done that.

The IRI is a well constructed instrument. Davis came up with over 50 items, submitted them to 201 male and 251 female subjects, and did a factor analysis which indicated four main concepts (which led to the four scales of the IRI). He then eliminated some items and added others to produce a better version of a four-factor instrument, submitted these 45 items to 221 male and 206 female subjects, ran a second factor analysis, then chose the best 7 items (based on this analysis) for each scale, submitted them to 579 new male and 582 new female subjects, and did a third factor analysis to ensure the instrument still measured four clearly separate concepts, which it did. In a separate paper (Davis, 1983), he then tested the external validity of the instrument by correlating scores on the different scales with other, related, scales, to see if they correlated the way they should. They did. All in all, the IRI is a well-designed scale designed to measure empathy as defined in a variety of ways in general populations. Since it was not intended to test any specific hypotheses, it is simply a general measure of empathy.

Baron-Cohen and Wheelwright, on the other hand, were not as rigorous. First, they generated a list of 60 items, including 20 filler items and 40 test items. They did not try to keep emotional (affective) and intellectual (cognitive) aspects of empathy separately, because they did not think they could do so.

Baron-Cohen and Wheelwright had four aims:

"to test whether adults with high-functioning autism (HFA) or Asperger Syndrome (AS) score lower on the EQ (study 1); to test whether the EQ is inversely correlated with the AQ (Autism Spectrum Quotient) . . . , as would be predicted if autism/AS is an empathy disorder (study 1) . . . ; to test whether the EQ inversely correlates with the FQ (Friendship Questionnaire . . . ) as an index of the validity of the EQ (study 1) . . . ; and to test for sex differences in empathy, given earlier reports of a female superiority (. . . study 2)." [Baron-Cohen and Wheelwright, 2004; p.166]

Note that the authors did not include the goal of first testing to see if the EQ measures empathy. Testing it against the Friendship Questionnaire (FQ - a scale that also has its own problems) is insufficient, especially since the FQ was created by the same people and likely reflects the same biases. Testing it against the AQ is problematic, since it seems the purpose of creating the EQ in the first place is to test if autistic people score low on empathy. Anything that correlates negatively with the AQ will fit that bill, without necessarily revealing anything relevant about either autistic people or empathy.

Basically, it looks as though the authors are looking for a scale that autistic people will score lower on, rather than looking to see if autistic people score lower on an empathy scale. The difference is critical.

Baron-Cohen intially tested the EQ on twenty control subjects, but did not do a statistical analysis of the results. They were just checking to see if the test items were understandable and that the scale gave a good range of scores. They also showed the test items to six experimental psychologists who were asked to rate the test items against their defintion of empathy: "Empathy is the drive or ability to attribute mental states to another person/animal, and entails an appropriate affective response in the observer to the other person's mental state." [Baron-Cohen and Wheelwright, 2004, p.168]. All the test items passed this subjective measure of construct validity by at least five of the six judges. This is not a substitute for statistical analysis, since it is hard to know what a question is measuring until you actually see what it is measuring.

They then gave the EQ to 90 autistic adults (65 males, 25 females) and 90 controls matched for age and sex. They also gave the 90 autistic adults the AQ, and gave the FQ to 45 of them. As predicted, the autistic subjects scored quite a bit lower on the EQ than the non-autistic controls. Also as predicted, the EQ scores correlated negatively with AQ scores and positively with FQ scores. If you find what you are looking for this easily, how do you know you were searching objectively in the first place?

A quick glance at the test items indicates that at least 12 of them measure social/conversation skills (test items 1, 8, 14, 19, 26, 29, 35, 41, 44, 54, 57 and 58, e.g. "35. I don't tend to find social situations confusing."). Given that autistic people generally have a hard time following conversations, isn't it a little unfair to include conversation skills in a questionnaire designed to measure empathy in autistic people?

"Our clinical interviews with a series of adults with AS (n=50) provided an opportunity to probe the reasons for their lower score on the EQ. They reported that even though they have difficulty judging/ explaining/ anticipating or interpreting another's behavior, it is not the case that they want to hurt another person." [Baron-Cohen and Wheelwright, 2004, p.169]

Only among autistic people is difficulty interpreting interpersonal situations defined as lack of empathy. Most people would see the two as two distinct things. Consider the following model:

1. Sensory information goes into the brain. (Individual sees, hears, smells, etc.)
2. The unconscious part of the brain interprets the sensory information so that it makes some sort of sense. ("Oh, that's Jenny. She's crying.")
3. The individual reacts on an instinctive/emotional/intellectual level to this information. This reaction may or may not be high in empathy, for a variety of reasons.
4. The individual then decides how to act.
5. The individual then acts, with varying degrees of competence.

The process can break down in any of these five stages. Only the third one involves empathy. The first two involve sensory perception and interpretation, and may be hampered by sensory or perceptual difficulties. The final two involve competence in the world, which is experience-based but may also involve difficulties in coordination of response. To assume the lack of an empathic response indicates lack of empathy when there are so many other factors involved is overly simplistic at best. You could also call it unempathic, when directed at a vulnerable demographic group, as it is in this paper. Worst of all, it is a missed scientific opportunity, since a properly accurate and detailed interpretation can lead to so many productive avenues of research.

Baron-Cohen and Wheelwright also found that non-autistic women tend to score higher than non-autistic men, which is no big deal. Other researchers have found the same thing with other scales, and it fits with general impressions of differences between the sexes. I'd be worried if they didn't find that, regardless of whatever it is they're actually measuring.

Recommendations

1. Combine EQ and SQ into one scale. Do factor analysis to see if they measure two factors (or more, or fewer). If they do, reduce the scale to a smaller more focussed scale, using the best test items only. ELIMINATE ALL ITEMS THAT ARE BIASED AGAINST PEOPLE WITH LANGUAGE OR COMMUNICATION DIFFICULTIES. Just as you do not test social skills in blind people by measuring their ability to read faces, do not test empathy in autistic people by measuring their ability to follow and participate in conversations.
2. Test validity of the scales by correlating EQ (and SQ) scores with other measures, created by other people not connected to these researchers, of the same or related concepts (e.g. the IRI). DO NOT assume that the EQ must be measuring empathy if autistic people score lower on it. First of all, there is no independent proof that autistic people are lower on empathy, just an assumption, not properly tested. Secondly, even if autistic people are lower on empathy and also score lower on the EQ, the EQ may still be measuring something else.
3. It is always dangerous to construct a scale to measure a general trait to test a specific hypothesis. Whenever possible, use a well-constructed scale previously designed by others, in order to avoid test construction bias.
4. DO NOT PUBLISH RESULTS related to specific demographic groups (e.g. autistic adults) until after you are sure the scale measures what it is supposed to. You can always print a retraction later, if it turns out you were wrong, but will anyone read it? And even if they do, can you undo the damage? Better to make sure you know what you're measuring first, before going out on a limb with it.

References

Davis, Mark H., 1980. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology, 10, p. 85. Retrieved June 14, 2008 at http://www.eckerd.edu/academics/psychology/files/Davis_1980.pdf

Davis, Mark H., 1983. Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44(1):113-126.

Baron-Cohen, Simon, and Sally Wheelwright, 2004. The Empathy Quotient: An investigation of adults with Asperger Syndrome or High Functioning Autism, and normal sex differences. Journal of Autism and Developmental Disorders, 34 (2):163-175. Retrieved June 9, 2008 at: http://www.autismresearchcentre.com/tests/eq_test.asp

EQ also available online at http://homepage.mac.com/lpetrich/Asperger/EmpathyQuotient.html

Postscript - Some reasons why autistic people might score low on empathy:

1. Stress: We might have empathy but we're too tired/overwhelmed to feel anything right now.
2. Self-fulfilling prophecy: We might score low if we think we're supposed to.
3. Diagnostic bias: People who are low in empathy might be more likely to be diagnosed with autism than people who are high in empathy, since they fit the stereotype better. So as a result, autistic people may end up having less empathy as a group (by definition).
4. Neurodiversity: Some of us are genuinely low in empathy. Same as for everyone else.

Postscript 2 - Ok, so someone did a factor analysis. But I'm still not impressed.

In a separate paper (E.J. Lawrence, P. Shaw, D. Baker, S. Baron-Cohen and A.S. David, 2004. Measuring empathy: reliability and validity of the Empathy Quotient. Psychological Medicine, 34:911-924, available at same link as EQ paper above), Baron-Cohen and colleagues did do additional analysis of the EQ, including a factor analysis of 29 of the 40 items scored in the EQ. 28 items loaded on three factors, including 7 items on a "social skills" factor, as well as 13 on a "cognitive empathy" factor and 12 on an "emotional reactivity" factor. Four items loaded on more than one factor, and one loaded on none. (They also found moderate correlations of the "emotional reactivity" factor of the EQ with the PT and EC scales of the IRI, for the 28 subjects who completed both scales.) To my knowledge, this factor analysis has not had an effect on the contents of the actual scale. I do not see a revised EQ (EQ-R) on their website. So why did they bother?