Interview with Andy Maul about Test Development (part 1)

Andy Maul, who took introductory psychology with me, is a graduate student in Educational Psychology at UC Berkeley.

1. What is your research about?

I’m taking a closer look at tests recently developed to measure the construct of emotional intelligence (EI). In particular, I’m looking at the Multifactor Emotional Intelligence Scale (MEIS) and the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT), which were both developed in the past decade and evaluated using traditional methods (confirmatory factor analysis [CFA] and classical test statistics such as alpha coefficients, along with correlations with other tests and hypothesized outcomes). I’m looking at these tests again, both though the traditional lens of CFA and through the newer lens of Item Response Theory (IRT). In the end, I hope to make points both for the development of EI tests, and for psychological measurement in general, by highlighting how newer methods can improve the construct- and test-building process.

2. How did you get interested in this line of research?

I became interested in emotions by working with Professor Dacher Keltner. At some point in graduate school my interests shifted to the more quantitative side of research, and I’ve since been working with Professor Mark Wilson on test theory and statistical measurement. I thought combining the two interests, by evaluating tests of emotional intelligence through a quantitative lens, would be a good idea.

3. What’s an example of research that shows the value of measuring emotional intelligence?

The MSCEIT appears to predict some life outcomes (such as grades, prosocial behavior, and self-reported life satisfaction), even controlling for IQ and personality. Other researchers have challenged these claims as being premature and based on insufficient evidence. There are multiple problems with the validity of existing EI tests that make them difficult to interpret, and make claims based on them highly suspect.

Some researchers feel that defining and measuring emotional intelligence could clarify and expand our definitions of intelligence and cognitive abilities in general, and provide information about an area of human functioning that could predict important personal and interpersonal outcomes (such as life satisfaction and the quality of one’s relationships) above and beyond traditionally-measured intelligence and personality. In today’s era of high-stakes testing, with so much riding on what many feel to be tests with limited utility, a new, well-validated test of emotional intelligence could provide insight into what makes students successful in schools and in life.

References

Mayer, J., Salovey, P., & Caruso, D. (2002). Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT): User’s manual. Toronto, Canada: Multi-Health Systems.

Mayer, J., Salovey, P., Caruso, D., & Sitatenios, G. (2003). Measuring emotional intelligence with the MSCEIT V2.0. Emotion, 3, 97-105.

O’Sullivan, M. (2005) Trolling for trout, trawling for tuna: The methodological morass in measuring emotional intelligence. In press.

Palmer, B., Gignac, G., Manocha, R., & Stough, C. (2005). A psychometric evaluation of the Mayer-Salovey-Caruso Emotional Intelligence Test Version 2.0. Intelligence, 33, 285-305.

Roberts, R. D., Schulze, R., Zeidner, M., & Matthews, G. (2005). Understanding, measuring, and applying emotional intelligence: What have we learned? What have we missed? In R. Schulze & R. D. Roberts (Eds.), Emotional intelligence: An international handbook (311—341). Cambridge, MA: Hogrefe & Huber.

One thought on “Interview with Andy Maul about Test Development (part 1)

  1. Hello,

    I found the comments about emotional intelligence to be very interesting but not entirely accurate. For a period of time I published the MEIS for Mayer-Salovey-Caruso and actually co-authored the certification training for others to use the test. At a later time my certification program was used to introduce the newer version of their test, the MSCEIT. I now certify people to use the MSCEIT and distribute both the MSCEIT and the BarOn EQi. My reason for commenting is that the MSCEIT actually is based on a sample of 5000 people from a variety of countries and ethnicities. It is scored in two ways, one as Andy suggests, general consensus and the second is based on emotions experts. For a detailed explanation I refer people to the MSCEIT User’s Manual. This is not to say that the MSCEIT is a perfect test, I just want your readers to have accurate information. I actually have found that the MSCEIT is incredibly useful, powerful and compelling. The actual report you get leaves much to be desired but when people are trained in how to interpret it the results are very meaningful.

Leave a Reply

Your email address will not be published. Required fields are marked *