Lie To Me on the Autobiographical Implicit Association Test

Lie to Me - Season 1 - "Moral Waiver" - Monica Raymund as Ria Torres
courtesy Adam Taylor/Fox

Lie detection is all the rage with the new TV show based on Paul Ekman's work¹ that uses "microexpressions" to detect deception.

The use of brain imaging technologies as lie detectors, and the admissibility of data obtained in this fashion as evidence in a court of law, has a high media profile as well - most recently (and notoriously) because of a juvenile-sex-abuse case in San Diego, recounted by Wired Science. The Stanford Center for Law & the Biosciences Blog has sounded the alarm in their post, No Lie MRI being offered as evidence in court:

The case is a child protection hearing being conducted in the juvenile court. In brief, and because the details of the case are sealed and of a sensitive nature, the issue is whether a minor has suffered sexual abuse at the hands of a custodial parent and should remain removed from the home. The parent has contracted No Lie MRI and apparently undergone a brain scan...

The defense plans to claim the fMRI-based lie detection (or “truth verification”) technology is accurate and generally accepted within the relevant scientific community in part by narrowly defining the relevant community as only those who research and develop fMRI-based lie detection.

The Neurocritic weighed in on the overblown nature of these claims three years ago, with Brain Scans and Lie Detection: True or False?, Would I Lie to You?, and More Lies... Damn Lies... But even better, check out the excellent Deception Blog for an updated overview of the field.

UPDATE: Request to admit No Lie MRI report in California case is withdrawn. The dependent’s counsel in the case sent a thank you note to Professor Hank Greely, Director of the Center for Law and the Biosciences at Stanford, who also notes:

Special mention should go to Marcus Raichle, M.D., of Washington University in St. Louis [and recent winner of the George A. Miller Prize in Cognitive Neuroscience], for agreeing to take time out of his very busy schedule to fly to San Diego and testify that use of fMRI for lie detection is not yet generally accepted by the relevant scientific community as reliable, especially for real world, high-stakes situations involving individuals...

Given the high cost and dubious accuracy of fMRI technologies -- as well as the questionable accuracy of older EEG and polygraph methods -- there has been some interest in developing faster, easier, more reliable lie detection methods. Ian Sample at the Guardian's Science Blog went with this futuristic headline about the potential use of pupillometry as a routine security screening measure:

Homeland Security seeks Bladerunner-style lie detector

Do our eyes betray us when we lie? The US government hopes to find out

. . .

Under the Small Business Innovation Research programme, the department has asked tech companies to bid for contracts to kick-start research in the area. Such a system, if it works, would undoubtedly be useful at airports and other high-security points.

Here's the original SBIR solicitation for applications, which were due in February 2008:

TITLE: Assess Ability to use Eye Tracking and Pupil Dilation to Determine Intent to Deceive

DESCRIPTION: Recent government sponsored research is working to produce a new line of flexible physiological and behavioral sensor technologies that are to be available for homeland security applications. These sensors, which must be non-invasive in nature and protect the privacy of the individual(s) involved, will be used to support human centered/behavioral screening processes in a variety of high and low volume venues. Security screening is conducted to evaluate the risk of individuals entering transportation and other critical infrastructure and requires efficient, rapid and accurate examination of a person. Persons involved in or planning to be involved in possible malicious or deceitful acts will show various behavioral or physiological abnormalities. Much of the technology and publications to date have focused on detection of guilty individuals using electrodermal measures. Research into other psychophysiological measures or the mechanisms underlying deception is still in its early stages. Early research has shown that pupil size varies with changes in a person's cognitive processing load. Current but unproven studies suggest that a cognitive decision to deceive or practice deception will result in a increased pupil size due to the greater cognitive processing required in comparison to truthful recall. An assessment study to determine the correlation between Pupillometry (dilation and contraction of the pupil relative to observed stimulus or emotion) and intent to deceive is required.

For the ultimate in low-cost methods for lie detection, computerized reaction time tasks take the cake. An article about one of these appeared last year in Psychological Science (Sartori et al., 2008). The task they used is a variant of the Implicit Association Test (IAT) franchise that pits competing response tendencies against each other. Without getting into a lengthy discussion of the IAT, and the debates between its proponents and detractors,² the autobiographical IAT (aIAT) employed by Sartori et al. ...

...allows one to evaluate which of two contrasting autobiographical events is true for a given individual. This is accomplished by requiring the respondent to complete two critical blocks of categorization trials, each of which pairs a different potentially autobiographical event with true events. Because pairing of a truly autobiographical event with true events should facilitate responses, the specific pattern of response times (RTs) in the two blocks indicates which autobiographical event is true and which is false.

The participants saw different types of sentences and had to classify them as true/false or guilty/innocent. Examples of the different stimuli are listed below.


adapted from Table 2 (Sartori et al., 2008).

The real trick was in the way these sentence types were matched to response keys. In a series of five blocks of trials, the subjects were told to respond to each sentence as rapidly as possible. Across five experiments (each of which had a different kind of "guilty knowledge"), the order was Block 1: logical discrimination (true/false) - Block 2: initial autobiographical information (guilty/innocent) - Block 3: initial double categorization (false/innocent and true/guilty) - Block 4: reversed autobiographical discrimination (false/true) - Block 5: reversed double categorization (true/innocent and false/guilty). The order of the critical blocks 3 and 5 was counterbalanced across participants (as was the order of 2 and 4, which counterbalanced the response mapping for autobiographical trials accordingly). The crucial measure was the comparison between RTs to the double categorization trials in Blocks 3 and 5 -- innocent participants were expected to be slower on the "conflict" trials in which true/guilty and false/innocent were matched to the same response key, while guilty participants (who always denied their crimes) were expected to show the opposite pattern, which would reveal they were lying about their innocence.

For the mock crime of stealing a CD for example, the results looked like this for the pairings of true/guilty and true/innocent:


Fig. 1C (Sartori et al., 2008). For Experiment 2, results are shown for the critical block associating true sentences with guilty sentences and for the critical block associating true sentences with innocent sentences.

Because the tests were able to discriminate between true and false events with 91% accuracy, the authors concluded that "the aIAT is an accurate method of detecting concealed knowledge that outperforms currently available lie-detection techniques." However, a brand new paper by Verschuere et al. (2009) has demonstrated that it's easy to fake your results in this aIAT! Oops. What Verschurere and colleagues did was provide the participants with instructions on how to beat the test. First they replicated the methods (and results) of Sartori et al. with naïve subjects on an initial aIAT. Before performing the aIAT a second time, however, the participants were told to slow down their responses in the true/guilty mapping condition. And the results for the faking version of the aIAT classified the majority of guilty liars as innocent. Imposing a response deadline, so the subjects had to respond within 1200 msec, did not alter the findings.

So there you have it. An extremely easy method for faking your results on the aIAT. In the past, The Neurocritic has taken the Human or Alien? test and the Dead or Alive? test. Turns out I'm neither human nor alien, and neither dead nor alive. Read those posts, and then try the tests yourselves.

Footnotes

¹ However, as noted recently by World of Psychology, a paper by Bond (2008) questioned whether Ekman et al. (1991, 1999) omitted data unfavorable to their previously reported lie detection success rate of 73% in some federal agents.

² The sadly defunct blog Mixing Memory was particularly critical of the IAT:

The IAT isn't the only test of implicit "attitudes." . . . However, the IAT is the most popular, and has received a great deal of attention in the popular press, due in large part to a public relations campaign by its authors and the NSF and NIMH. In my mind, giving the IAT so much publicity is the most irresponsible thing I've seen in psychology since I began studying it... While the IAT has been publicized (by its authors!) as a measure of implicit attitudes, and even more, as a measure of implicit prejudice, there is no real evidence that it measures attitudes, much less prejudices. In fact, it's not at all clear what it measures, though the fact that its psychometric properties are pretty well defined at least implies that it measures something. On top of that, the IAT (like all of the other implicit tests) has serious methodological flaws that are currently being discussed in the literature. It's just irresponsible to publicize work, and claim that it does something very particular, when the work is still in the early stages and it's not at all clear what it's actually doing (read paper, or this one, for discussions of some of the problems with the IAT and other measures, including whether they actually measure "attitudes").

References

Sartori, G., Agosta, S., Zogmaister, C., Ferrara, S.D., & Castiello, U. (2008). How to accurately assess autobiographical events. Psychological Science 19:772–780.



Verschuere, B., Prati, V., & Houwer, J. (2009). Cheating the Lie Detector: Faking in the Autobiographical Implicit Association Test. Psychological Science DOI: 10.1111/j.1467-9280.2009.02308.x.


Lie to Me - Season 1 - "Moral Waiver" - Kelli Williams as Gillian Foster and Tim Roth as Cal Lightman
courtesy Adam Taylor/Fox