Perspectives on Psychological Science: Blogs Don't Exist

The Seductive Allure of Neuroscience Explanations

The previous post, Voodoo Correlations: Two Years Later, was a retrospective on the neuroimaging methods paper that was widely discussed in the blogosphere before it was considered "officially" published (Vul et al., 2009). The article, a controversial critique of the statistical analyses used by fMRI investigators in social neuroscience, made its initial appearance on Ed Vul's website once it was accepted by Perspectives in Psychological Sciences. This caused considerable consternation among the criticized authors and the journal editor (Ed Diener).

Now, as part of the November 2010 issue of the journal (Diener's last as editor), six invited articles on Neuroimaging: Voodoo, New Phrenology, or Scientific Breakthrough? appear in a Special Section on fMRI (Diener, 2010). I was pleased to see that one of the articles addressed The Appeal of the Brain in the Popular Press (Beck, 2010), since this has been a major theme of my blog for almost five years. However, I was disappointed that the word "blog" was not mentioned at all in Beck's article.

This should have come as no surprise, given the journal's response to bloggers in May 2009. The Editor's Introduction is worth a mention for the issues it raises about peer review and publication in these modern times.

PREPUBLICATION DISSEMINATION

As soon as I accepted the Vul et al. article, I heard from researchers about it. People around the globe saw the article on the Internet, and replies soon appeared as well. Although my plan was to publish the article with commentary, the appearance of the article on the Internet meant that researchers read the article without the accompanying commentaries and replies that I had planned to publish with it.

In some fields such as economics, it is standard practice to widely disseminate articles before they are published, whereas in much of psychology this has been discouraged. An argument in favor of dissemination is that it speeds scientific communication in a fast-paced world where journal publication is often woefully slow. An argument against dissemination of articles before publication is that readers do not have the opportunity to simultaneously see commentary and replies. ... In the Internet age, the issue of prepublication distribution becomes all the more important because an article can reach thousands of readers in a few hours. Given the ability of the Internet to communicate so broadly and quickly, we need greater discussion of this issue.

In reply, I wrote:

Bloggers have discussed this specific issue months ago. For example, as noted in Mind Hacks,

The paper was accepted by a peer-reviewed journal before it was released to the public. The idea that something actually has to appear in print before anyone is allowed to discuss it seems to be a little outdated (in fact, was this ever the case?).

And The Neurocritic opined that...

[The aggrieved authors] are not keeping up with the way that scientific discourse is evolving. Citing "in press" articles in the normal academic channels is a frequent event; why should bloggers, some of whom are read more widely than the authors' original papers, refrain from such a practice? Is it the "read more widely" part?

-from The paper formerly known as "Voodoo Correlations in Social Neuroscience", by The Neurocritic

Diener originally solicited six commentaries on the Vul et al. paper for the May 2009 issue of the journal. Ironically, authors on two of the papers have serious blogs:

Statistical Modeling, Causal Inference, and Social Science is a blog written by Andrew Gelman, a Professor of Statistics and Political Science at Columbia. He was one of the first to blog about the paper in Suspiciously high correlations in brain imaging studies, with a more detailed post a month later (More on the so-called voodoo correlations in neuroscience). Lindquist and Gelman (2009) applauded the discussion engendered by "pre-publication dissemination":

Their article has in a short time given rise to a spirited debate about key statistical issues at the heart of most functional neuroimaging studies. The debate provides a useful opportunity to discuss core statistical issues in neuroimaging and ultimately provides a chance for the field to grow and move forward.

[citation needed] is the blog kept by Tal Yarkoni, a Post-Doc at the University of Colorado Boulder. He happens to be an expert in statistics for fMRI analysis, and another one of the authors invited to submit a paper for the Vul, Harris, Winkielman, and Pashler festschrift/verdammung (Yarkoni, 2009):

In this article, I argue that Vul et al.'s primary conclusion is correct, but for different reasons than they suggest. I demonstrate that the primary cause of grossly inflated correlations in whole-brain fMRI analyses is not nonindependence, but the pernicious combination of small sample sizes and stringent alpha-correction levels. Far from defusing Vul et al.'s conclusions, the simulations presented suggest that the level of inflation may be even worse than Vul et al.'s empirical analysis would suggest.

His blog started in October 2009, after the commentaries were published.


The Appeal of the Brain in the Popular Press

That brings us back to the article by Diane Beck, an Assistant Professor in Psychology and Cognitive Neuroscience at the University of Illinois Urbana-Champaign. She examines the distorted media coverage of neuroimaging studies, and possible reasons for it (Beck, 2010):

Since the advent of human neuroimaging, and of ... fMRI in particular, the popular press has shown an increasing interest in brain-related findings. In this article, I explore possible reasons behind this interest, including recent data suggesting that people find brain images and neuroscience language more convincing than results that make no reference to the brain (McCabe & Castel, 2008; Weisberg, Keil, Goodstein, Rawson, & Gray, 2008). I suggest that part of the allure of these data are the deceptively simply messages they afford, as well as general, but sometimes misguided, confidence in biological data. In addition to cataloging some misunderstandings by the press and public, I highlight the responsibilities of the research scientist in carefully conveying their work to the general public.

While reading through the examples of poor media coverage, imagine the shock of recognition if you were to realize that you have written several trenchant blog posts criticizing these very articles. Yet all this work (and the writings of many others) is rendered invisible to the mainstream of the Association for Psychological Science.

Why is blogging so non-existent in these circles? There's a large thriving community of science blogs. Go to ResearchBlogging.org and look under Psychology, for starters. An excellent example of a psychology blog written by a senior investigator is BishopBlog by Professor Dorothy Bishop, who studies children's communication impairments at Oxford. Two of my favorite posts are The difference between p < .05 and a screening test and Science journal editors: a taxonomy.

As for Beck's hit list and The Seductive Allure of Neuroscience Explanations (Weisberg et al., 2008), we have:

(1) Men see bikini-clad women as objects, psychologists say

In other words, the choice of tasks or conditions that the researcher will contrast is absolutely critical to obtaining activity, as well as any inference that can be made about this activity. By omitting this fact from a press article, one gets the impression that participants are asked to do a single task in the scanner, such as view pictures of women in bikinis, and voilà, a set of areas light up (Landau, 2009)...

Here's Spanner or Sex Object? by The Neurocritic:

OK, the abstract doesn't specifically mention the tool/bikini experiment, so we have to rely on newspaper articles and quotes from the first author. Judging from the first 4 paragraphs of the Guardian article, Fiske's conclusions rely on the logical fallacy of reverse inference - one cannot directly infer the participants' cognitive or emotional state from the observed pattern of brain activity in neuroimaging experiments. How do we know that the "sex object" neural response was related to tool use? Did the experiment require the subjects to use tools? Did it explicitly ask them to anticipate using tools? How about watching others use tools?

(2) Greene JD, Sommerville RB, Nystrom LE, Darley JM, Cohen JD. (2001). An fMRI investigation of emotional engagement in moral judgment. Science 293:2105-8.

What does it mean to say that moral decisions are associated with activity in regions implicated in emotional processing (Greene et al. 2001)? What exactly is meant by moral decisions or emotional processing? The only way to really understand these statements is to also know what are not moral decisions, and how emotional processing is being defined.

Here's Everybody's a Neurocritic!

In that paper, the authors

reported that the medial frontal gyrus and other brain regions linked to emotion become more active when people contemplate "personal" moral dilemmas--such as shoving the man onto the trolley tracks or removing a man's organs against his will to save five transplant recipients--compared with when they weigh impersonal moral dilemmas--such as flipping a switch to save the workers or declaring bogus business expenses on a tax return.

Besides the medial frontal gyrus [BA 9/10, which did not replicate in Experiement 2], what were these other brain regions linked to emotion? Did they include the insula? No, they did not. They included the posterior cingulate gyrus (which has some grounding in reality) and the L and R angular gyri (which does not).

(3) A Neuroscientific Look at Speaking in Tongues

...In the press article, the author describes a study that used single photon emission computed tomography to compare regional cerebral blood flow in women speaking in tongues to cerebral blood flow in the same women singing gospel music (Newberg, Wintering, Morgan, & Waldman, 2006). Although glossolalia and singing both involve verbal utterances and evoke religious meaning in practioners, the women described a lack of voluntary control over their vocalizations only during glossolalia. The study found decreased activity in the prefrontal cortex during glossolalia, a result that the study’s authors describe as consistent with the women’s descriptions of a lack of intentional control over their utterances. The press article does not explicitly endorse this conclusion but instead chooses to quote Andrew Newberg, the lead author on the study: “The amazing thing was how the images supported people’s interpretation of what was happening. The way they describe it, and what they believe, is that God is talking through them” (Carey, 2006). ... The study’s result, however, is neither amazing nor does it support any particular cause of glossolalia. Decreased activity in the prefrontal cortex could be due to any number of reasons...

Glossolalia from The Neurocritic still gets comments from those who feel they speak in tongues, probably because it appears on the first page of a Google search on the topic.

And The New York Times Is Speaking In Tongues:

As I mentioned, oh, just the other day, (1) the authors did not correct for multiple comparisons, (2) the spatial resolution of SPECT is not that great, and (3) the rCBF reductions in right dorsolateral prefrontal cortex and (especially) the left caudate were their most significant findings.

(4) Different brain responses found in homosexual, heterosexual men [original link from the article was broken]

...the Associated Press (Schmid, 2005) reported on a PET study showing that when homosexual men sniffed a derivative of testosterone, their hypothalamus responded more like that of heterosexual women than heterosexual men (Savic, Berglund, & Lindstrom, 2005). The Associated Press rightly stated that “the findings clearly show a biological involvement in sexual orientation.” However, they then make an erroneous jump from describing homosexuality as being biological to being innate, primarily in the form of a quote from Dr. Sandra Witelson: “It is one more piece of evidence … that is showing that sexual orientation is not all learned” (Schmid, 2005). A difference in the brain in no way indicates that the behavior under study is not learned...

See Sweat, Urine, and Sexual Orientation and The PNAS Word from 2006. [I got tired of waiting for the embargo to lift, so these aren't as critical as I might have liked.]

You get the idea. But in the end, it's all for naught because Diener (2009) has spoken:

I believe that the debate can itself stimulate useful discussions about scientific practices and communication. Further discussion of the issues should now take place in journals that are focused on imaging and neuroscience, so that the readers there can judge and benefit from the ensuing discussions.

References

Beck, D. (2010). The Appeal of the Brain in the Popular Press. Perspectives on Psychological Science, 5 (6), 762-766 DOI: 10.1177/1745691610388779

Diener E (2009). Editor's Introduction to Vul et al. (2009) and Comments. Perspectives on Psychological Science 4:272-273,

Diener E (2010). Neuroimaging: Voodoo, New Phrenology, or Scientific Breakthrough? Introduction to Special Section on fMRI. Perspectives on Psychological Science 5:714-715.

Lindquist MA, Gelman A (2009). Correlations and Multiple Comparisons in Functional Imaging: A Statistical Perspective (Commentary on Vul et al., 2009). Perspectives on Psychological Science 4:310-313.

Vul E, Harris C, Winkielman P, Pashler H (2009). Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition. Perspectives on Psychological Science 4:274-290.

Weisberg DS, Keil FC, Goodstein J, Rawson E, Gray JR. (2008). The seductive allure of neuroscience explanations. J Cogn Neurosci. 20:470-7.

Yarkoni T. (2009). Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power—Commentary on Vul et al. Perspectives on Psychological Science 4:294-298.

The Seductive Allure of Neuroscience Blogs