Neurocinema, Neurocinematics

Is there a difference?



In The Neurocinema Collection™, we established that the term "neurocinema" doesn't really refer to movies with "Neuro" in the title or films about neurology. Instead, neuromarketers have have used the word in the following fashion:

Neurocinema is a new filmmaking process that studies a viewer's sensorimotor, cognitive, and affective response to film stimuli. Researchers use technologies such as functional magnetic resonance imaging (fMRI) to measure changes in activity in parts of the brain,electroencephalography (EEG) to measure activity in specific regional spectra of the brain response, and/or sensors to measure changes in one's physiological state (heart rate, respiratory rate, galvanic skin response) to learn exactly what scenes excite or disinterest the viewer.

There have been no peer-review studies on this methodology, only articles in the popular press (e.g., Wired.com and CNN.com). And from what was available in those reports, there was plenty to be neurocritical about.

On the other hand, Uri Hasson and his NYU colleagues published an article entitled Neurocinematics: The Neuroscience of Film in Projections: The Journal for Movies and Mind. Granted, that article is lacking the methodological details one would see in NeuroImage or Human Brain Mapping, but in fact the authors have published their work in prestigious journals (like Science and Neuron) without the "Neurocinematics" title.

In their Projections article, Hasson et al. (2008) state:

We propose that ISC [inter-subject correlation analysis] may be useful to film studies by providing a quantitative neuroscientific assessment of the impact of different styles of filmmaking on viewers’ brains, and a valuable method for the film industry to better assess its products. Finally, we suggest that this method brings together two separate and largely unrelated disciplines, cognitive neuroscience and film studies, and may open the way for a new interdisciplinary field of “neurocinematic” studies.

Hasson and colleagues first described the ISC method in their 2004 paper. In that study five subjects watched 30 min of The Good, The Bad, and The Ugly while laying supine in an MRI scanner. Unlike a typical fMRI experiment involving controlled presentation of individual pictures and performance of a categorization task, free viewing of a dynamic film is a more naturalistic situation. It also presents a more challenging data analysis problem. To tackle this, the researchers eschewed standard hypothesis-testing and adopted a purely data-driven approach to determine commonalities in brain activation across subjects.¹


Figure 1B (Hasson et al., 2008). The ISC analysis measures similarity in brain activity across viewers by comparing the response time course in each brain region from one viewer with the response time courses obtained in the same brain region from other viewers during movie watching.

The results (Hasson et al., 2004) demonstrated that...

Despite the free viewing and complex nature of the movie, we found an extensive and highly significant correlation across individuals watching the same movie. Thus, on average over 29% ± 10 SD of the cortical surface showed a highly significant intersubject correlation during the movie.....

Close inspection of this across-subject correlation revealed that the synchronization was far more extensive than the boundaries of well-known audiovisual sensory cortex defined with conventional mapping approach. ...the across-subject correlation covered most of the visual system, including early retinotopic areas as well as high-order object areas within the occipitotemporal and intraparietal cortex. Moreover, the correlation extended far beyond the visual and auditory cortices to the entire superior temporal (STS) and lateral sulcus (LS), retrosplenial gyrus, even secondary somatosensory regions in the postcentral sulcus, as well as multimodal areas in the inferior frontal gyrus and parts of the limbic system in the cingulate gyrus.

When corrected for non-selective elements, the intersubject correlation remained at 24% ± 8.5. The other 75% of the brain not "in sync" across subjects included parietal lobe areas (supramarginal and angular gyri) and the bulk of prefrontal cortex. Subcortical areas important for emotion and memory (amygdala and hippocampus) were not examined.

A second analysis was also performed, which took a "reverse-correlation" approach to examine activity in two well-know category-specific areas: the fusiform face area (for faces) and the parahippocampal place area (for buildings). This type of functional specialization was demonstrated in an earlier paper as well (Bartels & Zeki, 2004). In that study, the participants watched a James Bond movie (Tomorrow Never Dies). Activity in category-specific regions (for color, faces, language, and human bodies) was correlated with the perceived intensity of these specific attributes (as rated by a separate group of subjects who watched the same movie).

The most fascinating result in my opinion was obtained in subsequent experiments (described in Hasson et al., 2008) that compared the ISC for The Good, The Bad, and The Ugly to those observed for an episode of the TV series Alfred Hitchcock Presents (which is on Hulu!!!), an episode of the semi-improvised comedy Curb Your Enthusiasm, and a real-life unedited video shot in Washington Square Park. Not surprisingly,

The extent of ISC differed for the four movies (Figure 7A). The percentage of cortex exhibiting high ISC provided a measure of the overall effectiveness, or collective engagement power, of each movie to induce similar responses across viewers (Figure 7B). The Hitchcock episode (green) evoked similar responses across all viewers in over 65 percent of the cortex, indicating a high level of control of this particular episode on viewers’ minds. The high ISC was also extensive (45%) for the Good, the Bad and the Ugly (blue), but much less so (18%) for Curb Your Enthusiasm (red). Finally... the unstructured segment of reality (orange) induced high ISC only in a small fraction of the cortex (less than 5%).

Figure 7 (Hasson et al., 2008).

Glossing over the technical details [I didn't read the 5 other cited papers] and discrepancies between the 2004 and 2008 studies [divergence in ISC values of 25% vs. 45% for the same film], the implication is that different directors, and different filmmaking styles, exert varying levels of "control" over audio-visuo-higher-level cortical responses in the brains of the viewers:

The fact that Hitchcock was able to orchestrate the responses of so many different brain regions (Figure 7, green), turning them on and off at the same time across all viewers, may provide neuroscientific evidence for his notoriously famous ability to master and manipulate viewers’ minds. Hitchcock often liked to tell interviewers that for him “creation is based on an exact science of audience reactions” (Douchet 1985).

Different filmmakers strive to achieve different levels of control over their viewers’ reactions. Our findings provide empirical evidence to support the long-lasting distinction in film theory between films that remain faithful as much as possible to reality and those that seek to control or distort it.

The amount of control employed by filmmakers exists on continuum that ranges from real life --> documentaries --> art films --> Hollywood --> propaganda. The authors conclude with a discussion of collective engagement in film and suggest that it can be measured in an "objective" fashion using ISC. Overall, the 2008 Projections paper was a BOLD proposal for a true merger of art and science in academia.

Neurocinema, Neurocinematics? A semantic distinction perhaps, but the terms first appeared in different contexts. The former was used to describe an unvalidated filmmaking "process" [and underdeveloped commercial application] aimed squarely at attracting Hollywood megabucks, whereas the latter was used in a scholarly publication to describe a new field of inquiry. Will a merger (of sorts) be far behind? Or more to the point, has the NYU group applied for a patent yet?


Footnote

¹ The methods for this were pretty brief and largely contained within Footnote 15 of the paper:

15. In order to search for correlation between two corresponding regions across subjects (intersubject correlation), we first aligned all brains into Talairach coordinate system and used a Gaussian filter of 12 mm full width at half maximum value (FWHM) to the data... To remove preprocessing artifacts, we excluded the first and last 10 time points of the experiment from the analysis. We then used the time course of each voxel of the source subject as a GLM predictor for modeling the activity in the corresponding voxel of the target subject. Only voxels whose P value was no more than 0.05 (corrected) were considered significant.

References

Bartels A, Zeki S. (2004). Functional brain mapping during free viewing of natural scenes. Hum Brain Mapp. 21:75-85.

Hasson U, Landesman O, Knappmeyer B, Vallines I, Rubin N, Heeger DJ. (2008). Neurocinematics: The Neuroscience of Film. Projections 2:1-26. [PDF]



Hasson, U. et al. (2004). Intersubject Synchronization of Cortical Activity During Natural Vision. Science, 303 (5664), 1634-1640. DOI: 10.1126/science.1089506

Further Reading

Neurocinematics - The Valve - A Literary Organ

Neurocinematics - Neurophilosophy blog

Neurocinematics! Where Neuroscience Meets Filmmaking - NYU panel discussion held May 4, 2009