Talk by Jean-Michel Hupé ( CerCo lab )
The false promises of MRI in cognitive neuroscience
Amphi Jean Jaurès
ENS, 29 rue d'Ulm 75005 Paris
with financial support from
Most published brain imaging studies contain errors; the complexity of the data is not captured by any adequate model; the logic of statistics used in data analysis, null hypothesis significance testing, is not appropriate anyway. For those reasons, a revision of brain imaging software was called for (Hupé 2015, Frontiers in Neuroscience). Since then, the authoritative American Statistical Association published (at last!) a statement banning p-values from valid scientific reasoning (Wasserstein & Lazar 2016, The American Statistician). This concerns the whole scientific production, not only brain imaging studies. Fortunately, when frequentist models are correctly applied to the data, misleading interpretations of p-values can be easily corrected by computing confidence intervals (e.g., Hupé 2017, Science). However, for fMRI, a recent study confirmed empirically that statistical procedures based on parametric models were unable to control the rate of false positives: Eklund, Nichols & Knutsson (2016, PNAS) suggested that up to 80% of so-called “significant” activations revealed by those analyses could be false positives, even when performed without errors and using state-of-the art analysis. How are we going to solve this major scientific crisis? Eklund et al. advocated that non-parametric procedures (permutation tests) allowed the correct control of the risk of false positives. However, to my knowledge, permutation tests do not allow any inference on physiological measures (that is, the computation of confidence intervals of parameters interpretable within a theoretical model). Trying to rely on statistical tricks highlights a general problem in cognitive neuroscience, where the measure tends to be confounded with what is measured (akin to mythic thinking in the sense of Barthes). It seems that the understanding and validity is not questioned anymore of what psychologists used to call “hypothetical constructs” (MacCorquodale & Meehl 1948, Psychological Review). Many if not most attempts of “brain behavior correlations” face the difficulty to construct relevant measures, that is, measures constructed based on solid theories. Instead, the very lack of any solid theory (we have no idea of the neural code of thoughts) is replaced by the computation of averages of what is measurable, without questioning what the mean means. Claude Bernard had yet warned that the only possible science of living systems may be a science of the individual (Introduction à l’étude de la médecine expérimentale, 1865). I would therefore suggest that cognitive neuroscience has not been short of technology but epistemology. Without reconsidering what the valid objects to measure are, we may continue to “avancer à l’aveuglette, dans un empirisme dénué de sens », as already diagnosed in 1998 by the great molecular biologist Michel Djerzinski. I shall illustrate my claim with a couple of popular results of cognitive neuroscience, which are in fact brain myths. But maybe in our post-truth world it does not matter anymore if story-telling has taken the place of science (or was there ever anything but story-telling?).
My research is focused on visual perception, with more and more emphasis on its subjective aspects.
I did my Ph.D. thesis (1995-1999) in Lyon (France) with Jean Bullier on the role of feedback connections in motion perception. Then I spent three years at New York University working with Nava Rubin on motion perception using psychophysics methods and learning fMRI. I started to work on the bistable perception of ambiguous stimuli. I came back to France in 2002, in Toulouse's CerCo lab, where I was recruited in 2004 as a CNRS research scientist.
I have been working on three research projets (multistable perception, synesthesia and mental images), while trying to improve on methodological issues.