Neuroscience: Who do voodoo? You do, apparently (you might if you are a neuroscience researcher) - another paper says so

In "It's those Voodoo correlations again ... brain imagers accused of 'double dipping'", the British Psychological Society's Research Digest Blog reports,

This time there's no explicit naming and shaming, and the title may not be as colourful, but a new study out today in prestige journal Nature Neuroscience, echoes many of the same concerns voiced earlier this year in the leaked paper "Voodoo Correlations in Social Neuroscience" (since renamed as "Puzzlingly High Correlations ...").

And the new paper's implications are surely just as profound for the cognitive neuroscience community. Nikolaus Kriegeskorte and colleagues analysed all the fMRI studies published in Nature, Science, Nature Neuroscience, Neuron and Journal of Neuroscience, in 2008, and found that 42 per cent of these 134 papers were guilty of performing at least one non-independent selective analysis - what Kriegeskorte's team dub "double dipping".

This is the procedure, also condemned by the Voodoo paper, in which researchers first perform an all-over analysis to find a brain region(s) that responds to the condition of interest, before going on to test their hypothesis on data collected in just that brain region. The cardinal sin is that the same data are used in both stages.

A similarly flawed approach can be seen in brain imaging studies that claim to be able to discern a presented stimulus from patterns of activity recorded in a given brain area. These are the kind of studies that lead to "mind reading" headlines in the popular press. In this case, the alleged statistical crime is to use the same data for the training phase of pattern extraction and the subsequent hypothesis testing phase.

Read the rest here.

Far from wanting to dump on neuroscience research, I see this as an opportunity for housecleaning.

People should not be doing neuroscience on subjects like "why women love to shop" or "why anyone would vote for Sarah Palin."

The sheer weight of cultural baggage pretty much guarantees that most such research will be a waste of time. You can learn more about women's shopping preferences by studying their charge card bills in relation to a given economic climate than by studying their brains. And if you want to know why some Americans voted for Sarah Palin, well, ask them. Chances are, the reasons they give you will be reliable.

I myself believe that neuroscience should stay anchored in medicine, and similarly practical projects.