I recently co-authored a paper on how a little-known parapsychology journal was years ahead of its time.
Our story starts in 2011, when psychologist Daryl Bem reported several experiments that appeared to support the existence of psychic ability. Soon after, Stuart Ritchie, Chris French and I tried to replicate the studies but obtained null results. Several other academics also criticised Bem’s statistics and procedures. This type of ‘I have evidence for psychic ability – Oh no you don’t’ back and forth has occurred many times over the years. However, this time, something odd happened.
Several researchers noted that the criticisms aimed at Bem’s work also applied to many studies from mainstream psychology. Many of the problems surrounded researchers changing their statistics and hypotheses after they had looked at their data, and so commentators urged researchers to submit a detailed description of their plans prior to running their studies. In 2013, psychologist Chris Chambers played a key role in getting the academic journal Cortex to adopt the procedure (known as a Registered Report), and many other journals quickly followed suit.
However, many academics are unaware that a little-known parapsychology journal – The European Journal of Parapsychology – implemented an early version of this concept in the 1970s. For 17 years, around half of the studies in the paper were registered in advance. If the critics were right, these papers should be less likely to contain problems with their statistics and methods, and so be more likely to report spurious positive results. To find out if this was the case, I recently teamed up with Caroline Watt and Diana Kornbrot to examine the studies. The results were as expected – around 8% of the analyses from the studies that had been registered in advance were positive, compared to around 28% from the other papers. Other academics are now conducting the same sort of analyses in psychology and medicine, and finding the same pattern.
Academics often criticise parapsychology, but this episode is a good example of how the field is sometimes ahead of the game and can help to improve mainstream psychology.
The full paper describing this work is here.
And Diana is now looking more broadly at openness in science. If you are a researcher with an interest in the area, you can take part in her survey here.