Psychologist Daryl Bem of Cornell University is just about to publish a parapsychology paper entitled ‘Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect’ in the Journal of Personality and Social Psychology (there is a draft of the article here). Bem’s paper suggests that future events can affect participants’ performance on well-known psychological tasks. The work has received lots of media attention, and several journalists have asked what I think about it.
Bem describes several studies in the paper, but two of them (Studies 8 and 9) have been the centre of much of the attention because (i) Study 9 produced the largest effect size of any of the experiments and (ii) Bem has released the software from these studies to researchers interested in replicating his work (Caroline Watt (Edinburgh University) and I have set up a registry for anyone attempting to do this here).
Stuart Ritchie (Edinburgh University) and I are planning to replicate the study. Yesterday we went over the procedure in detail and I think that the studies contain a potential methodological problem.
The studies were run by student experimenters, with other students acting as participants. The study software presented participants with a list of 48 words (e.g., CAT, SOFA, MUG, DESK), and then asked them to type all of the words that they could remember into the computer. The software then randomly selected half of the words in the original list (e.g., CAT, MUG) and presented them to the participants again. The participant did not see the non-selected words (e.g., SOFA, DESK). Let’s refer to the selected words as the ‘target words’ and the non-selected words as the ‘control’ words. Accoding to Bem’s results, participants were significantly more likely to remember the words in the ‘target’ than ‘control’ list (i.e., they appeared to be better able to remember those words that they would later see a second time.).
The potential problem is in the scoring. The experimenters used a second piece of software to score participants’ responses. Of course, participants may have misspelled remembered words (e.g., typing ‘CTT’ instead of ‘CAT’) or come up with words that were not on the original list (e.g., typing ‘CAR’ instead of ‘CAT’). To deal with this, the scoring software was designed to automatically go through the participant’s responses and to flag up any words that were not absolutely identical to the words that were not in the original list. The experimenter then had to go through these ‘unknown’ words manually, and either correct the spelling or tell the software to ignore them because they did not appear on the original list. To prevent any possibility of unconscious bias, the experimenter should have been doing this blind to the words in the ‘target’ and ‘control’ lists. Unfortunately, this was not the case.
The scoring programme listed the words submitted by the participant in one column. To the right of this were two more columns showing the ‘target’ and ‘control’ lists. Furthermore, when the experimenter made each decision about an ‘unknown’ word they had to change data in the columns containing the ‘target’ and ‘control’ lists. This procedure presented an opportunity for subjective bias to enter the scoring system. For example, if one of the words presented in the original list was ‘CAT”, and the participant typed ‘CAR’, does the experimenter re-code this as CAT? Or what if the participant typed ‘CTT’ – again, how should this be scored? In making these decisions the experimenter could have been unconsciously biased by whether the word CAT appears in the ‘target’ or ‘control’ lists.
The good news is that Bem’s programme stores all of the original data, so it should be possible to go through and recode the participants’ responses blind to whether the responses are present in the target or control lists. Until that happens it is problematic to interpret the results from these two studies. In addition, it is important that any replications of these studies don’t duplicate this error.
UPDATE 22/11/10. Daryl Bem replied as follows:
This is a response to Richard (Wiseman’s) concern about the ability of the experimenter to correct misspelled words while being able to observe which corrections will help the psi hypothesis (because the misspelled word is a practice word) or work against the psi hypothesis. This is a legitimate concern and I will modify the database so that that the category information is not available to the experimenter when he or she makes spelling corrections.
The program that runs the experiment automatically calculates the results of the session, ignoring all words it doesn’t recognize as literal copies of the test words. This analysis is also transferred to the database, which is set up so that the experimenter cannot change it or any of the original words as typed by the participant. Any changes made by the experimenter in the database are explicitly shown as changes, and a security check flags records in which the experimenter has corrected any of the original words. In other words, there is a complete record of the original data that cannot be altered. As an additional check, the critical data appear in the output file in both unencrypted and encrypted form, and only I know the encryption formula. If anything is changed in the output, the security flag in the database will read “False.”
Any experimenter who wishes can simply ignore the option to correct misspellings. It will make little difference to the results, as the following shows.
My two experiments included 150 participants, who recalled a total of 2,920 words, of which 45 (1.5%) were misspelled. 23 of those were practice words and 22 of those words were non-practice control words, for a net “gain” of one word for the psi hypothesis. Here are the results reported in my article (in which I corrected misspelled words) compared with the original program-calculated results (which ignores all unrecognized words). The score is a Differential Recall% score, which can range from -100% to +100%, with scores > 0 being in the “psi-predicted” direction.
Experiment 8:
Corrected DR% score = 2.27%, t(99) = 1.91, p = .029, d = .19
Uncorrected DR% score = 2.29%, t(99) = 1.95, p = .027, d = .20
Stimulus Seekers: Corrected DR% = 6.46%, t(42) = 3.76, p = .0003, d = .57
Uncorrected DR% = 6.50%, t(42) = 3.91, p = .0002, d = .60
Experiment 9:
Corrected DR% = 4.21%, t(49) = 2.96, p = .002, d = .42
Uncorrected DR% = 4.05%, t(49) = 2.86, p = .003, d = .40
As can be seen, Experiment 8 is trivially helped by the corrections; Experiment 9 is trivially hurt.
Additional observations: Half of the words used in this experiment are common words, as determined by “Frequency Analysis of English Usage” by Francis and Kucera (e.g., apple, doctor) and half are uncommon (e.g., gorilla, rabbi) Although Richard uses CTT and CAT as examples to illustrate the ambiguity of correcting misspellings, in fact only a few different words were misspelled by anyone, and they are among the uncommon words or commonly misspelled words in the list (e.g., potatoe for potato). So, Richard’s hypothetical example, notwithstanding, in practice the correction of misspelled words is actually very straightforward and unambiguous. “Intrusions,” words that aren’t in the original list, are also very easy to spot. (I can furnish the list to whoever wants to try a blind correction exercise, but I don’t want to publish it here lest it ruin future participants.)
Richard Wiseman 
Out of interest, what would be the problem with simply never scoring words which didn’t match exactly? Would that introduce some other form of bias?
It would seem to me that if they don’t match exactly then they don’t match – end of story!
It would introduce a bias against poor typing, I suppose. It’s very easy to mis-type things.
I’m glad Richard is trying to log all replica studies. It is important to get ALL hte data not just the studies confirming the hypotheses.
(Note: I have left in my typo above – it seemed appropriate to do so 😉 )
If they don’t match, surely they should be ignored. Re-coding them blind as you suggest would introduce a bias if the person doing the recoding is clairvoyant or has precognition and later sees the target words.
The posted replication seems to avoid this problem:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1699970
Those authors do a blind coding of imperfectly spelled words, though they mention that these are relatively uncommon and generally straightforward to recognize.
Unclear if Nick Day is serious, but the coder wouldn’t have to ever know which words came from which set for a particular participant; simply correct misspellings without knowing anything about conditions and then the computer supplies a final score.
My feeling is that you are absolutely correct to point out the possibility for experimenter bias, but it seems unlikely to account for the effect. Boring Type I error feels more likely.
I have a feeling the results won’t be replicated 😉
Anyway if ESP was so straight forward, wouldn’t it have a huge effect in real life? Just sayin’.
no, you wouldn’t notice its effect in situations like this – any “memory” (whether it’s past or future-based), your brain justs attributes to the past.
Flavio:
Playing woo-woo’s advocate, that doesn’t follow. Special relativity doesn’t have a huge effect in real life, but it’s pretty straightforward, at least mathematically.
Well, no, relativity isn’t that straightforward… I meant that this experiment is basically saying “tell me a word” and people are guessing right (statistically). That’s the claim.
It’s as if people would guess at flipping a coin. But that just doesn’t happen.
It doesn’t have a huge effect? I would say it effects everyone with a GPS.
Without SR, cathode ray tube TV displays would be dramatically out-of-focus. That has been the case since precision vacuum tube displays have been designed.
Hypothesis, test that can falsify it, all data available, test reproducable.
Wacky – but so is quantum theory.
All in all excellent science.
Peer-review is extremely important in science.
High on the checklist for a randomised, controlled trial such as this is double-blinddness.
That was a pretty big blunder. “Please ignore the big box in which the researcher makes his own results up by what he feels should have happened.”
This test seems to limit the pre-cognative effect to the words the were rehearsed immediately after the test; might it not be the case (and I am playing devil’s advocate here) that the test-subject might be better at remembering those words that will be more significant in their future life. Maybe a subject remembers LION and CAGE because in 5 years time they will become a lion-tamer.
This seems to be an argument that is impossible to disprove… Woo-believers who look hard enough will always be able to find a future-predicting connection.
Would this test work better with nonsense words? Or rare-groupings of words (FROG & CUBIC) that are unlikely to be ‘predicted’ from anywhere other than the study?
To Anonymous: ‘Unclear if Nick Day is serious’ — I don’t believe the ESP effects I mentioned do exist, but that’s beside the point. The experiment should be designed to measure effects only in the subject and not the experimenter, ‘just in case’ such effects exist.
I would think it best to retype the words, the point of the test is to see how much they remembered, and they did remember it, but simply misspelt it.
Re: poor typing: I was a student in the 90s and it was mandatory then to submit all work printed out from a computer. (For lots of good reasons.) Surely all students can type tolerably well by now? I really doubt that typing is a problem.
Besides this, there doesn’t seem to be a reasonable hypothesis here. If people are aware of future events, wouldn’t this be manifested somehow? I even used to go out with a psychic, and she predicted the numbers of the California lottery one week. Would have been more impressive if she’d bought a ticket as well, of course. I mean, there’s no good reason for believing that this talent even might exist, and plenty of reasons for assuming that it doesn’t.
you’re confusing an automatic effect with a consciously applied process – psychological information isn’t all of a single kind. if the effect is real, that doesn’t imply we can use it whenever we want to, in any situation, under all conditions. Even water doesn’t boil below 100 degrees.
The thing that bugs me most about this whole affair is how readily it gets equated with ‘quantum’ effects. It seems to me that many people think:
Quantum = weird
These results = weird
Quantum = real, therefore,
These results are quite possibly genuine.
It is of course bereft of logic, but numerous commenters raise this idea, and many of the sites that carried the New Scientist article were champing at the bit to point it out.
The association between these results and anything to do with quantum physics is a conjecture entirely separate from the experiment at hand. The word ‘quantum’ associated with some kind of paranormal event is inevitably a red flag in my experience.
I see a different association. People have a tendency to dismiss the strange and inexplicable out of hand. In other words, there is a tendency to say that ESP must not be real because it is so strange. Quantum is relevant only because it shows us that strange does not necessarily mean fake. It doesn’t necessarily mean ESP is real. It doesn’t that it is related to quantum mechanics if it is real. It means that we have to disregard what is “obviously” real or fake and realize that reproducible results constitute the only reliable arbiter.
In quantum physics experiments it appears that particles can either see into the future or travel in time.Amazing as it seems nobody can think of an alternative explanation for the observed behavior, at least one that isn’t even more far-fetched.
This suggests the possibility that it is possible to see into the future at a non quantum level. I don’t believe it myself but I can see the train of thought.
@Pvblivs: I understand your point, but this is expressly not the way believers in the paranormal see it. People who hold irrational belief systems are usually desperate for a ‘mechanism’ on which they can pin them (otherwise they could just offer them up as magic and be done with it). Thus, they have in the past invoked magnetism and electricity and ether and many other ‘edge science’ phenomena – not as an analogy, as you suggest, but as an explanation. You don’t have to go far to find the same kind of thing happening for quantum physics. In most cases it’s a very literal transposition; scientists can’t ‘explain’ quantum effects, and therefore what you’re seeing with our method/product is a direct effect of quantum physics. The net is bulging with examples.
@Glen: There is at least one other explanation for the ‘time travelling’ phenomenon in particle physics. It might be that we are experiencing an illusion based on our limitation of being able to experience reality in only 3 dimensions. In this case, as bizarre as it seems, time is happening all at once, just like the rest of our 3 dimensional world. We are simply unable to perceive it like that. Of course, this gives another ‘explanation’ for precognition (and a much better one I think) but it’s still not anything to do with the Bem experiments. My point is that proving the veracity of the results is one experiment, and linking them to any quantum effects (or any other explanation) is another experiment entirely. The logic ‘because something strange exists, other strange things are possible’ as suggested by Pvblivs, is not an explanation so much as a conjecture. If you understand it like that, there’s no problem, but unfortunately that’s not (in my experience) the way it’s taken in the world of pseudoscience.
@anaglyph & Glen
Just to throw my hat in with anaglyph on the whole quantum physics doesn’t necessarily show time traveling thing, I would say to Glen that it’s fallacious to say quantum physics show particles traveling in time. We don’t know why particles act the way they do in Quantum Mechanics. The way physics work at that level is so foreign to our way of seeing the world that it’s nearly impossible for our macro-scale brains to comprehend. You can say if you like that quantum physics suggests time travel, but there’s actually no evidence for anything except for the fact that particles on a quantum level act according to rules we don’t even come near understanding. Even if we can’t come up with a different argument for how particles act so strangely besides time travel or foreknowledge (and as anaglyph point out, there are other possibilities), to say that it can’t be anything else because we can’t think of anything else is an enormous argument from ignorance. Quantum Mechanics is this huge mess of stuff we don’t understand. To try to force in time travel or ESP or anything else we don’t see real evidence for strikes the same tone for me as creationists jamming in God as an explanation for abiogenesis.
@Glen “In quantum physics experiments it appears that particles can either see into the future or travel in time. […] This suggests the possibility that it is possible to see into the future at a non quantum level.”
Not in any that I know of. As far as I know no experiments have disconfirmed quantum theory and there is nothing in QT to suggest that possibility. From what I read of his work after I found out about these *ahem* strange experiments, it appears even Bem recognises that.
Interestingly, it turns out that even the fundamental structure of spacetime in our cosmos may have been chosen so as to confound the poor psi researchers:
http://en.wikipedia.org/wiki/Causal_dynamical_triangulation
http://www.perimeterinstitute.ca/index.php?option=com_content&task=view&id=551&Itemid=568&lecture_id=9134
TO: anaglyph
The reason people provide plausible mechanisms that might explain their data is essentially because this is the basis of Bayesian statistics.
Bayesian statistics is a kind of rival (but infrequently used) system to the regular system of statistics employed in the sciences.
Bayesian statistics can seem strange initially, since the Bayesian equation requires you to actually include a probability factor index for your a priori subjective estimation of the likelihood of the truth of the hypothesis being tested.
Your a priori estimation of the likelihood of the truth of the hypothesis will often relate to whether you can know of any theoretical mechanism that may underpin the phenomenon under investigation.
In this way, Bayesian statistics can sometimes give a more accurate probability result than regular statistics.
Thus offering quantum mechanics as a possible theoretical mechanism to underpin the phenomenon is entirely legitimate from the point of view of Bayesian statistics.
Plausible mechanisms, perhaps. Unfortunately choosing quantum physics as a ‘plausible’ mechanism happens in a lot of pseudoscience simply because it’s weird. The equation goes something like this:
My experiment is offering up results that are weird. Quantum physics is weird. Therefore, my experiment is obviously explicable via quantum physics.
The Bayesian interpretation (which I do understand) does not excuse sloppy reasoning. In fact, if anything it demands even tighter logical protocols because of the way in which it is deployed, and because of its capacity to be misunderstood.
As you say – quantum mechanics IS legitimate as a theoretical mechanism underpinning these results – IF such a mechanism can be posited in such a way as makes sense. Bem has posited no such mechanism. It is not good enough to have only speculation. That’s not science.
TO: anaglyph
Agreed that quantum mechanics (QM) is all too often cursorily invoked to justify any bit of pseudoscience, without giving any details of the mechanism. It is not adequate to point to quantum mechanics and say “Maybe an explanation for this phenomenon can be found there”. A more definite QM mechanism must be provided. In this case, in Ben’s experiment, it is not too difficult to provide a plausible QM mechanism.
In Ben’s experiment, what you see is an effect within the brain that purportedly breaks the everyday classical Newtonian laws on space and time: information known only later the future appears to be affecting what happens in the present. It appears that the “forward in time only” law of cause and effect has been broken, and there is an apparent backward-in-time causally appearing.
Now, there are aspects of QM that that also have this backwards in time feature. Unlike classical systems, quantum systems are not subject to the 2nd law of thermodynamics. The second law is what makes the everyday classical reality we experience run only forwards in time. By contrast, all quantum systems are time-reversible. So cause does not have to precede effect in a quantum system. Temporally speaking, QM is not one-way street of causality, but a two-way street.
In the case of the human brain, if there are quantum systems employed as part of the brain’s data storage (memory) and data processing (computation) apparatus, then sensory data that enters into brain in the future will determine the state of this quantum system at that time, and this future quantum system state may conceivably exhibit some type of backward causality effect on the state of the very same quantum system in the brain in the present moment.
In other words, both data from the present moment and data from the future moment are held in the same quantum system within the brain, and so may conceivably affect each other via this two-way quantum system causality.
However, the backward causality effect, if it does exist, is clearly very slight, as otherwise we would have observed it in normal everyday life more.
Daryl J. Bem does mention time-reversed QM effects in his paper.
Note that this proposed explanatory mechanism does not say you can sense the future in general, ONLY the future state of your own brain. If you were not shown those words in the future, and that data did not enter you brain via your senses, you brain would not have access to that information, and so you could not possibly exhibit any backward-in-time causality effect. This proposed QM mechanism says that at best, you can only perhaps get some information from the the future state of your own brain, but NOT from the future of the world in general.
What I would like to see done in the replications of Ben’s experiment is to try different time delays in different runs of the experiment. By time delay, I mean the time interval between point when the subjects type in the words, and the time in the future when these subjects are shown a random subset of some of the words. If there is a genuine backwards-in-time transmission going on, it is conceivable that this backwards transmission effect will get weaker if a longer time delay is set. I imagine if the subset of words were shown one hour in the future after the typing in, the strength of effect seen in the experiment would be diminished. And if they were shown say 24 hours in the future, the effect may be even further diminished.
If the effect does diminish with longer time delays, this would tend to support the idea that this is a genuine backwards-in-time data transmission. You may even be able to plot the strength of this backwards-in-time data transmission effect against the time delay interval, and see if the graph conformed to a standard mathematical function. Maybe the effect strength follows an inverse square law, for example.
It would also be good to see what happens if, when later showing the random subset of words in the future, more effort is made to embed those words into the subject’s mind (using say multimedia approach: big red letter on screen, plus an audio utterance of the word, plus say an image of what the word represents – for example, showing and actual image of cup in the case of the word “cup” ). If you really “etch in” that selection of words, does it increase the strength of this backwards-in-time transmission effect, or is there little difference?
You might further investigate whether this purported backwards-in-time transmission effect is boosted by relaxing the subjects before the experiment, say via a mild relaxing drug or supplement, or via meditation. The dietary supplement L-theanine, for example, is amino acid that has a very interesting relaxation effect: it has been shown to boost alpha wave activity in the brain. Might this supplement boost the the strength of this effect?
You might investigate whether the effect is disrupted by agitating the subjects before the experiment, with noise, angry voices, and the like.
You might investigate whether priming the subjects into a very logical mind state, by having say a logical and scientific person running the experiment, alters the subjects’ performance. Then you might see what happens is you prime the subjects into a more intuitive and/or trance-like mind state, by having say a warm and spiritual type of person run the experiment – this is the so-called “experimenter effect” on psi.
Often these fascincing experiments in precognition receive more criticism that they do support. Before they are dismissed, perhaps other researchers should actually first try to figure out how to AMPLIFY the effect. We would have never developed radio transmission, for example, if the very first electromagnetic transmission phenomena observed (inducing a spark at a distance) were dismissed.
If there is a backwards-in-time data transmission going on within brain, it is certainly very feeble. We should consider what can be done to boost the signal, so that we can better study it.
I can think of dozens of ways that you might prime the subject and their brain to achieve a better performance in these type of experiments. These experiments really need to be approached as an engineering problem: to be tinkered with and optimized. You perhaps need to stoke up a stronger, more robust quantum state in the brain, which may then yield better performance in these time-reversed data transmission experiments.
Having said that, it is always extremely important to keep a skeptical view, because nearly all strange phenomena turn out to have a simple explanation, or arise from experimental error – or even a result of human deception! You have to consider the myriad ways that experimental error, etc, might creep in and invalidate the results of these precognition and similar experiments, before you can even begin to believe that something unusual might just be happening.
There is such a strong desire in many people to believe in or want to experience transcendental phenomena, that these people will wooly-mindedly ascribe more or less anything unusual to being transcendental (lights in the sky, crop circles, or the latest bit of nonsense: “orbs” on of video footage, otherwise known as lens flare to photographers). I hate this wooly-minded stuff, as it clouds genuine research.
This is why the skeptical stance is so important – and probably offers the correct explanation in at least 99.9% of anomalous phenomena.
My last above post should have been posted under my online name of “G”, but I accidentally posted under my other online name of “Hip”.
However, both are in fact me.
He does. But he doesn’t give any plausible explanatory mechanism as you suggest he does. Just speculation.
Indeed. But we see no evidence of this effect in our normal world, and this is something that currently puzzles quantum physicists and mathematicians. So Bem is asking us to accept two things; that he has found the first physical evidence of this happening, and that it supports his very contentious findings. When I see these kinds of unlikely events being used to shore one another up, my skeptical antennae quiver a little bit more than usual.
And therein lies the beauty of science. I am skeptical of Bem’s work. However, my mind will be instantly swayed when I see results coming back that give weight to his hypothesis. So far, none have.
If there is anything at all to Bem’s ideas, then he faces a monumental task. His results are so insignificant that he must design experiments that tease his alleged tiny deviations from chance into something that is far more chewable. Right now, even if he is onto something, that something is barely discernible in the noise floor of his experimental protocol. On balance, this makes me think there is an error of method somewhere.
But, as always, I am willing to be proved wrong.
I’ve just read through the first two experiments of the Bam-pdf in which the human picks left or right then the computer randomly generates left or right. My statistical capabilities have 25 years of rust so I’ll accept that there’s nothing to challenge in the data and that one part of one of the tests showed a significant deviation from normal, 53 percent is more than half.
My conclusion would not be that humans can see the future but that computers are psychic and guessed 53 percent of the human choices (unless they cheated by looking at the human response before they generated their random guess).
oops, I meant BEM-pdf of course.
Did the blind controllers spot and correct it?
The scoring seemed to be one of the weakest points to me anyway. For example, the scoring forumla gives a higher weight to a session with more remembered words, but completely ignores the misses. I wonder if the outcome would be different if the scoring would also take the non-remembered words into consideration. For instance, are there fewer non-remembered words in the target list? How sensitive are his results to the scoring formula used? Or to not throwing away the data that is present in the non-remembered words?
These sorts of studies invariably take some action which is impossible to calculate odds for, to run their test. Often it is one person thinking of a scene, and another making a sketch of it. Judging if they are the same is totally subjective = “Ahhh, they both had an animal in them.”
Also it is clear that the people who invoke Quantum Mechanics have NO understanding of Quantum Mechanics – except some platitudes they’ve heard about it.
Well, I admit that I haven’t looked at in detail, but from what I recall the point was to take existing proven psychological effects where past experience can affect future performance and see if it works the other way around. Thus, the presumption would be that they followed standard procedures for detecting those phenomena, which presumably were vetted to death for those proven effects to get to be proven. Thus, looking at what they specifically did and nitpicking it is a waste of time. What you need to do is determine how their procedure differs from the standard procedure in those cases, and see if that could introduce more positives.
Otherwise, your examinations would essentially be calling the ORIGINAL methods — and therefore effects — into question as well.
This seems like a trivially easy thing to just fix; no need to speculate. I’d be willing to run a double blinded coding.
Email me a list of the incorrect words and I’ll have ’em coded.
Prof. Bem –
1. I can run the spelling correction, and you’ll be welcome to use my results. Please email the relevant data to bem@saizai.com.
2. Any ETA on anonymizing and releasing the rest of your data for reanalysis? It’d be good to have independent review.
3. Any particular reason for choosing 1-tailed p rather than 2-tailed?
It seems somewhat more plausible to me that precog would facilitate rather than hamper recall, but since there’s no good theoretical or empirical basis on which to make this claim, you’d be somewhat sounder to use 2-tailed.
This would change your best p values for above to:
#8 0.054 (n.s.)
#8 SS 0.0004
#9 0.004
… which would wipe one erstwhile significant result but leave the other two with reasonable margin.
4. On encryption: perhaps you meant that only you know the encryption key?
Keeping secret the encryption *formula* – aka “security through obscurity” – generally never works.
If you’d like more information on how to securely encrypt databases, just email me; I’d be happy to help. I’ve done it professionally in large production environments with sensitive data before.
Thanks!
Daryl Bem. The guy who’s kid said “Daddy, that boy said that everybody has a penis, but only girls have long hair”. And now with an ESP study, which is to psychology research as virtual particles are to particles. Not that a lot of psychology research isn’t like that; but raising a kid certainly isn’t like that.
Moral: cute anecdotes are the primary value produced by psych departments. (That moral has p=0.0866 – which, for a blog comment is mega-significant.)
I’m a bit late in reading this, but it seems to me that there is a really simple way to retest the validity. Simply use the program to display the words and have the subjects either write what they remember on paper or say the words into a recorder. The subjects should have no issue misspelling the words on paper or saying them aloud.
I do, however, like the use of a program that stores all entries. Since these are simple words that are difficult to misspell, it wouldn’t hurt to instruct the subjects to make sure they take enough time to spell the words correctly. Though this is also an issue because a couple seconds of ensuring spelling is correct can cause the loss of recalling another word or more.
Regardless, I look forward to hearing how your replication goes and hope you post it here.
Nice thanks u
http://www.citysiam.com/
Free application, software, setup, install, webboard, it, computer, antivirus, security, travel, tour, north, central, east, south, northease, national park, park, mountian, waterfall, beatch, sea, island, thailand, tourism thailand, torism, online, hotels thailand, travel information, thai tour, siam, citysiam, iphone, ipad, ipod touch, android, windows mobile, symbian
Richard,
don’t do it! You will replicate the experiment but not the results, Bem will criticize the changes you made to the protocol you will defend your choices he will defend his -> a long drawn out heated discussion will ensue.
Look at http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1456528
same story orders of magnitude better data and methodology.
Yeah don’t do it! You wouldn’t want to gradually replicate and improve experimental protocols and engage in intellectual debate until we finally discover the truth of what’s going on! That would be far too close to SCIENCE!
nice post…
please visit this..
klik
thanks
This week’s SGU podcast addressed the paper:
http://www.theskepticsguide.org/archive/podcastinfo.aspx?mid=1&pid=281
They make good points against Bem’s claims. I tend to agree with them!
I analyzed the results of Bem’s 8th experiment, and I believe I have pretty clearly demonstrated that the results of it are well within the expected range of outcomes that random chance would generate. My findings running 100 trials of 100 randomly generated participants (through the use of pseduo random number generators) in the experiment indicated that in 34 of the 100 trials there were higher DR% values than the 2.27% Dr. Bem found. I did this with some pretty basic calculations using Microsoft Excel.
The results of my work can be seen at: http://blauney.blogspot.com/2010/12/psigh-response-to-daryl-bems-work.html
I welcome any comments on my blog.
Bem’s research thoroughly disassembled by Skeptical Enquirer. It’s pretty devastating.
http://www.csicop.org/specialarticles/show/back_from_the_future
Yes. Good to see a critique placing this silly and tiresome pathological science in context: something conspicuously absent from this – http://dl.dropbox.com/u/1018886/Bem6.pdf – otherwise excellent and equally devastating disassembly.
I was surprised to see the sloppiness of Bem’s process. Well, not really surprised at that, I guess – more at the indiscriminate reporting of New Scientist and others. No mention of the flakey experimental procedure in a grab for sensational headline and a few more bucks on the news stand.
Heh! New Scientist…
http://golem.ph.utexas.edu/category/2006/09/a_plea_to_save_new_scientist.html
“This week’s SGU podcast addressed the paper:
http://www.theskepticsguide.org/archive/podcastinfo.aspx?mid=1&pid=281
They make good points against Bem’s claims. I tend to agree with them!”
The points might have been more convincing if the speakers hadn’t had so much trouble in remembering the most basic details of the study they were discussing!
Ha! Hilarious! Live by the p-value, die by it (or become a laughingstock). This “science” is an indictment of null-hypothesis statistical testing – one of the worst things that ever happened to (mainstream) psychology, second only to the abysmal state of its concepts. The latter is, of course, related to mainstream psychology’s insipid interpretation of Bridgeman’s operationism. Other than that I have no strong opinion. 😉
By the way, the “Glen” above is not the same “Glen” that posted earlier comments. I (the guy whose post is immediately above) will now go by Glen S.
First, why should the mistyping be corrected at all? We should assume that typing mistakes are randomly distributed between psi and non psi participants, right? It seems just a way to mess up the results, post hoc.
Second, I was always taught that no hypothesis can be offered without some theory, or explanatory mechanism, that results in the hypothesis. How does Bem get to offer a completely disconnected hypothesis with such popular appeal? It seems like an effort to get noticed.
Dr. Bern: Great. Just great. So if ESP is real, do us a favor. Make all the time spent on this worthwhile.
Next time warn people in Australia that a tsunami is coming, giving day and time. Warn people in the Middle East about the next suicide or roadside bomber. Find Madeleine McCann or Natalee Hollaway by giving exact location. Tell us tonight’s lottery numbers.
I think that would be really of great value. Just a suggestion for the future.
Cheers.
Hi,
It will be interesting to see a comparative analysis of studies which attempt to replicate this effect.
It is possible, depending on what equipment or environment was used, that this somehow affected the results. Why it would be in favor of the correct answers if they were indeed random, I do not know. For example, technically speaking, the images should appear on one screen or another and the participant guesses which of these will display said image.
Even if one side was favored (say monitor was somehow slightly different, or that position in the room affected choice), even if they favored the right versus left side, that would likely hurt them (if images were randomly displayed).
If the study is replicable, there is likely some affect going on, but I doubt it is ESP. Bias, etc may be the answer.
Adam
Fortasse.com
Dear ESP researcher: I have finished a book on dreaming the future and it is a life work. I need help to publish it. Do you have any contacts who could help? Thanks
Maybe I’m crazy… But I’ve always had dreams of events, even had dreams about medical cures and what those cures are, what makes the universe tick, conflicts, deaths, air/land/sea accidents and my own death images.
Strange things have always happened around me and those close to me. I have felt that some guiding force surounds me and these images/impulese are a survival course for my betterment.