Enter your Email:
Preview | Powered by FeedBlitz

« unbelievable. | Main | buckets of tears »


Eric Schwitzgebel

This is terrific, Simine! I think there's a lot of dubious stuff published by people working in approximately this way. I've been thinking a bit recently about what we should expect a psychological sub-literature to look like if there is a non-effect underneath but a positive bias on the part of the researchers.

First effort:

Ryne Sherman

Great post Simine. When I first encountered Bem's paper I was (quite frankly) appalled. But other's had directed me to read it, and I saw no criticisms in the literature about it, so I thought "well, that just must be the way things work."

Then (a few months later) I read a wonderful paper by Arina K. Bones (a great psychologist in her own right) and Navin R. Johnson (http://pps.sagepub.com/content/2/4/406.full.pdf+html). On p. 409 the paper reads "We did not know whether to predict a rapid or slow change in animate associations because of conflicting existing evidence about the malleability of implicit cognition. The fragility of the hypothesis did not pose a difficulty because we proposed a priori to write the article as a good story and as if the ultimate results were anticipated all along (Bem, 2003). After all, this is psychology--only actual scientists would want to read how the process of scientific discovery actually occurred."

Exactly. And now 7 years later, your post is the second time I have seen Bem's advice criticized. I am almost sure to have missed another criticism of it here or there somewhere, but I am still amazed that this is considered mandatory reading nay, advice, for graduate students in psychology.

I found this quote from Albert Einstein on the internet one day (i.e., I am not giving accuracy to its attribution; nonetheless I find it useful): "Anyone who doesn't take truth seriously in small matters cannot be trusted in large ones either."

Sometimes science is full of boring details. So be it.

Brent Donnellan

Hi Ryne,

I would just like to plug a paper by Norbert Kerr from my department that took issue with some of the advice in the older Bem chapter about HARKing. (It would have been nice if Bem cited it in the 2003 chapter!)


Lynne Cooper

I too used to assign this article & stopped some years ago for the exact same sorts of reasons! Thanks for taking the time to put these concerns in print.

Liz Wuehrmann

Nice job, Simine. What was said above plus the balance between forcing a twisted story on one hand vs. bald presentation of narrative-free facts.


I can understand how preregistration has become popular because of the ills of post-data story-telling, but how does this make Bayesianism popular? The Bayesian algorithm permits beliefs and hunches to be introduced into the data analysis; many allow the prior to be altered based on the data--thereby fitting in swimmingly with your description of Bem's methodology. Perhaps you mean it encouraged the use of low enough Bayesian priors in order to criticize Bem. I think this is the wrong way to mount a methodological critique, and those who proceeded this way actually weaken their critique. One ought to be able to show what flaws were permitted such that the alleged evidence for H in fact does not constitute a sufficiently critical probe of H.


Like it or not, Bem's advice has been followed by many of today's most successful scientists.

Science is not an a-political field and if scientists don't 'sell' their results, then they will cease to get funding and even published in high quality journals.

If you question the ethics, then we need to change the incentive structure - the way scientists are funded to do their research, otherwise this behaviour will continue.


Bem's advice is precisely why full datasets need to be made available as a pre-requisite alongside any published paper, so that the data can be re-analyzed by disinterested parties to see whether it truly is a jewel, as Bem suggests, or instead is merely a polished turd as can so often be the case.

The comments to this entry are closed.