magical things can still happen in used bookstores.
i was killing time in one and came across a book by richard feynman called 'the meaning of it all'. i noticed that the first essay was called 'the uncertainty of science'. i bought it. it was the best $4 i've spent in a while.
it is tempting to just re-type the entire essay here, but in the interest of trying not to violate copyright, and trying to contribute something of my own, i will share some excerpts and reflections.
first, feynman on p-hacking:
'it is necessary to look at the results of observation objectively, because you, the experimenter, might like one result better than another. you perform the experiment several times, and because of irregularities, like pieces of dirt falling in, the result varies from time to time. you do not have everything under control. you like the result to be a certain way, so the times it comes out that way, you say, "see, it comes out this particular way." the next time you do the experiment it comes out different. maybe there was a piece of dirt in it the first time, but you ignore it.'
in psychology, we don't have literal pieces of dirt messing up our experiments, but we have the other kind. we sometimes call them pieces of &*$%. and then we call them outliers and drop them from our studies. it is kind of amazing that feynman was talking about how motivated cognition can corrupt the scientific process. if a physicist in 1963 can recognize this, then social/personality psychologists in 2014 surely must.
next, feynman makes an interesting observation about disagreement among scientists:
'the relations among scientists were at first very argumentative, as they are among most people. this was true in the early days of physics, for example. but in physics today the relations are extremely good. a scientific argument is likely to involve a great deal of laughter and uncertainty on both sides, with both sides thinking up experiments and offering to bet on the outcome. [...] many sciences have not developed this far, and the situation is the way it was in the early days of physics, when there was a lot of arguing because there were not so many observations.'
psychology is much younger than physics. it is forgivable that we are not yet at their stage. adverserial collaborations are a good start, but eventually we need to get to the point where adversaries regularly settle things the adult way: bets and laughter.
next, feynman on guessing vs. knowing:
'i come now to an important point. the old laws may be wrong. how can an observation be incorrect? if it has been carefully checked, how can it be wrong? why are physicists always having to change the laws? the answer is, first, that the laws are not the observations and, second, that experiments are always inaccurate. the laws are guessed laws, extrapolations, not something that the observations insist upon. they are just good guesses that have gone through the sieve so far. and it turns out later that the sieve now has smaller holes than the sieves that were used before, and this time the law is caught. so the laws are guessed; they are extrapolations into the unknown.[...]
it would have been unscientific not to guess. it has to be done because extrapolations are the only things that have any real value. [...] knowledge is of no real value if all you can tell me is what happened yesterday. it is necessary to tell what will happen tomorrow if you do something--not necessary, but fun. only you must be willing to stick your neck out. [...] it is better to say something and not be sure than not to say anything at all.'
there are many things i like about this passage. one is that laws (or even conclusions) are not something that the observations insist upon. too often in discussion sections of papers i see people claim conclusions and implications from their results, as if their data insist upon those conclusions. observations insist upon very little, except themselves, and sometimes not even that. that doesn't mean we shouldn't stick our necks out. but we should be aware and honest that this is what we are doing when we draw inferences from our observations. say something, but don't pretend that your 74 participants' worth of data mean that it has to be true.
finally, on living without knowing:
'so what we call scientific knowledge today is a body of statements of varying degrees of certainty. some of them are most unsure; some of them are nearly sure; but none is absolutely certain. scientists are used to this. we know that it is consistent to be able to live and not know. some people say, 'how can you live without knowing?' i do not know what they mean. i always live without knowing. that is easy. how you get to know is what i want to know.'
if you cannot be happy without knowing, then i don't think you're cut out for science. some people need to know. that is fine. but those people should not become scientists, and especially not social scientists. predicting human behavior is not for the faint of heart. you must be prepared to be wrong, and wrong again, and again, and again. richard feynman seems to have been an extremely happy man. knowledge is not necessary for happiness.
'knowledge may have its purposes
but guessing is always
more fun than knowing'
-w. h. auden