Enter your Email:
Preview | Powered by FeedBlitz

« submit your papers to SPPS! | Main | Guest Post by Laura Scherer »

Comments

Sam Schwarzkopf

Great post. Considering that you have mentioned the cosmic background radiation result I don't know why we should single out psychology here. The media are full of doubts about science - just think of climate research or evolution. Another interesting point about the expansion of the universe is that I have a hunch that the theory is probably still true. It's just that this "smoking gun" type evidence for it didn't turn out to be.

Anyway, your conversation with the journalist sounds like interviews normally go with me as well (and then they can still somehow manage to misquote you).

You are probably right that we can trust the average person to get that science is a gradual homing in on the truth. But all to often you hear people about how science has proven this or that. I think it doesn't hurt to remind everyone (including scientists) that this isn't how it works.

L.J Zigerell

For what it's worth, here are questions that I ask when evaluating research:

1. Did the researchers preregister their research design choices so that we can be sure that the research design choices were not made based on the data? If not, are the research design choices consistent with the choices that the researcher has previously made in other research?

2. Have the researchers publicly posted documentation and all the data that were collected, so that other researchers can check the analysis for errors and assess the robustness of the reported results?

3. Did the researchers declare that there are no unreported file drawer studies, unreported manipulations, and unreported variables that were measured?

4. Were the data collected by an independent third party?

5. Is the sample representative of the population of interest?

Chris C.

Science is not "homing in on the truth," nor is it "getting closer and closer to the truth." If that were so, how would we know we were making progress?

Because we don't already know truth (if we did, why would we need science?), we cannot know if we are getting closer to it. There is no treasure map with waypoints.

Science has to be judged by its efficacy, by solving problems, by DOING things like enhancing public health, getting to the moon, decreasing suicide rates, lowering human costs, by reducing uncertainty and affording explanatory coherence.

This is not trivial. How can we judge scientific practice if we cannot measure scientific progress? Since truth is unknowable, and suggesting that we are moving toward an uncharted and unknowable destination, we must measure ourselves with a yardstick that does not rely on airy-fairy concepts.

In such a world, replication is not evidence of truth (and failure to replicate evidence of untruth), but rather it evidence of the power or effectiveness of the idea.

And so we are, as Simine suggests, working toward greater effectiveness, and the pathway is not straight. But the move toward a better position is almost always met by public setback--there is no Obergefell without DOMA and Prop. 8 first--public humiliation is almost always part of social change. It passes.

Sam Schwarzkopf

Chris C: Science is not "homing in on the truth," nor is it "getting closer and closer to the truth." If that were so, how would we know we were making progress?

I think this question is based on a misreading. I tend to liken scientific progress to a model fitting procedure. Essentially this is what science is, trying to come up with models to explain a chaotic world.

We know we are making progress if we increase the explanatory power of our models while minimising their complexity. No one scientist can know if their models are really getting closer to "the Truth".

Of course, sometimes any algorithm can get stuck in a local minimum that it may be hard to get out of. This means the process can go massively astray (like, say, social priming research :P). It takes some additional energy to get out of those holes. Homing in does not mean that the only direction is always downhill.

But in the long-term the process is doing circling the drain of the true explanation. If it didn't we probably shouldn't even be trying it at all.

Andy DeSoto

Love the post -- thanks for sharing. Not sure I agree, but I am glad that you feel that the reproducibility discussion is making us look good. "It's not news to anyone that we are a young science" -- that's interesting to me. I wonder if others would agree.

Would love to collect some data on public perceptions of psychology at some point. Thanks again for writing.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)