Taking Hope Out of the Equation

December marks the time of year where final year undergraduates at my institution present their proposed research programme via a conference poster. At their allotted time, two faculty members will visit the student’s poster, and ask the student to talk them through their planned project (just as if you were at a conference). After this 3-5 minute speech, the floor is open to the two faculty members to ask questions.

This is always an enjoyable assessment from the staff point of view (and even the students enjoy it!). However, this year I noticed a worrying trend from pretty much every student I spoke to.

When discussing their proposed analysis and predicted results, most students said something along the lines of “I hope to find…” or “I’m hoping that the results will show…”.

Hope is a dangerous concept in science; we should work to rid all of our hypothesising and scientific investigations from hope. Hope is the slippery slope that at best leads to confirmation bias and a myopic scientific outlook, and at worst outright data fiddling and fraud. (As a quick aside, I don’t blame the students for having this hope; as we will soon see, hope is likely rife across academia. And, I don’t for one second suggest that the students are likely to fiddle data or commit scientific fraud!)

Nature is there to be discovered as she is, not as we hope to find her. Researchers should be as independent as they can from their hypotheses and theories; although it is very hard sometimes to dissociate yourself from your ideas—especially in academia where your career can be catapulted based on your “new” and “innovative” ideas—you must try if you wish to be an objective scientist. You should not even blink even if the theory that you have held dear to your heart for many years has just been falsified by a newly published set of studies.

It’s difficult, and I’m not pretending it’s not. Without ideas, a researcher is nothing. We all want nothing more than to discover a really neat novel experimental effect that will change the (psychological science) world. But—and it’s worth repeating—hope can be a dangerous thing.

The Problem

Hoping to find certain experimental outcomes might even lead to questionable research practices, which according to a recent paper are rife among psychological scientists. Didn’t quite get the results you expected? Don’t worry; there are plenty of things you can try to present anything as significant.

Let’s form a concrete example. Let’s say you are interested in the effect of a new drug on response time. You plan to give one group your new drug and the other a placebo. You “hope” to show that your new drug produces faster response times, and can thus be declared as a wonder drug to boost alertness and cognitive performance the world over. (Imagine what a superstar you might be! No, forget that; think about how RICH you will be!)

You run your test and find that, disappointingly (the result of failed hope), response times are equivalent between the two groups. Aren’t you even a LITTLE tempted to look closer at the data, and see if a different outlier criteria might uncover your hoped-for effect? To see if there aren’t a few subjects’ data that “conveniently” don’t look quite right, and removal of which make your results fall the way you had hoped? Sure, you can convince yourself that those 3 subjects in the drug group are outliers; after all, your a priori outlier criteria was likely too liberal.

Even better, decide post-hoc that actually you were interested in MALE performance only, so get rid of those pesky females’ data (don’t worry that the females’ data were showing no effect of the drug; after all, who will know?).

This is the path that hope can lead you down. It’s a dark path, prevailing with demons ready to drag you into the bowels of scientific hell.

The Solution?

Be objective. Take pleasure in discovering nature as she really is, not as you wish to find her. If you have a theory of how something operates, be the one to throw every empirical result at it to see if it remains. Attempt to falsify your own hypotheses. Take hope out of the equation.

I leave you with the wise words of Richard P. Feynman, my scientific hero. These words adorned the first page of my PhD thesis, and I have them pinned to my office door, acknowledging them every day I am researching. If you live your research life by these words, you will not go far wrong.

Feynman

“…if you are doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated. Details that throw doubt on your interpretation must be given, if you know them. You must do the best you can—if you know anything at all wrong, or possibly wrong—to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it… ….although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work. Science is a way of trying not to fool yourself. The first principle is that you must not fool yourself, and you are the easiest person to fool.”

 

Advertisements

2 thoughts on “Taking Hope Out of the Equation

  1. Sian says:

    Also worth noting the recent case of Diedrick Stapel who hoped to find certain things http://news.sciencemag.org/scienceinsider/2012/12/social-psychologists-lash-out-at.html and the infamous Cyril Burt.

    • Jim Grange says:

      Thanks for your comment!

      Absolutely! I was going to mention Stapel, but I thought his case has been done to death in the media. He certainly was a naughty boy, though! This sort of thing is only going to get more common with the pressures of academia; I’m not condoning the behaviour at all, but I can understand why it happens. Issues surrounding this type of malpractice (the stage before outright fraud) were the focus of a recent special issue of Perspectives on Psychological Science. (http://pps.sagepub.com/content/7/6.toc?etoc).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: