Tag Archives: Replication

Understanding p-values via simulations

As I mentioned in an earlier post, p-values in psychological research are often misunderstood. Ask students (and academics!) what the definition of the p-value is and you will likely get many different responses. To jog your memory, the definition of the p-value is the probability of observing a test statistic as extreme—or more extreme—than the one you have observed, assuming the null is true. But, even with this definition in hand, many struggle to conceptualise what the p-value reflects. In this blog post, I take inspiration from a lovely paper I have recently read that advocates using computer simulation to understand the p-value. Continue reading

Tagged , ,

Replication: the most important statistic

Forget pretty much everything you have heard in statistics classes about using statistical inference to interpret whether an experimental effect is ‘real’ or not. Want to know if an effect is real? Replicate, replicate, replicate.

Only if an effect replicates—preferably in a different lab, by different researchers—can the scientific community start to believe that the effect is real.

Replication is thus the most important statistic. Period.

But, psychology as a discipline has notoriously eschewed publishing failed replication attempts. So it’s great to see that the topic of replicability in psychological science has been given center-stage in a special issue of the high-impact journal “Perspectives on Psychological Science”: http://pps.sagepub.com/content/7/6.toc?etoc

Cover image expansion

There are many exciting articles in this special issue—not least an article that I contributed to with the Open Science Collaboration (http://pps.sagepub.com/content/7/6/657.full.pdf+html), which outlines the OSC’s project aiming to assess the proportion of studies in psychology that are replicable.

Given the importance of this issue to all psychologists (especially given the ‘year of horrors’ that faced psychology as a science last year – http://www.ejwagenmakers.com/2012/Wagenmakers2012Horrors.pdf) I hope all readers find this of interest.