Monthly Archives: November 2012

Livin’ in a Box—A Boxplot, that is!

Before I do any form of statistical analysis on my data, I always like to get a good look at it. Getting familiar with the data before conducting inferential tests is a good way to understand the data.

Data exploration can take on many forms; most common is perhaps to produce means (or medians) of each group you are investigating and plot them.

This is, of course, fine. However, of late I have become interested in looking at the distributions of my results rather than just an estimate of their central tendency. Looking at just the mean—a point estimate—”throws” a lot of information away. Did all participants in each group perform equally? Were there any participants who performed substantially differently from the group (so-called “outliers”)? Continue reading

Tagged , ,

Tackling Students’ Concerns

A few days ago, I posted a Wordle word cloud which highlighted students’ thoughts on research methods classes. (If you are yet to have the pleasure, you can read the post here.) This showed overwhelming support for the notion that they are ‘boring’, ‘confusing’, and just about ‘maths’. What can we do about this?

Instead of approaching this top-down, I thought I would ask the very people who raised these concerns. So, in two lectures last year, I asked students what they would like to see implemented in research methods lectures and classes. Again, I collated the responses and pushed them through Wordle. The two word clouds below represent the opinions of Level one students and Level two students, respectively.

These clouds aren’t quite as efficient as the one last week, as students didn’t just write one word each; instead, responses came in in sentences. But, Wordle is still able to pick out the main themes common to most responses.

What jumps out at me is that students are craving examples. We give plenty of examples in lab classes which supplement lectures, so I interpret this as referring to examples in the lectures themselves. Interesting idea!

Linked to this, students are also after more interaction. This is likely true of most lectures: I recall my days as an undergraduate fighting sleep as the monotone orator at the front droned on. I particularly empathise for those students who suffer from maths-anxiety (a typical affliction!), who—after a dribble-ridden lecture experience—might be worrying they suffer also from narcolepsy.

It doesn’t have to be this way!

What can lecturers do about this? The first step, I contend, is to appreciate the problem and listen to students’ opinions. It’s all too easy to think student-disengagement in research methods classes is the fault of the student. “They should be interested in this stuff! I am!”. It’s our responsibility to make student engagement happen.

Going forward, I aim to break the lectures up with “hands-on” examples. Give the students a small data set to work on; give them a discussion point to talk to their neighbours about; use gap-notes (lecture notes which have gaps in that students must complete from the information you provide in lectures).

What can students do? Engage with your tutors; tell them your honest opinion and inform them what you would like to see happen in class. Do you also want more examples/interaction/practice? Say so! You might be surprised; lecturers aren’t all THAT bad. (Honest!)

Replication: the most important statistic

Forget pretty much everything you have heard in statistics classes about using statistical inference to interpret whether an experimental effect is ‘real’ or not. Want to know if an effect is real? Replicate, replicate, replicate.

Only if an effect replicates—preferably in a different lab, by different researchers—can the scientific community start to believe that the effect is real.

Replication is thus the most important statistic. Period.

But, psychology as a discipline has notoriously eschewed publishing failed replication attempts. So it’s great to see that the topic of replicability in psychological science has been given center-stage in a special issue of the high-impact journal “Perspectives on Psychological Science”:

Cover image expansion

There are many exciting articles in this special issue—not least an article that I contributed to with the Open Science Collaboration (, which outlines the OSC’s project aiming to assess the proportion of studies in psychology that are replicable.

Given the importance of this issue to all psychologists (especially given the ‘year of horrors’ that faced psychology as a science last year – I hope all readers find this of interest.


What do students think about statistics?

Students typically abhor statistics and research methods classes. It’s one of those facts of life: taxes, hating stats, and death (the latter two aren’t—despite your fears—related). So when I was asked last year to develop a new Year 2 research methods course for my department, you can understand my trepidation.

However, as I am odd and I love statistics and research methods, I was very keen. To boost the potential success of the module, I was interested in getting to understand what it is about statistics classes that students don’t like. This is an ongoing project for me—below is just stage 1—but I thought it best to start by asking students their opinions.

What words come to mind when they think of statistics and research methods? I was interested in getting to know what pre-conceived ideas and concerns students bring to their first day of class.

So, using my Year 2 students (N=104 respondents) as guinea pigs, I asked them to write down one or two words that they associate with research methods.  This was in lecture 1, before any aspect of the module had been discussed.

Being a stats nerd, I am always on the lookout for interesting ways to present data, and I had recently discovered word clouds. Word clouds present words with their size related to its frequency in the text. (For an excellent online generator, try You can feed in any block of text, and Wordle will present you with your word cloud; the larger the word, the more frequent it occurs. Below is the result:

As can be seen, most students view this topic as “boring”, “confusing”, “dull”, and “difficult”. (We will steer clear of the one who finds it “sexually arousing”; it wasn’t me, honest.)

This perhaps is not a surprising result, but it highlights the pre-conceptions students bring with them from prior modules; thus, part of the battle is overcoming these pre-conceptions. I really see these words as barriers to students learning in statistics, and tackling them will likely increase the success of student learning.

What are your thoughts?

P-off and leave me alone

Psychology has a problem. It’s not a piffling problem, but a pronounced problem; one pregnant with perpetual pain and potential to induce poor health. That’s right—p-values!

P-values are the end result of most common statistical analyses in psychology, and they “allow” the researcher to determine whether their effect is statistically “significant”; in psychology, we use a criterion of p<.05 as our marker of significance. (As an aside, one surprisingly common mistake students make is confusing'<‘ with ‘>’; ‘<‘ means less than. If x < y, we are saying x is less than y; just think that the arrow needs to point to the smaller value. So, x > y would mean that x is greater than y.)

Why is it problematic? There are many reasons—it is a test of a null hypothesis, but the null is never really true; it depends on sample size; it doesn’t answer the questions we, as researchers, are interested in—but I want to focus on just one: the p-value—primarily its interpretation—is severely misunderstood.

Consider this question: how would you answer it in an exam situation?

Which of the following is a definition of p?

  1. The probability that the results are due to chance; i.e. the probability that the null is true.
  2. The probability that the results are not due to chance; i.e. the probability that the null is false
  3. The probability of observing results as extreme (or more so) as the ones you have obtained, if the null hypothesis is true.
  4. The probability that the results would be replicated if the experiment was conducted a second time.

You might think that 1 is the correct answer…well…give yourself a pat on the…HEAD!

Don’t feel bad. I ran this question in class to level 2 undergraduates, and all got it wrong. I was also approached by the internal exam board at my university who suggested that I had marked the wrong answer as correct in the end-of-semester exam. (Don’t believe for one second this is unique to my department; give the above question to professors in your department and count on one hand how many get it correct.)

Yes, this problem runs deep, but it’s not surprising; I have in front of me several undergraduate statistics textbooks, and only one—yes, ONE!—gives the correct definition.

The p-value is the probability of observing results as extreme as yours—or even more extreme—if the null is true. Although most people are surprised by the answer (were you?), it makes perfect intuitive sense once you think about it.

Imagine we are interested in whether caffeine improves cognitive function (having mainlined some coffee this morning just so I feel half-human, this is a pertinent question); let us also assume that the true effect of caffeine on cognitive function is real—that is, caffeine does improve function. We give one group of participants a caffeine pill, and the others a placebo, and then expose them to some arbitrary test of cognitive function. We get the following means (standard deviations in parentheses), assuming that in this arbitrary case a lower mean signifies better performance:

  • Caffeine—26.75 (15.26)
  • Placebo—38.55 (14.07)

We run our tests on the two means—for those interested, it would be an independent-samples t-test—and get p=0.008.

With our correct definition of p in-hand, we can state that the probability of observing the scores we have—or scores more extreme—if the null hypothesis is really true (that is, if caffeine actually has no real effect) is 0.008; i.e., if there really is no true effect of caffeine, then it is very unlikely that we would obtain the results that we have. We can therefore suggest that caffeine does have an effect on cognitive function (if nothing else, it surely helps with writing blogs about p-values at 8am on a Saturday). In psychology, because the p-value falls below the criterion of 0.05, we would declare this a significant result. Bingo!

There are many other things wrong with the p-value, and there are always murmurings and movements in the literature to abolish the p-value once and for all. I agree with these sentiments. In the meantime, though, I think communicating the true interpretation of the p-value will help researchers understand their data better, until a viable, accessible, alternative comes along. I will discuss some alternatives through the lifetime of this blog.

Now, back to my coffee…

Tagged ,

APA-style graphs in Excel

So, you have some neat data and want to bung them in your report with funky colours and whacky axes? Sorry to burst your excitable bubble, but APA have strict guidelines for graphs, too. This short video shows you how to navigate the APA-graph-jungle.

Tagged , ,

APA (6th Edition) formatting in MS Word

So, you have conducted some nice research and wish to write it up for publication (or just to submit to your University). Most students are put off right from the get-go as they have to tackle the fiddly-demon that is APA formatting. (For those unfamiliar, APA stands for American Psychological Association.) These APA guidelines provide a uniform structure to all articles, and it is good practice to adhere closely to their guidelines.

This short video takes away some of the sting by showing you how to set up your Microsoft Word document according to APA (6th edition) standards.

Tagged ,

T-test in Excel

I had requests from some viewers of my videos asking how to conduct t-tests in Microsoft Excel. It’s easy, and this video shows you how. Excel is a very powerful program, with lots of neat functions and capabilities that most users never explore. Expect more posts about Excel in future.

Tagged ,

Independent t-test in SPSS

..and what if the means come from two separate, independent groups? Yup, you guessed it! It’s the independent t-test in SPSS!

Tagged ,

Related (paired) t-test in SPSS

This video shows how to conduct a related (paired-samples) t-test in SPSS

Tagged ,