COMPARISON OF TWO MEANS WITH STUDENT’S T-TESTS

Comparison of two means with Student’s t-test

The t-test is a test that can be applied to compare the mean of two samples. In order to be able to use this test, the variable of interest has to follow a Normal distribution (at least approximately), or you should have at least 30 individuals per sample.

Fantastic, in our previous dataset, we have available the size of a group of 10 men and 8 women. While not quite there for “the 30 individuals per sample” rule, we are fortunate enough that human size notoriously follows a Normal distribution (please don’t ask for reference on this one). Let’s take a look and see if our 2 groups have different means:

t.test(men, women)
        Welch Two Sample t-test
data:  men and women
t =4.1368, df =11.628, p-value =0.001472
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 2.846265 9.228735
sample estimates:
mean of x mean of y 
 177.3500  171.3125

Hard to make more concise, isn’t it? Let’s take a look, line after line, at the response given by R

Welch Two Sample t-test	<- It’s a particular type of t test, we’ll talk about this below.
data:  men and women	<- R is so nice! Reminding us what our data  are!
t = 4.1368, df = 11.628, p-value = 0.001472	<- Value of t, degree of freedom, and corresponding probability
alternative hypothesis: true difference in means is not equal to 0	<- It implies null hypothesis “the means of our 2 samples are equal”
95 percent confidence interval:  2.846265     9.228735	<-  95%CI of the difference between the 2 means… It’s with this kind of details that you can see that R has been made by people who know what really matters!
sample estimates: mean of x mean of y 177.3500  171.3125	<- And to finish, a quick reminder of what the means of our samples are.

Remember when I was talking about details that can be surprising? (If not, just look a couple of lines above, and stop interrupting me because you’re not following!) One example: degrees of freedom with decimals… If you’re not surprised, congratulations, you either didn’t care about them, or you already know what I am about to explain. This comes from the fact that the t-test performed here is a little special. In order to be extra careful, by default, R uses an approximation of the t-test (Welsh’s t-test) that does not assume that variances of our 2 samples are equal.

In order to use the historical Student’s t-test, we simply have to specify to R that the variances are supposed to be the same (by setting the argument “var.equal” to TRUE):

t.test(men, women, var.equal=T)
        Two Sample t-test
data:  men and women
t =4.3341, df =16, p-value =0.0005129
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 3.0844058.990595
sample estimates:
mean of x mean of y 
 177.3500  171.3125

You can see that the degrees of freedom are now higher (and an integer). Moreover, the test is now even more affirmative regarding the significance of the difference between the 2 means, and that the corresponding confidence interval is shorter (more informative). This is, of course, normal, since we are making here another assumption (about the equality of the variances).

---

Comparison of two means with Student’s t-test for paired data

Let’s take a look at another situation where we are dealing with large samples or variables following the normal distribution. When our samples are paired, such as in a situation “before treatment” vs “after treatment”, it is important to “keep track” of which data are paired with each data, in order to be able to accurately detect variations in the mean once the treatment has been applied.

In a situation where locations have been sampled before and after the application of wildlife management decisions, or where individuals have been measured before and after having faced a treatment, it is necessary to adapt our t test to take the pairing into account. For example, let’s take a look at the IQ scores of 8 students before and after watching an episode of Honey-Boo-Boo…

IQbefore=c(125,130,116,105,97,112,131,124)
IQafter=c(123,128,114,104,93,106,130,121) 

If we compare the two means:

mean(IQafter)-mean(IQbefore)
[1]-2.625

Nothing really impressive… If we only look at the means, we could easily assume that this is due to stochastic variations between the scores of the 2 IQ tests. As a matter of fact, if we don’t take into account the fact that data are paired, a t test would lead us to conclude that there is no difference between before and after watching an episode of Honey Boo Boo.

t.test(IQbefore,IQafter, var.equal=T)
        Two Sample t-test

data:  IQbefore and IQafter
t =0.4163, df =14, p-value =0.6835
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -10.90002  16.15002
sample estimates:
mean of x mean of y
  117.500   114.875

But as soon as we let R know that our data are paired, by setting the argument “paired” to TRUE:

t.test(IQbefore,IQafter, var.equal=T, paired=T)
        Paired t-test

data:  IQbefore and IQafter
t =4.4063, df =7, p-value =0.003133
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 1.21629 4.03371
sample estimates:
mean of the differences
                  2.625

It’s another story! There is a clear difference. Our students have lost, on average, 2.63 (1.21;4.03) IQ points after watching just one episode of Honey Boo Boo. So, do yourself a favor: don’t watch Honey Boo Boo… and remember to use the correct statistical test!