Within-Subjects t-test

Outline:
problem of individual differences
the goal of within-subject designs
within-subject t-test
an example

problem of individual differences
People differ.  On any given measure some people will score high and some people will score low and lots of people will score in between.  This is due to their basic abilities and history of experiences.

The problem arises when you have great individual differences on your dependent measure.  This can be a problem of inability to see an effect of your IV due to the noise of individual differences.

The basic formula for hypothesis testing statistics:

differences between groups caused by IV
variability within the groups

The bottom part of the equation (variability within groups) is composed of two parts:
1.  due to measurement error on any given day
2.  due to individual differences

differences between groups caused by IV
measurement error + individual differences

If individual differences are large, then the overall ration will be low.  This means even for important effects you may not be able to pick them up in your stats because of large individual differences.

the goal of within-subject designs
The goal of within-subjects variables is to remove individual differences from the bottom part of the equation.  Consider the example in the handout.

• high individual differences in reading speed
• 7 of 8 subjects were slower on more complex piece

The individual differences means that if you do this as a between-subjects design you do not find any difference between the two conditions.

But you know something is going on because 7 of 8 went in the same direction.  You want to pick this up and that is the goal of within-subjects designs:  To allow you to find the effect in spite of great variability among subjects.

within-subject t-test
The way we go about doing this in a t-test is to compare each subject to his/herself in pretty much the way we did by eyeballing the effect.  For each subject you compute a difference score.

di = Xi1 - Xi2
This difference score ignores the range of scores for that subject -- it equates subjects who started high and subjects who started low by looking to see if they changed.

It is then possible to compute a mean of these difference scores:

Md = SUMi (di)

What we then ask with the within-subjects t-test is:  Is the average difference score large compared to the variability in the difference scores?

t = Md / SEMd
= Md / (sd / sqrt N)

Since in computing the difference scores we ignored the starting range of that person’s scores, we have effectively removed individual differences from this equation.

differences between groups caused by IV
measurement error (without individual differences)

an example

Students read two stories of equal length that vary in complexity of the the writing.  The DV is the number of minutes it takes the students to read the stories.

Text Complexity
 Student simple complex 1 10.4 14.3 2 16.9 20.4 3 6.7 7.5 4 25.9 26.8 5 12.4 14.6 6 18.5 20.3 7 13.5 12.3 8 8.2 10.2 . . . M 14.06 15.80 SD 6.24 6.31

Assume a between subjects design

t = (M1  - M2) / sqrt (s12/n1 + s22/n2)

= (14.06  -  15.80) /  sqrt(6.242/8 + 6.312/8)

t (14) =  - 0.55  p > .20

Assume a within subjects design

Text Complexity
 Student simple complex d 1 10.4 14.3 -3.9 2 16.9 20.4 -3.5 3 6.7 7.5 -0.8 4 25.9 26.8 -0.9 5 12.4 14.6 -2.2 6 18.5 20.3 -1.8 7 13.5 12.3 1.2 8 8.2 10.2 -2.0 . . . . M 14.06 15.80 -1.74 SD 6.24 6.31 1.62

Within-subjects t-test

t = Md / SEMd
= Md / (sd / sqrt N)

t  =     - 1.74 / (1.62 / sqrt 8)

t (7)  =  - 3.038, p < .02