Lecture 4 Ė t-test

Outline:
-- probabilty and causality
-- hypothesis testing and probability
    differences between groups
    types of explanations
-- between-subjects t-test
    formula as a concept
    actual formula
    example
-- writing
-- relation to H0 and HA
     stat hypotheses
     research hypotheses
-- the t distribution
     tails
     distribution and df
-- types of errors
-- return to the conceptual formula and predicting t-tests

Probabilty & Causality
I now want to begin to introduce you to probability as it will be used in hypothesis testing.  Weíll start with the very concrete approach.

p = event/number of possibilities

Start with the cards example and start out with the card on the top of the deck.
What is the probability that the first card is an Ace;  a face card;  a heart;  the ace of spades
    (Convert to decimals: .077; .231; .250; .019)

If I turn over the ace of spades as the top card, why is it there?

*** There are two explanations -- chance by shuffling deck or caused by cheating
*** How do you decide??? (Based on How UNLIKELY)

 Now what are the chances that I will turn over another ace?

 Sidenote -- As long as we are on this topic -- let us dismiss the gamblerís fallacy.  What is it?  Before I turn over any cards, what are the chances that I will turn over two aces?  4/52 * 3/51 = .0045; but once I have turned over the first ace, if it happened, what are the chances that the second card will be an ace?  3/51 = .059.
 Before you start the night, the chances are that you will have some good hands and some bad hands, but once you are into the night, what is past is past -- on each next hand, regardless of previous hands, the odds of getting a good hand remain the same: low!
 

Probability and Hypothesis testing
 Now what does this have to do with experiments?

Think back to our experiment design:
 
IV DV
Pop    --> Sample    --> A Level 1 measure Compare groups
B Level 2 measure

Differences between groups:  There will most likely be some difference when we are done.  But the question is to what do we attribute that difference?

Types of explanations:  Explanations are chance or caused; same as with the cards.  When were you willing to accuse me of stacking the deck?  When the odds of it occurring by chance are very low.
 Caused:  due to experimental manipulation
 Chance:  due to random sampling & measurement noise
   Think of this as shuffling the subjects to the groups.
   (Assume no TRUE Difference)

When difference is so large as to be very unlikely, you then claim that it was most likely caused by the experimental manipulation.

 This is the purpose of hypothesis testing statistics -- to tell you the probability that the difference you observed occurred by chance.
That allows you, the researcher, to decide if you want to attribute the outcome to the manipulation or not.

Between subjects t-test

Formula as a concept
           Diff B/T groups
         Variability W/I groups

You want to see if the difference between the groups is large compared to what you migh expect based on the chance variability (which is the within group variability).

Actual formula

 Diff B/T the means of the two groups    = M1  - M2
 a combined measure of variability =  Combined SEM (Standard Error of the Mean; S / sqrt n)

 When n1 = n2

t = (M1  - M2) / sqrt (s12/n1 + s22/n2)

When n1 does not equal n2

t = (M1  - M2) / sqrt [ ( (SS1 + SS2 ) / (n1 + n2 - 2) ) * (1/ n1 + 1/ n2) ]
 

Example
 Want to know how important having information in a coherent organization is for memory of the information.  Schema Theory suggests that it is important -- information that conforms to prior knowledge is easier to process and store.  New Associations Theory suggests new links being formed so order doesnít matter.
 Take the story with 80 idea units as basic.
IV:  organization
 L1:  correct order
 L2:  random order
DV:  number of idea units recalled after five minutes
 

Between Subjects t-test

                            Presntatio Order
Correct Order Random Order
1 53  43
2 58  44
3 60  45
4 60  47
5 61  49
6 63  50
7 64  51
8 65  52
9 65  52
10 66  53
11 66  54
12 66  54
13 67  54
14 68  56
15 68  56
16 69  57
17 71  58
18 72  61
19 75  62
20 78  70
Sum Xi 1315 1068
n 20 20
M 65.75 53.40
s 5.84 6.51

 

t =              65.75  - 53.40
         sqrt [(5.839)2/20 + (6.508)2/20]

t = 6.317

df = df1 + df2  = N1 - 1 + N2 - 1
 = NT - 2
 = 38

t (38) = 6.317, p < .002

Writing

1.  What do we know? (brainstorming)
 

2.  How shall we order things?  (outlining)
 

3.  Writing  (1st draft)
 A between-subjects t-test revealed that subjects to whom the story was presented in the correct order recalled more idea units than subjects to whom the story was presented in random order, t (38) = 6.317, p < .002.
  -- noted the stat
  -- used an active verb:  found, revealed, indicated
  -- stated which level was higher than which other level of IV
  -- used OpDef of DV
  -- gave supportive information
  -- can break this apart
   stat in a sentence
   that there was a difference in a sentence
   and the direction of difference in a sentence
The mean number of idea units recalled by the correct order and random order groups was 65.75 and 53.40, respectively.  The standard deviation for the correct order and random order groups was 5.84 and 6.51, respectively.
 

Relation to H0 and HA
Now you probably remember hearing something about a null hypothesis and an alternate hypothesis and cut-off levels when doing stats.  Let me talk a little bit about these concepts.

Stat hypotheses
 Null and Alternative hypotheses are what Iíll refer to as stat hypotheses.  They have to do with things about your t-test.

H0M1  = M2
HA:   M1  not equal M2

 Cut-off levels are for you to know when you reject the null and turn to the alternative.  It is what I was talking about:  chance vs. caused:  is the probability low enough to assume caused.  There is not just one cut-off level.

Research hypotheses
 Research hypothesis have to do with our theory or theories.  Come straight from our view of science.
        Observe -> Theory -> Hypothesis -> Observe.
Thus they are in the language of the theory (usually English).

 For this example:
    Organization matters theory predicts that the ordered group will recall more than the random group.
    The learning new association theory predicts that there will be no difference.

 Look at the t-test as disproving 2nd and supporting the 1st, because the p is so low.
 What if it hadnít have worked?  p = .24.  Supports view of no difference, but doesnít disprove the other.
 

The t distribution
 t scores are normally distributed around zero.  Thus numbers far away from zero occur infrequently (or are low probability).  How far away to be low probability?

Tails
 t distributions also happen to have two tails.
Letís talk about why.
 Because either mean could be larger.
If you know which way you expect things to go, then you can do a one-tailed test.  Otherwise you need to do a two-tailed test.
 1-tailed puts all of probability (like .05) in one tail
 2-tailed divides prob between the trails (.025 in each)
 1-tailed is easier to find something in the direction of interest
 2-tailed allows you to look both ways

What about our example?

Distribution and df
 That depends on the df.  The t distribution is actually a family of distributions.  The higher the df, the tighter the distribution and thus the smaller the numbers are that make something an unlikely t score.
 
 

Types of errors
 
 

State of 
World
Outcome of Experiment
No difference
Yes difference
No difference
correct
Type 2 Error
Missed it
Yes difference
Type 1 Error 
False Alarm
p level
correct

Return to the conceptual formula and predicting t-tests

Formula as a concept
           Diff B/T groups
         Variability W/I groups

Actual formula

 Diff B/T the means of the two groups    = M1  - M2
 a combined measure of variability =  Combined SEM (Standard Error of the Mean; S / sqrt n)

holding other things constant, what happens to t?
if diff b/t means increases
variability increases
n increases

Rule of thumb for predicting t-tests
If the difference between the means is greater than the average of the standard deviations and if N is ok, then the probability associated with t will be < .05.
 ok N is 10-15 per group
 large N compensates for diff b/t means being slightly smaller than sd
 small N means diff b/t means has to be much larger than sd to compensate

Examples
 
 
 
Ordered
Random
Mean
65.75
53.40
Standard Deviation
15.55
16.58
n
10
10

Prediction?
 
 
 
 
Ordered
Random
Mean
65.75
48.40
Standard Deviation
15.55
16.58
n
10
10

Prediction?
 

 
 
 
Ordered
Random
Mean
65.75
70.40
Standard Deviation
4.55
5.58
n
15
17

Prediction?
 
 

 
 
Ordered
Random
Mean
165.75
128.40
Standard Deviation
35.55
26.58
n
15
12

Prediction?
 
 
 
 
 
 
Ordered
Random
Mean
0.75
0.40
Standard Deviation
0.55
0.58
n
10
10

Prediction?
 
 
 
 
 
 
 
 
Ordered
Random
Mean
0.78
0.56
Standard Deviation
0.19
0.21
n
12
15

Prediction?