-- independent variable
-- dependent variable
-- true experimental variables 1: between-subjects
basic 2 group between-subjects
-- true experimental variables 2: within-subjects
basic 2 group within-subjects
-- quasi-experimental variables: cross-sectional or status variables
basic 2 group within-subjects
-- factorial designs
-- limiting noise
-- control groups
All members of a group
Group can be variously defined
A subset of a population
Unbiased sample vs. biased sample
All members of the population have equal chance of being selected
Generalizing from sample to the population
A single member of the sample
Human subjects are often called participants
Independent Variable (IV)
Treatment or Factor
The manipulated variable in an experiment
IV has various levels that different groups/conditions experience
Theory claims IV is causal
Measured in experiment
Theory claims IV affects DV
Today we are going to worry about a basic step in designing and controlling your observations: experimental design.
OBS --> Theory --> Hypothesis --> OBS
I want to start by explaining something about this design process:
Everything is determined by theories.
· Theories tell you what IVs should affect the variable of interest
· You can then make precise hypotheses concerning what levels of the IVs should lead to high and low outcomes on the DV
· You then design an experiment (observation) that causes those levels, controls other possible confounds, and measures the outcome
· Thus you start with theories in a given domain.
One other point: the experimental design and the scale of measurement
determine your statistics: so once you know your question, everything
Thus start thinking about what questions are worth asking. Do not do it the other way around, say I want to do this stat so what design, what question.
True experiments: between-subjects variables
True experiments are where you manipulate the Independent Variable and you control all other factors that may influence the Dependent, or outcome, Variable. In this case you can really answer a question; you can know if changes in the IV cause changes in the DV.
If you want to know the differences between X and Y, you set things up such that X happens to one group and Y happens to another. You then measure outcomes. If you want to know the difference between X, Y, and Z, you set things up so that X, Y, and Z happen to different groups and you measure outcomes.
Basic 2 group between-subjects true experiment
Here is the basic design
|Pop -->||Sample -->||A||Level 1||measure||Compare groups|
Keep in mind that you start with a question and you then figure out which design will allow you to investigate that question.
Imagine that you wanted to investigate the decay theory of forgetting. Say, for example, you were interested in how time since learning affects memory. Then IV would be time; level 1 might be 0 (immediate recall) and level 2 might be 1 minute. DV would be determined by how you operationally defined memory. Say number of words from a word list.
Multiple levels between-subjects true experiment
Say if you had different ideas about how forgetting works. If
you thought interference was important and you wanted to know if visual
information interferes with verbal information, but you realize that just
time itself does allow for some decay in memory.
|Pop --->||Sample -->||B||Level 2||measure||Compare groups|
IV: Type of Interference
Levels: 1 = verbal inference; 2 = visual interference, 3 = no interference (control)
This is another way to control for individual differences: measure each subject before and each after IV, then the DV is the change. Say effects of different types instruction but you know some people know a lot before the course and some know less.
|Pretest DV||IV||Posttest DV|
|Pop --->||Sample -->||A||measure||Level 1||measure||compare changes|
True experiments, II: Within-subjects designs
Sometimes it is possible to conduct within-subjects designs: This, like a pretest-posttest design, eliminates the problem of individual differences.
In this case, each subject acts as his/her own control and experiences all conditions. So for time study each subject would get one word list at each delay:
You can do this with multiple levels as well, each subject experiences every level of IV and gets measured on the DV after each.
Controls for Individual Differences
Uses fewer subjects
Design is transparent to subjects
May lead to practice effects
Quasi-experimental designs: Cross-sectional or status variables
What if we want to investigate the differences between two different types of people. Say you are still concerned about learning little words on a list, but what about the difference between people with large and small vocabularies. You have reason to believe that people with more knowledge of words will be better able to process and store words in a list and thus should recall more of the a word list.
Now in this case, we can’t randomly assign some people to have large vocabularies and some to have small. Instead you are drawing samples from two separate populations
|Pop 1 -->||Sample 1 -->||measure||Compare groups|
|Pop 2 -->||Sample 2 -->||measure|
IV: Vocab size
Levels: 1= large vocab; 2 = small vocab,
Both experience word list and are asked to recall it.
Of course, you can have multiple levels here just like in true experiments.
True Experimental Variables vs. Quasi-Experimental Variables
Difference is in your ability to infer
Can infer that IV is related to DV, but what about the IV?
Why are people that way?
Is there something causing them to have large vocabs and do well on word list tasks?
Cannot determine causality!
What factor (IV) you are interested in determines whether or not you can manipulate it and thus whether you can conduct a true experiment.
What if you think two IVs affect the DV in question?
Say for forgetting, you want to look at both time and interference.
1= 1 minute
2= 2 minute
IV2: Type of Interference
2 = visual
Draw as a box
|IV 1 - Time||
|L1 - 1 minute||
|L2 - 2 minutes||
Note: In a factorial design the IVs can be between-subjects, within-subjects,
or cross-sectional. Can have lots of IVs
What if you think that vocabulary size determines ability to do these simple word list memory experiments.
You could do a cross-sectional quasi-experiment where you grouped people by vocab size into high and low, and measure memory for a word list.
Or you could take each persons score on vocab measure and number of words recalled and correlate to memory.
(Draw graphs with two groups vs. 20 individuals.)
There are other styles of collecting observations but they are not the topic of this course really.
Any factor that varies with your IV and that could explain observed differences in your DV.
You do not want confounds.
Creep in because of
Nature of IV
Affects causal claims
In true experiment, something caused effect on DV
IV or Confound?
Limiting noise/random variability
Variability within groups
Minor differences in experimental sessions
You want to limit
Contributes to total variability and limits ability to see effects
Formula for most stats is:
Variability due to the
Baseline comparison conditions
Time and memory
Immediate is control
Stroke victims and cognitive functioning
Other head injury
What is the best comparison condition
Existing treatment in medicine
Eliminate other explanations
Debriefing group for conformity effects
Is it the IV or just getting “special treatment”?
Other effective instruction techniques
Wait list control
Blind and Double Blind
Blind – Subject is unaware of IV level
Double Blind – Subject and experimenter are unaware