Spring 2018 Professor Delamater Experimental Research Basic Concepts 1 The Main Features of an Experiment a Types of Independent Variables b The Nature of the Dependent Variable ID: 682887
Download Presentation The PPT/PDF document "Psychology 3450W: Experimental Psycholo..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Psychology 3450W: Experimental Psychology
Fall, 2019
Professor DelamaterSlide2
Experimental Research: Basic Concepts
1. The Main Features of an Experiment
a. Types of Independent Variables
b. The Nature of the Dependent Variable c. Extraneous Variables2. Validity in Experimental Research
Lets look at each of these in more detail…Slide3
Features of an Experiment
The Main Point of an experiment:
Look for a causal relationship between “independent” and “dependent” variables.
IV
DV
?Slide4
Features of an Experiment
The Main Point of an experiment:
Look for a causal relationship between “independent” and “dependent” variables.
IV
DV
?
Independent variable
: Some variable whose presumed causal influence we are studying.
Dependent variable
: This is the variable that we are measuring, e.g., behavior, and is thought to be caused by the independent variable (i.e., it is dependent upon some environmental event).
For example: heat causes water to boil, taking an aspirin causes your fever to subside, the thought of public speaking causes anxiety, receiving an A on an exam causes elation,
etc
Slide5
Features of an Experiment
Types of Independent Variables:
1.
Manipulated – the experimenter alters (controls) the environment in some way and assesses the effects of this upon subjects’ performance. In this case, subjects may be randomly assigned to different groups.
2.
Selected
–
the experimenter places subjects into distinct groups on the basis of some selected subject variable (some pre-existing attribute, e.g., gender).Slide6
Features of an Experiment
Types of Independent Variables:
1.
Manipulated – the experimenter alters (controls) the environment in some way and assesses the effects of this upon subjects’ performance. In this case, subjects may be randomly assigned to different groups.
Causal conclusions are easier to reach.
2.
Selected
–
the experimenter places subjects into distinct groups on the basis of some selected subject variable (some pre-existing attribute, e.g., gender).
Causal conclusions are NOT as easy to reach.Slide7
Features of an Experiment
Example of the difference between manipulated and selected independent variables:
Lets study the effects of Anxiety on Test performance (timed math test)
Scenario 1:
Randomly assign people into three distinct groups.
Group 1 (Experimental Group 1) – These people are given relaxation training.
Group 2 (Control Group) – These people are NOT given any special instructions.
Group 3 (Experimental Group 2) – These people are given instructions designed to make them feel slightly anxious (e.g., your performance on this test will reflect how smart you are and what kind of person you are).
This is an example of a Manipulated IV because we are randomly assigning people to the different conditions of our experiment and we are manipulating different levels of anxiety.
Our dependent variable is performance on the timed math test.Slide8
Features of an Experiment
Scenario 1:
Randomly assign people into three distinct groups.
Group 1 (Experimental Group 1) – These people are given relaxation training. Group 2 (Control Group) – These people are NOT given any special instructions. Group 3 (Experimental Group 2) – These people are given instructions designed to make them feel slightly anxious (e.g., your performance on this test will reflect how smart you are and what kind of person you are).
In this case it looks like different levels of anxiety have a clear effect on test performance.Slide9
Features of an Experiment
Scenario 2:
Assign people into three distinct groups on the basis of their anxiety questionnaire results.
Group 1 (Low) – These people scored low on the anxiety questionnaire. Group 2 (Control Group) – These people scored an intermediate level. Group 3 (High) – These people scored high on the anxiety questionnaire.In this case it looks like different levels of anxiety have a clear effect on test performance.
However, this conclusion is more difficult to reach. Why?Slide10
Features of an Experiment
Scenario 2:
Assign people into three distinct groups on the basis of their anxiety questionnaire results.
Group 1 (Low) – These people scored low on the anxiety questionnaire. Group 2 (Control Group) – These people scored an intermediate level. Group 3 (High) – These people scored high on the anxiety questionnaire.Uncontrolled subject variables may
covary
with anxiety and explain these differences. For instance, maybe IQ or math proficiency is correlated with anxiety.
This isn’t a problem with random assignment because these subject variables are equally distributed across the different groups of the experiment.Slide11
Features of an Experiment
Levels of an Independent Variable:
1.
Two – experimental vs control conditions. 2.
> Two
–
3 or more levels. This allows for “non-linear” effects to be seen.
Different conclusions when looking at the graph on left vs right.Slide12
Features of an Experiment
Control Groups:
A control group is used to provide a baseline assessment against which the impact of the experimental variable of interest can be determined. This group is usually treated exactly like the experimental group with the exception of the presence of the key variable of interest (e.g., caffeine vs no caffeine).
In this case, the 0 mg group is the control group, but they should drink decaf
…
Slide13
Experimental Research: Basic Concepts
1. The Main Features of an Experiment
a. Types of Independent Variables
b. The Nature of the Dependent Variable c. Extraneous Variables
2. Validity in Experimental Research
Lets look at each of these in more detail…Slide14
Experimental Research: Basic Concepts
Dependent Variables
a. This is the response we are measuring
b. All of the issues discussed last time apply here (measurement scales, reliability & validity, statistical power).Slide15
Experimental Research: Basic Concepts
1. The Main Features of an Experiment
a. Types of Independent Variables
b. The Nature of the Dependent Variable c. Extraneous Variables
2. Validity in Experimental Research
Lets look at each of these in more detail…Slide16
Features of an Experiment
Types of Extraneous Variables:
1.
Confounding extraneous variables – these are uncontrolled variables that
covary
with the different levels of the independent variable.
2.
Non-confounding extraneous variables
– these uncontrolled variables do not
covary
with the different levels of the IV, but they can add random noise to the results that may obscure the effect of interest from being observed. I call this the “dilution effect.”
Extraneous Variables – These are variables other than those of interest to the experimenter in the investigation that may influence, i.e., contaminate, the results of the study if not properly controlled.Slide17
Features of an Experiment
Example of extraneous variables:
Suppose we were to study the effects of music on memory.
List Learning Retention Interval Recall Test List A No Music Write as many words as you can List B Classical Music Write as many words… List C Jazz Music Write as many words...
List D Rock & Roll Music Write as many words...
Gp
C
Gp
E-Cl
Gp E-J
Gp
E-RR
What is the important confound here?Slide18
Features of an Experiment
Example of extraneous variables:
Suppose we were to study the effects of music on memory.
List Learning Retention Interval Recall Test List A No Music Write as many words as you can List B Classical Music Write as many words… List C Jazz Music Write as many words...
List D Rock & Roll Music Write as many words...
Gp
C
Gp
E-Cl
Gp E-J
Gp
E-RR
What is the important confound here?
We wish to study the effects of music on memory, NOT list type.
List type
covaries
with the levels of our music manipulation and is, therefore, confounding.
How can this be overcome?Slide19
Features of an Experiment
Example of extraneous variables:
Suppose we were to study the effects of music on memory.
List Learning Retention Interval Recall Test List A No Music Write as many words as you can List A Classical Music Write as many words… List A Jazz Music Write as many words...
List A Rock & Roll Music Write as many words...
Gp
C
Gp
E-Cl
Gp E-J
Gp
E-RR
What is the important confound here?
We wish to study the effects of music on memory, NOT list type.
List type
covaries
with the levels of our music manipulation and is, therefore, confounding.
How can this be overcome?
Use 1 list to train ALL groups.Slide20
Features of an Experiment
Example of extraneous variables:
Suppose we were to study the effects of music on memory.
List Learning Retention Interval Recall Test List A No Music Write as many words as you can List A Classical Music Write as many words… List A Jazz Music Write as many words...
List A Rock & Roll Music Write as many words...
Gp
C
Gp
E-Cl
Gp E-J
Gp
E-RR
What is the important confound here?
We wish to study the effects of music on memory, NOT list type.
List type
covaries
with the levels of our music manipulation and is, therefore, confounding.
How can this be overcome?
Use 1 list to train ALL groups.
Are there any other variables that may be extraneous to the experiment?Slide21
Features of an Experiment
Example of extraneous variables:
Suppose we were to study the effects of music on memory.
List Learning Retention Interval Recall Test List A No Music Write as many words as you can List A Classical Music Write as many words… List A Jazz Music Write as many words...
List A Rock & Roll Music Write as many words...
Gp
C
Gp
E-Cl
Gp E-J
Gp
E-RR
What is the important confound here?
We wish to study the effects of music on memory, NOT list type.
List type
covaries
with the levels of our music manipulation and is, therefore, confounding.
How can this be overcome?
Use 1 list to train ALL groups.
Are there any other variables that may be extraneous to the experiment?
Time of day, gender, noise in the laboratory, subjects experience with music
…Slide22
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are the important confounds here?Slide23
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are the important confounds here?
Location is confounded with alcohol amount.Slide24
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are the important confounds here?
Location is confounded with alcohol amount.
Volume consumed is also confounded with alcohol amount.
Slide25
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are the important confounds here?
Location is confounded with alcohol amount.
Volume consumed is also confounded with alcohol amount.
The experimenter is also a confound.Slide26
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are some non-confounding extraneous variables here?Slide27
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
What are some non-confounding extraneous variables here?
dart boards differ, distance to dart board can vary in each place, time of day/night, type of beer, prior alcohol consumed, experience playing darts, etc.Slide28
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
All of these extraneous variables can add random noise to the situation that may prevent us from seeing any effect of alcohol on performance. This is the “dilution effect.”
They can all be controlled by randomly assigning people to the different conditions.
But it would be better to
eliminate
as many of these extraneous variables as possible by running everyone under exactly the same conditions in a carefully controlled environment, such as the laboratory.Slide29
Features of an Experiment
Example #2 of extraneous variables:
Suppose we were to study the effects of alcohol on sensorimotor performance. 4 different experimenters go to different pubs throughout the city and engage people at random to play in a dart game. They buy people different numbers of beers and ask them to throw the darts at the dart board and keep track of their accuracy. Experimenter 1 goes to the Bronx, 2 to Brooklyn, 3 Manhattan, and 4 to Staten Island. Each experimenter runs people in one of the 4 groups listed below.
Drinking Period Dart Throwing Period
0 pint Throw for bullseye
1 pint Throw for bullseye
2 pints Throw for bullseye
4 pints Throw for bullseye
Gp
0
Gp
1
Gp
2
Gp
4
All of these extraneous variables can add random noise to the situation that may prevent us from seeing any effect of alcohol on performance. This is the “dilution effect.”
They can all be controlled by randomly assigning people to the different conditions.
But it would be better to
eliminate
as many of these extraneous variables as possible by running everyone under exactly the same conditions in a carefully controlled environment, such as the laboratory. Also need to eliminate any confounds.Slide30
Validity in Experimental Research
4 Types of Validity concerning the research and its conclusions
1. Statistical Validity
2. Construct 3. External 4. InternalThese ideas are all issues that may lead us to question the validity of the research findings and/or conclusions.Slide31
Validity in Experimental Research
4 Types of Validity concerning the research and its conclusions
1. Statistical Validity – Have the data be properly analyzed?
2. Construct 3. External 4. InternalSlide32
Validity in Experimental Research
4 Types of Validity concerning the research and its conclusions
1. Statistical Validity – Have the data be properly analyzed?
2. Construct – Has the IV been manipulated adequately and is the DV valid and reliable? Sometimes a “manipulation check” is conducted. 3. External 4. InternalSlide33
Validity in Experimental Research
4 Types of Validity concerning the research and its conclusions
1. Statistical Validity – Have the data be properly analyzed?
2. Construct – Has the IV been manipulated adequately and is the DV valid and reliable? 3. External – To what extent can the results be generalized to other populations of subjects, to other conditions, and times? 4. InternalSlide34
Validity in Experimental Research
4 Types of Validity concerning the research and its conclusions
1. Statistical Validity – Have the data be properly analyzed?
2. Construct – Has the IV been manipulated adequately and is the DV valid and reliable? 3. External – To what extent can the results be generalized to other populations of subjects, to other conditions, and times? 4. Internal – To what extent can a causal conclusion be drawn about the relationship between IV and DV?Slide35
Validity in Experimental Research
There is an inverse relationship between Internal and External Validity
When internal validity is high the studies are generally conducted under well controlled laboratory situations which are fairly removed from the real world. Therefore, the external validity may be low under these circumstances.
However, when external validity is higher, the studies are generally conducted in the real world. But we have less control over extraneous variables there. Thus, the internal validity is generally lower in these cases.Slide36
Validity in Experimental Research
A closer look at Internal Validity: Threats to Internal Validity
1. History
2. Maturation 3. Regression to the mean 4. Subject Selection 5. Attrition 6. Interaction effects (e.g., selection x attrition, etc)
But note that this is a partial list. The general rule is that whenever an extraneous variable
covaries
with the independent variable, then internal validity is reduced.Slide37
Validity in Experimental Research
History Effect
– When some historical event other than the variable of interest intervenes between pre and post measures. This historical event, and not the variable of interest, could explain any differences in performance.
e.g., Effects of a change in drunk driving laws on traffic fatalities.
Looks like the law has had a beneficial effect.
But from the baseline (pre) to post measurement times there was a strong nationwide MADD campaign (Mothers Against Drunk Driving).
This MADD campaign is the other “historical event” that could explain the decrease in fatalities, and this questions the importance of the change in driving laws.Slide38
Validity in Experimental Research
Maturation Effect
– When a change in behavior is caused by some growth variable, e.g., hormone change, physical growth
change,etc.e.g., Effects of a diet on sexual behavior in hamsters.
Diet A Diet B
Looks like Diet B causes an increase in sexual behavior.
But could this be explained by a change in maturational variables (e.g., hormones)?Slide39
Validity in Experimental Research
Maturation Effect
– When a change in behavior is caused by some growth variable, e.g., hormone change, physical growth
change,etc.e.g., Effects of a diet on sexual behavior in hamsters.
Diet B Diet A
Looks like maturational variables explain the increase in sexual behavior.Slide40
Validity in Experimental Research
Regression to the Mean
– This refers to the statistical tendency of any extreme score to approach the mean on a subsequent measurement.
e.g., RTs to respond to a visual stimulus presented on the screen. Over many trials RTs will fluctuate. An especially fast RT on one trial will tend to be slower on the next trial, and vice versa.e.g., Suppose a basketball coach wanted to give training to their players on foul shooting. Initially, the coach asks all players to shoot 20 foul shots. Those scoring less than 50% are taken aside and given special instructions. Those scoring > 50% are allowed to leave. Following the special training the coach asks those low-scoring foul shooters to shoot another 20 shots. This time the group average is > 50%. Is this due to a real effect of the “special instructions” on how to shoot foul shots, or is it due to regression to the mean?Slide41
Validity in Experimental Research
Subject Selection Effect
– This refers to the fact that in experiments with selected independent variables, some subject variable other than the one of interest could potentially explain the results.
e.g., Cancer patients are recruited to participate in a study involving a new cancer drug. Some patients volunteer to remain with an old drug with a known likelihood of success, and other patients volunteer to receive some new drug.Slide42
Validity in Experimental Research
Subject Selection Effect
– This refers to the fact that in experiments with selected independent variables, some subject variable other than the one of interest could potentially explain the results.
e.g., Cancer patients are recruited to participate in a study involving a new cancer drug. Some patients volunteer to remain with an old drug with a known likelihood of success, and other patients volunteer to receive some new drug.
Looks like the new drug is not as effective as the old drug.
But this study DID NOT involve random assignment, so it can potentially be explained in terms of a subject selection effect.
Maybe certain variables other than which drug patients receive could explain the decreased survival rate in the new drug group. Like severity of their cancer and willingness to try something newSlide43
Validity in Experimental Research
Attrition
– This refers to the fact that subjects sometimes drop out from a study. If this happens to a substantial degree then the composition of the group can be very different across the study.
e.g., Suppose you wish to investigate the effects of an educational program with a population of recently-released prisoners on the ability of those prisoners to find employment.Slide44
Validity in Experimental Research
Attrition
– This refers to the fact that subjects sometimes drop out from a study. If this happens to a substantial degree then the composition of the group can be very different across the study.
e.g., Suppose you wish to investigate the effects of an educational program with a population of recently-released prisoners on the ability of those prisoners to find employment.
Looks like the program works
But, suppose that the dropout rate (from the program) also steadily increased over weeks.
This means that those who remain in the program may be the only ones with jobs.Slide45
Validity in Experimental Research
Attrition
– This refers to the fact that subjects sometimes drop out from a study. If this happens to a substantial degree then the composition of the group can be very different across the study.
e.g., Suppose you wish to investigate the effects of an educational program with a population of recently-released prisoners on the ability of those prisoners to find employment.
Looks like the program works
But, suppose that the dropout rate (from the program) also steadily increased over weeks.
This means that those who remain in the program may be the only ones with jobs.
By restricting our attention to only those who remained in the program throughout, we might better assess the effects of the program. Slide46
Validity in Experimental Research
Attrition
– This refers to the fact that subjects sometimes drop out from a study. If this happens to a substantial degree then the composition of the group can be very different across the study.
e.g., Suppose you wish to investigate the effects of an educational program with a population of recently-released prisoners on the ability of those prisoners to find employment.
By restricting our attention to only those who remained in the program throughout, we might better assess the effects of the program.
Also, we would need a control group that does not get the educational program to assess its effectiveness.Slide47
Validity in Experimental Research
Interactive Effects
– Sometimes different threats can interact in their impact on a study.
e.g., Subject x History Interactive Effect. Suppose we wish to evaluate the effects of some new psychotherapy on different populations of psychiatric patients in a mental health facility. We conduct a baseline measure of the severity of the patients’ psychiatric symptoms, then we expose different schizophrenic and depressed patients to our new therapy, and then take a post-measure of the severity of their symptoms. However, part way through the treatment phase, there was a city-wide loss of power due to a particularly bad hurricane in the area. This loss of power occurred for 3 days. Slide48
Validity in Experimental Research
Interactive Effects
– Sometimes different threats can interact in their impact on a study.
e.g., Subject x History Interactive Effect. Suppose we wish to evaluate the effects of some new psychotherapy on different populations of psychiatric patients in a mental health facility. We conduct a baseline measure of the severity of the patients’ psychiatric symptoms, then we expose different schizophrenic and depressed patients to our new therapy, and then take a post-measure of the severity of their symptoms. However, part way through the treatment phase, there was a city-wide loss of power due to a particularly bad hurricane in the area. This loss of power occurred for 3 days.
Looks like the new therapy benefits Depressives more than Schizophrenics.
However, the power outage (history effect) could have impacted schizophrenics more than depressives (subject x history effect).
This complicates our interpretation.Slide49
Validity in Experimental Research
A closer look at Internal Validity: Threats to Internal Validity
1. History
2. Maturation 3. Regression to the mean 4. Subject Selection 5. Attrition 6. Interaction effects (e.g., selection x attrition, etc)
But note that this is a partial list. The general rule is that whenever an extraneous variable
covaries
with the independent variable, then internal validity is reduced.
In short, there are a lot of ways in which internal validity might be threatened in any given experimental situation. We need to be aware of any and all potential confounds.