Experimental Designs Conducting Experiments BetweenSubjects Design Betweensubjects design Different participants are observed one time in each group or at each level of a factor Betweensubjects experimental design Levels of a betweensubjects factor are manipulated then different pa ID: 640215
Download Presentation The PPT/PDF document "Chapters 10-12 Between-Subjects, within..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Chapters 10-12
Between-Subjects, within-subjects, and factorial
Experimental DesignsSlide2
Conducting Experiments: Between-Subjects Design
Between-subjects design – Different participants are observed one time in each group or at each level of a factor
Between-subjects experimental design – Levels of a between-subjects factor are manipulated, then different participants are randomly assigned to each group or to each level of that factor, and observed one
timeSlide3
Conducting Experiments: Between-Subjects Design
Control: (a) the manipulation of a variable and (b) holding all other variables constant
Experimental or treatment group: Participants are treated or exposed to a manipulation, or level of the IV, that is believed to cause a change in the DV
Control group: Participants are treated the same as those in an experimental group, except that the manipulation is omitted
Placebo: An inert substance, surgery, or therapy that resembles a real treatment but has no real effectSlide4
Manipulation and the Independent Variable
Experimental
manipulation
Natural manipulation: Manipulation of a stimulus that can be naturally changed with little effort
Typically involves manipulation of a physical stimulus
Ex
. Dimmed or brightly lit room, soft or loud sounds
Staged manipulation: Manipulation of an IV that requires the participant to be “set up” to experience some stimulus or event
Often requires the help of a confederate
Confederate:
Coresearcher
in cahoots with the researcherSlide5
Manipulation and the Independent VariableSlide6
Manipulation and the Independent Variable
Random assignment and control
Random assignment: Procedure used to ensure that each participant has the same likelihood of being selected to a given group
Can be confident that any differences observed between groups can be attributed to the different levels of the IV and not individual
differences
Restricted measures of control
Restricted random assignment: Restricting a sample based on known participant characteristics, then using a random procedure to assign participants to each group
Control by matching
Control by holding constantSlide7
Manipulation: Control By MatchingSlide8
Overlap and Identifying ErrorSlide9
Comparing Samples
Selecting Two Independent Samples
Independent sample: Different participants are independently observed one time in each group
Selecting multiple independent samplesSlide10
Comparing Independent Samples
The use of the test
statistic
Independent-Samples T-test for factor with 2 levels
One-way
between subjects ANOVA:
one
factor with two or more levels concerning the variance among group means
Post hoc
test: Computed following a significant ANOVA to determine which pair(s) of group means significantly differ
These tests are needed with more than two groups because multiple comparisons must be
madeSlide11
Advantages and Disadvantages of the Between-Subjects Design
Advantages
It is the only design that
can include all three:
random assignment,
manipulation, inclusion of a comparison/control
group
Disadvantages
Sample size required can be large, particularly with many groupsSlide12
Conducting Experiments: Within-Subjects Design
Within-subjects design, also called a repeated-measures design – Design in which the same participants are observed one time in each group of a research study
Within-subjects experimental design – The levels of a within-subjects factor are manipulated, then the same participants are observed in each group or at each level of the
factorSlide13
Conducting Experiments: Within-Subjects Design
Two common reasons that researchers observe the same participants in each group are as follows:
1. To manage sample size
2. To observe changes in behavior over time, which is often the case for studies on learning or within–participant changes over time
The within-subjects experimental design does not meet the randomization requirement for demonstrating cause and effect
Because the participants are observed in each group, we cannot use random assignment, therefore do not use randomizationSlide14
Controlling Time-Related Factors
Time-related factors must be controlled or made the same between groups, such that only the levels of the IV are different between groups
Time related factors include those introduced in chapter 6, such as maturation, testing effect, regression toward the mean, and attrition
Participant fatigue: State of physical or psychological exhaustion resulting from intense research demands typically due to observing participants too often, or requiring participants to engage in research activities that are too demandingSlide15
Controlling Time-Related Factors
To control for time-related factors, researchers make efforts to control for order effects
Order effects: A threat to internal validity in which the order in which participants receive different treatments or participate in different groups causes changes in a DV
Carryover effects: A threat to internal validity in which participation in one group “carries over” or causes changes in performance in a second group
Two strategies to control for order effects are to control order and control timing Slide16
Controlling Time-Related Factors
Counterbalancing – The order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment
1. Complete
counterbalancing
(K!)
2. Partial counterbalancing
Left
Right
Center
Left
Center
Right
Right
Left
Center
Right
Center
Left
Center
Left
Right
Center
Right
LeftSlide17
Controlling Time-Related FactorsSlide18
Individual Differences and Variability
Individual differences
The within-subjects design minimizes individual differences between groups because the same participants are observed in each group
When the same participants are observed in each group, the individual differences of participants are also the same in each groupSlide19
Individual Differences and Variability
Sources of variability
Between-groups variability: Source of variance in a dependent measure that is caused by or associated with the manipulation of the levels (or groups) of an IV
This variability is measured by the group meansSlide20
Individual Differences and VariabilitySlide21
Comparing Two Related SamplesSelecting two related samples
Related sample, also called a dependent sample: The same or matched participants are observed in each group
There are two ways to select two related samples:
1. The same participants are observed in each group
2. Participants are matched, experimentally or naturally, based on the common characteristics or traits that they shareSlide22
Comparing Two Related Samples
The use of the test statistic
Test statistic: Mathematical formula that allows us to determine whether the manipulation or error variance is likely to explain differences between the groups
Related-samples
t
test, also called a paired-samples
t
test: Statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data for two related samples in which the variance in one population is unknown
t
= Mean differences between groups
Mean differences attributed to errorSlide23
Comparing Two Related SamplesSlide24
Comparing Two or More Related Samples
Selecting multiple related samples
Only the repeated-measures design can be used to observe participants in more than two groupsSlide25
Comparing Two or More Related Samples
The use of the test statistic
One-way within-subjects analysis of variance (ANOVA): Statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when the same participants are observed at each level of a factor and the variance in a given population is unknown
F = Variability between groups
Variability attributed to
errorSlide26
Comparing Two or More Related SamplesSlide27
Testing Multiple Factors in the Same Experiment
Factorial design – Research design in which participants are observed across the combination of levels of two or more factors
In stats class, this was referred to as Two-Way ANOVA (or more)Slide28
Testing Multiple Factors in the Same Experiment
Factorial experimental design – Research design in which groups are created by manipulating the levels of two or more factors (can be between-, within- and mixed-design)
Completely crossed design: A factorial design in which each level of one factor is combined or crossed with each level of the other factor, with participants observed in each cell or combination of levelsSlide29
Selecting Samples for a Factorial Design in Experimentation
We select ONE sample from a population, then randomly assign the same or different participants to groups created by combining the levels of two or more factors or IVs
Create the groups by combining the levels of each IV
Identify a factorial design by the number of levels for each factor
Then assign participants to groupsSlide30
Selecting Samples for a Factorial Design in Experimentation
Decaf
Reg
Coffee
Easy Task
Difficult Task
Easy Task
Difficult Task
Decaf
Reg
Coffee WaterSlide31
Types of Factorial Designs
Between-subjects design – Levels of two or more between-subjects factors are combined to create groups, meaning that different participants are observed in each group
Ex. Researchers recorded how well participants comprehended a passage that varied by type of highlighting and text difficulty (
Gier
,
Kreiner
, &
Natz
-Gonzalez, 2009)Slide32
Types of Factorial Designs
Within-subjects design – Levels of two or more within-subjects factors are combined to create groups, meaning that the same participants are observed in each
groupSlide33
Types of Factorial DesignsMixed factorial design – Different participants are observed at each level of a between-subjects factor and also repeatedly observed across the levels of the within-subjects factorSlide34
Main Effects and Interactions
Two-way analysis of variance (ANOVA) – Statistical procedure used to analyze the variance in a DV between groups created by combining the levels of two factors
F = Variability between groups
Variability
attributed to error
The
test statistic can also be used in quasi-experiments however, because the quasi-experiment does not methodologically control for individual differences, the design cannot demonstrate cause and effectSlide35
Main Effects and Interactions
Two-way factorial design – Research design in which participants are observed in groups created by combining or crossing the levels of two factors
Using this design we can identify three sources of variation:
Main Effect for Factor A
Main Effect for Factor B)
I
nteraction Effect (the combination of levels of Factors A and BSlide36
Main Effects and Interactions
Main effects – Source of variation associated with mean differences across the levels of a single factor
A significant main effect indicates that group means significantly vary across the levels of one factor, independent of the second factor
Interactions – Source of variation associated with how the effects of one factor are influenced by, or depend on, the levels of a second factor
A significant interaction indicates that group means significantly vary across the combined levels of two factors
In a table summary, an interaction is a measure of how cell means at each level of one factor change across the levels of a second factorSlide37
Identifying Main Effects and Interactions in a Graph
Even if a graph shows a possible main effect or interaction, the use of a test statistic is still needed to determine whether it is significant
Graphing only main effects
We would observe changes at the levels of one factor, independent of the changes in a second factor
When significant, look at the row and column means to describe the effectSlide38
Including Quasi-Independent Factors in an Experiment
The factorial design can be used when we include preexisting or quasi-independent factors
Participant variable – A quasi-independent or preexisting variable that is related to or characteristic of the personal attributes of a participant
Typically demographic characteristics (ex. age, gender)
An effect of a quasi-independent variable shows that the factor is related to changes in a DV
It does not demonstrate cause and effect because the factor is preexisting