04
Control Systematic control is core consideration of experimental design
Manipulation of independent variable to potentially influence dependent variable<br>
05
Internal Validity Dependent on experiment design
Strength of proposed conclusions about cause-and-effect relationships
Conflicts with external validity to the extent that controls restrict group participants to certain characteristics<br>
06
Confounding Variables Alternative explanation for observed change
Variations between groups other than independent variable
Single group designs cannot make claims that confounding variables had no influence<br>
07
Mitigating Confounding Variables Ensure that influential factors aside from experimental condition are applied the same to each group
Include a control group which is entirely equivalent to experimental group except for the experimental condition
Eliminate potential for bias by conducting a double blind experiment
Make groups similar on average and reduce differences between groups to chance by random assignment
Expose all groups to experimental and control conditions over time
Statistical analysis<br>
08
Types of Experimental, Quasi-Experimental, and Ex Post Facto Designs In experimental research, a researcher manipulates an independent variable and observes the effects on the dependent variable.
Many designs exist to achieve this task, but they differ by how the independent variable is manipulated and how the researcher controls for confounding variables.
The degree of internal validity changes between designs.<br>
09
Types of Experimental, Quasi-Experimental, and Ex Post Facto Designs This table will be used to help describe each design.
Groups are in each row and different events occur over time.
These events could be:
Tx: treatment is presented - independent variable
Obs: observation is made- dependent variable
—: Nothing occurs during the current time.<br>
10
Pre-Experimental Designs Not possible to show cause-and-effect relationships because either
1. independent “variable” does not vary
2. experimental groups are not comprised of equivalent or randomly selected participants.
Only helpful formulating tentative hypotheses.<br>
11
One-Shot Experimental Case Study Primitive
Introduction of experimental treatment (Tx) and then a measurement (posttest) is administered to determine effects.
Root of common misconceptions
No observation before experiment, single observation cannot determine if situation has changed or not
Results are essentially meaningless.<br>
12
2. One-Group Pretest-Posttest Design ONE single group:
1. undergoes observation/eval
2. is administered experimental treatment
3. Is observed/evaluated again
We can know that a change has taken place
Does not rule out other explanations for the change<br>
13
3. Static Group Comparison Experimental group is exposed to experimental treatment. Control group is not. Then, both groups observed and performance is compared.
Groups are not equal or examined for similarity
Do not know if treatment causes differences between groups<br>
14
True Experimental Designs Offer a greater degree of control.
Greater internal validity
Random assignment guarantees differences between groups are quite small and due to chance<br>
15
4. Pretest-Posttest Control-Group Design Groups randomly assigned to either experimental or control group.
Experimental group observed, subjected to experimental treatment, observed again.
Control group isolated from experiment, only observed at beginning and end.
CAN determine whether changes occur after treatment and can eliminate other possible explanations (confounding variables)
Gives reasonable basis for conclusion about cause-effect relationship.
Observation may influence the response to treatment.<br>
16
5. Solomon Four-Group Design Extension of Pretest-Posttest control-group design.
Adds two groups which are not pretested.
Enhanced external validity (do group 3 and 4 differ similarly to group 1 and 2?)
Larger sample, more time and energy
Goal is to eliminate pretest influence.<br>
17
6. Posttest-Only Control-Group Design You cannot always pretest.
Think of as last two groups of Solomon four-group design.
Random assignment to groups is critical.<br>
18
7. Within-Subjects Design AKA repeated-measures design
All subjects receive treatments.
Optimally, treatments are administered very closely together in time or simultaneously.<br>
19
Quasi - Experimental Designs Sometimes randomness in groups is impossible or impractical.
Don’t control for all confounding variables.
Cannot rule out some alternative explanations for results obtained.
Must take variables/explanations not controlled for into consideration when interpreting data.<br>
20
8. Nonrandomized Control-Group Pretest-Posttest Design Compromise between design 3 and 4.
Involves 2 groups, participants have not been randomly assigned.
Pretreatment observations.
Without random assignment, no guarantee that two groups are similar.
Could be strengthened by identifying matched pairs.<br>
21
9. Simple Time-Series Design Making a series of observations, then introducing an intervention or another dynamic into the system and then making additional observations.
Can reasonably conclude that the intervention was the cause of change if a substantial change is observed in the second set of observations.
Possibility for other event outside of experiment could happen around same time as treatment.<br>
22
10.Control-Group Time-Series Design Variation of Time-Series Design
Two groups observed over a period of time, one does not receive treatment
Greater internal validity than design 9.<br>
23
11. Reversal Time-Series Design Within subjects approach to minimize probability of effect of outside impacts.
Experimental treatment sometimes present, sometimes absent
Measure dependent variable at regular intervals.<br>
24
12. Alternating-Treatments Design Variation on reversal time-series design.
Includes two or more different experimental treatments (Txa and Txb)
Over a long enough time span, hope to see different effects for each treatment.<br>
25
13. Multiple-Baseline Design Design 11 and 12 are not useful if treatment is not temporary and has a long-lasting effect.
Requires at least 2 groups.
Baseline data from all groups is collected before the treatment, then treatment is introduced at a different time for each group.<br>
26
Activity In breakout rooms, you will be working in groups of 2-3 to come up with a study design that works for your research proposal based on the designs we discussed today.
Consider the following questions:
1. Which of the designs presented today did you choose?
2. How does the specific design fulfill your research needs?
3. How does your choice impact the internal validity of your study design?<br>