Topics in Special Education Research - PowerPoint Presentation

Slide1

Topics in Special Education Research

Session 3-Experimental & Quasi-experimental Research

Slide2

This Evening’s

Agenda

Review Course Requirements & Upcoming Assignments

Ethics in Research

Review for, Take, & Correct Quiz

Discussion & Lecture on Experimental Research

Activity

Dismissal

Slide3

Discussion Guides- Please submit to

dropbox folder and label3 main points for all of the readings for that session.No need to double space

Address all of the headings (even the difficulties, concerns, questions section).

Use complete sentences

These are designed to guide your discussions.

Please put your questions for ME on the feedback guide, so if they aren’t answered by your discussion group, type them on the feedback guide!

Updates/Questions

Slide4

Today

NIH/CITI training modules August 5th :

Annotated Bibliography (individual assignment)

Each individual

reviews 3 research articles regarding their topic

See Example: http://rxsped596.pbworks.com/w/file/fetch/54804527/Example%20Annotated%20Bibliography.pdf

August 5

th : Conceptual Framework (group assignment)Group submits short summary of literature and presents a conceptual framework for theories that drive their proposal.August 7th : Article Review Assignment (group assignment)August 12th : Written Research Proposal (group assignment)August 14th : Presentation of Research Proposal (group assignment)

Upcoming Assignments

Slide5

Briefly and clearly state how each research question will be addressed.

For example, “This research question will be answered by comparing the end-of-year state wide test scores of students who received the intervention and those who did not receive the intervention.”

Briefly present the proposed theoretical and practical implications of the findings.

e.g.,

“The results of this study may have implications for the use of the evidence-based Super-Duper Reading Intervention by elementary special education teachers…”

Research Question(s) Guidelines

Slide6

Framed based to operationalize (clearly define so that it can be replicated) the objectives of the proposed research project.

Mention the IV and DV and how they will affect each otherFramed based on methodology:

Experimental/Quasi-experimental/Single-subject: “Is there a causal of functional relationships between IV and DV”

Correlational

: “Is there a

correlational relationship between IV and DV” Descriptive/Qualitative: Describe a phenomenon or issue better, “What is the prevalence of intellectual disabilities in African-American middle school students?”

Research Question Tips

Slide7

What did you learn from doing the CITI course?Regarding…

Research misconductData ManagementResponsible authorshipCollaboration in Research

http://www.youtube.com/watch?v=

W7sfIA1dIGQ

http://www.youtube.com/watch?v=

yr5cjyokVUshttp://www.youtube.com/watch?v=b8McGyYAwcU

Ethics in Research

Slide8

1. Respect for Persons-

Required to: obtain informed consent & respect privacy of participants2. Beneficence- Required to: use best possible design to maximize benefits & minimize harms, show they can perform the procedures and handle the risks, prohibit research that that is without a favorable risk-benefit relationship

3. Justice

Required to: select participants equitably & avoid exploitation of vulnerable populations or populations of convenience.

The Belmont Report (1979), National Commission—3 Principles

Slide9

PSU Human Subjects Research Review Committee (HSRC)

http://www.rsp.pdx.edu/compliance_human.php

Portland State University (PSU) is responsible for the rights and welfare of human subjects involved in research sponsored or conducted by the university.   In order to meet this responsibility, the University established the Human Subjects Research Review Committee. 

Members are charged with reviewing all research conducted under the auspices of PSU that involves human subjects to ensure adequate protections are in place.

Slide10

Review for Quiz

Slide11

In-text Citations: Formatting Quotations

Caruth

(1996) states that a traumatic response

frequently entails a “delayed, uncontrolled repetitive appearance of hallucinations and other intrusive phenomena” (p.11).

A traumatic response frequently entails a “delayed, uncontrolled repetitive appearance of hallucinations and other

intrusive phenomena

” (

Caruth

, 1996, p.11).

When quoting, introduce the quotation with a signal phrase. Make sure to include the author’s name, the year of publication, the page number, but keep the citation brief—do not repeat the information.

Slide12

In-text Citations: A Work with Two Authors

When citing a work with two authors, use “and”

in between authors’ name in the signal phrase yet “&” between their names in parenthesis.

According to feminist researchers

Raitt

and Tate (1997), “It is no longer true to claim that women's responses to the war have been ignored” (p. 2).

Some feminists researchers question that “women's

responses to the war have been ignored” (Raitt & Tate, 1997, p. 2).

Slide13

In-text Citations: A Work with Three to Five authors

When citing a work with three to five authors, identify all authors in the signal phrase or in parenthesis.

(

Harklau

,

Siegal, & Losey, 1999)

In subsequent citations, only use the first author's last name followed by "et al." in the signal phrase or in parentheses.

(Harklau et al., 1999)

Slide14

In-text Citations: A work with 6 or more authors

When citing a work with

more than 6

authors, you do NOT have to identify all authors in the signal phrase or parenthesis.

You identify the first author and use

et al. after his/her name with the date of the publication (Carr et al., 1999) OR

signal phrase in text e.g.:

“Carr et al. (1999) conducted a meta-analysis of behavior support practices…”

Slide15

Steps in the

Research/Scientific Process

1. Identify

socially important issue

2. Review current literature

3. Define conceptual model4. Define specific hypothesis(es

) and research question(s)

5. Define dependent variable(s)/measure6. Identify independent variable(s)/measures7. Select appropriate research design8. Obtain consents9. Collect data10. Analyze data11. Communicate resultsWritten presentationOral presentation

Slide16

Theory that drives the research.

Guides our thinking and provides “rules”, “principles” that guides the research and practice.Structure of assumptions, principles, and rules that holds together the ideas of a broad concept. Outlines your research

Conceptual Models

(or theoretical models)

Slide17

“Logical Flaws” of FBA use in public schools

(Scott et al., 2005)

FBA is used mainly as a reactionary approach.

opportunity is lost to utilize FBA technology to develop interventions that address minor behaviors that usually precede more serious problems.

FBA is restricted to set of procedures used by “experts”

The rich supply of information from people with whom the student interacts with the most is lost.

FBA is restricted to rigorous procedures that are unrealistic for public school settings.

Disincentive for using FBA technology. Cynicism as to the practicality of FBA .

Slide18

Proactive…Parsimonious…Practical FBA in schools

FBA conceptualized by Scott et al. as a proactive pre-referral routine that uses the most parsimonious procedures required to create an effective behavior support plan.

Given the time & resource constraints in schools, we must encourage schools to “work smarter” to develop capacity to implement technology to effectively support more students.

Use Practical FBA procedures to develop capacity within a school to utilize FBA technology.

Slide19

Practical FBA Logic Model

School-wide Positive Behavioral Supports

80% of Students

Secondary Group Supports

10-15% of Students

Individualized Supports

5% of

Students

Behavior Specialist responsible for 25 FBAs in school of 500

Personnel with “flexible” roles conduct proactive Practical FBA to expand the scope of FBA, prevent intensive problem behaviors, & decrease reliance on specialist.

Slide20

A) You should educate readers about the topic and provide a clear

rationale as to why the study is important and necessary based on the previous research and writing on the topic.

B) Within

your literature review you will present the

logic or conceptual framework

as to why and how your current study (topics, methods, designs) is organized the way it is.C) Make this section compelling.

Concisely explain the social importance of what you are studying. e.g., Start with a powerful statement or statistic Literature Review Guidelines:

Slide21

Quiz

Slide22

Correct Quiz

Slide23

Get together with 2-3 other people and use your discussion guides to “guide” your discussion.

Spend more time on the chapter reading and the Quality Indicator Article as we will be doing an activity on the other two articles later!Discussion

Slide24

Experimental & Quasi Experimental ResearchResearch Designs & Threats to Validity

Attempt tonight to apply the principle of teaching “less more thoroughly”….

https://

www.youtube.com/watch?v=qtLnBz6lbRQ

Lecture

Slide25

“Gold Standard” in Research (Moore & McCabe, 1993;

Feuer, Towne, & Shavelson, 2002; Slavin, 2002).

Rooted in

postpositivist

paradigm.

Seeks to make causal conclusions.Difference between experimental design and quasi-experimental design is the use of random selection of participants.

Experimental Design

Slide26

Emphasis is on operationally defining the variables

(dependent & independent) and the context of the research.Dependent variable(s)- outcome variables (e.g., reading scores)

Independent variable(s)- variable that is manipulated (the intervention or practice; e.g., reading curriculum)

Context- defined clearly so replication can occur (e.g., K-3 school with 200 students, etc.)

In both experimental & quasi-experimental designs….

Slide27

Experimental (or treatment group)- receives the intervention

Control group- business as usual.For true “experimental” research, participants are randomly assigned to each group. In order for to be considered random, every person must have an equal chance of being in either group

Experimental Group

vs

Control Group

Slide28

Direct manipulation of an independent variable (intervention)

Experimental and quasi-experimental research involves…

Slide29

Refers to whether a study is able to scientifically answer the questions it is intended to answer.

Extent to which your test (or study) measures what it intends to measure.

Validity

Slide30

Changes observed in the dependent variable (outcome) are due to the effect of the independent variable (intervention)…..& not to some other unintended variables (extraneous, alternative explanations)

12 threats to internal validity (noted by Mertens,

2010)

https://www.youtube.com/watch?v=_

UPUtlHDM0A

Internal Validity

Slide31

History- events other than IV affected results?

Maturation- changes in participants?Testing- participants became “test-wise”?

Instrumentation- difference between pre- and posttests?

Statistical regression- extreme groups used?

Differential selection- groups differed in ways other than exposure to IV?

12 Threats to Internal Validity

(think when something other than the IV affect the results of a study)

Slide32

7. Experimental mortality- drop outs of study?

8. Selection-maturation- was selection a problem based on the characteristics of the sample? (e.g., participants in one group may have been older)9. Experimental treatment diffusion- treatment & control groups shared information?

10. Compensatory rivalry by the control group

11. Compensatory equalization of treatments –extra resources given to control group?

12. Resentful demoralization of the control group

Threats to Internal Validity continued

Slide33

External Validity= extent to which findings in one study can be applied to another situation.

AKA: ecological validity, generalizability10 threats posed as questions (noted by

Mertens

, 2010)

External Validity

(think generalizability)

Slide34

Were the variables, context, and treatment described in sufficient detail?

Were multiple treatments used? Did they interfere with each other?Was the Hawthorne effect (special attention as part of study affecting results) operating?

Was the treatment influenced by being novel or disruptive?

What was the influence of the individual experimenter?

10 Threats (questions) to External Validity

Slide35

6. Were the participants sensitized by taking a pretest?

7. Were the participants sensitized by taking a posttest?8. What was the influence of the type of measurement used for the dependent variable? 9. Was there an interaction of history and treatment effects?

10. What was the influence of the time period that elapsed between the treatment and administration of the dependent variable?

Questions to external validity continued

Slide36

Treatment fidelity- implementer of the independent variable follows the exact procedures specified for administering the treatment(s).

Treatment Fidelity…another threat

Slide37

May not be reasonable to expect participants to improve given the “strength” of the intervention.

Intervention may not have been tried long enough or delivered with adequate intensity.

Strength of treatment…another threat

Slide38

R= Random assignment of subjects to conditionsX=Experimental treatment (e.g., reading curriculum)

O= Observation of the dependent variable (e.g., test or observation measure)

Coding system used for research designs

Slide39

One-shot case study X O

Threats:History, maturation, & mortality (drop out)Other concerns using this design?

No control group & No pretest to know if it was the intervention that affected outcome.

Very weak design

Single Group Designs

Slide40

One-Group Pretest-Posttest DesignO X O

Threats: History, maturation, What would help control for these threats?

Control group- both groups taking the tests at same time, but one not receiving the intervention

But sometimes it is difficult to find a control group

Slide41

Involves measurement of the dependent variable at periodic intervals.

O O O

O

O X O O O O O

If behavior is stable in baseline (before intervention), then change can be attributed to intervention.

Controls for several threatsMaturation, testing, differential selection (same persons involved) https://www.youtube.com/watch?v=GUq_tO2BjaUTime Series Design

Slide42

Pre-test-Post-test Control Group Design

R O X OR O OControls for what threats?

history, differential selection, mortality (pre-test can show differences in drop-outs)

Posttest-only Control Group Design

R X O

R OControls for what threats?Same as above, except for mortality (no pretest)

Experimental Designs

Slide43

Single-Factor Multiple-Treatment Designs

X1= intervention 1X2= intervention 2…….R O X1 O

R O X2 O

R O O

Controls for threats because participants randomly assigned to comparison groups and pre-post-tests conducted.

Slide44

If worried about pretesting affecting validity

R O X OR O O

R X O

R O

https://

www.youtube.com/watch?v=7RRVW4iO7gA

Solomon 4-group Design

Slide45

Similar designs to experimental designs, except for……Lacking….

Random assignmentCan’t make a strong causal statementhttps://www.youtube.com/watch?v=_

WpBxERGNVw

Quasi-experimental designs

Slide46

How do we know if a research study involves rigorous, systematic and objective procedures?

CEC-Division for Research

Sponsored prominent researchers to author papers to propose

Parameters for establishing that reported research has been conducted with high quality (quality indicators)

Criteria for determining whether a practice has been studied sufficiently (enough high-quality research studies conducted on its effectiveness) and shown to improve student outcomes (effects are strong enough)

Graham, S. (2005). Criteria for evidence-based practice in

special education [special issue].

Exceptional Children, 71.

Slide47

Exceptional Children

(2005) volume 71(2)

Group Experimental and Quasi-Experimental Research

(Gersten, Fuchs, Comptom, Coyne, Greenwood, & Innocenti)

Single-Subject Research

(Horner, Carr, Halle, McGee, Odom, & Wolery)Correlational Research (Thompson, Diamond, McWilliam, Snyder, Snyder)Qualitative Studies (Brantlinger, Jimenez, Klingner, Pugach, & Richardson)

Slide48

Quality Indicators (QIs) for Experimental

(and Quasi-Experimental) Research

Describing Participants

Sufficient information about participants and interventionists, selection procedures as well as comparability across conditions

Implementation of Intervention and Description of Comparison Conditions

Clear description of intervention (and comparison conditions) with

implementation fidelity assessed

Outcome MeasuresUse of multiple measures at appropriate timesData AnalysisAnalysis techniques appropriate to questions and unit of analysis with effect size calculated

Slide49

Statistics, statistics

Descriptive Statistics

Who is in your data?

sample

population

Inferential Statistics

What your sample says about the population

sample

population

Mean, Median, Mode,

Standard

D

eviation, Variance

Tests of significance (t-, F-Tests)

Slide50

Tests of Significance

Statistical analyses to determine whether a difference is statistically significant (probability for result to occur by chance).

Yes or No answer

Alpha

level (p=)

An established probability level which serves as the criterion to determine whether to accept or reject the null hypothesis

Common levels in education

.01.05.10Objectives 4.1 & 6.1

Slide51

Statistics, statistics

Descriptive Statistics

Who is in your data?

sample

population

Inferential Statistics

What your sample says about the population

sample

population

Mean, Median, Mode,

Standard

D

eviation, Variance

Tests of significance (t-, F-Tests)

Slide52

T tests- used when have two groups to compare.

Independent samples t- if groups are independentDifferent people in each groupDependent samples t-: if two sets of scores are available for the same people (e.g., pre and post-tests of same group)

Matched groups

ANOVA (analysis of variance)- when you have more than 2 groups to compare OR more than one independent variable (reports an F-statistic, which is basically a t-value squared)

ANCOVA (analysis of covariance)- ANOVA that allows for control of the influence of an IV (e.g., characteristics of people) that may vary between your groups before treatment is introduced.

Post-hoc method for matching groups on variables such as age, prior education, SES, or a measure of performance

Inferential Statistics

Slide53

Way of quantifying the difference between two groups.

Not just was there an effect, but the magnitude of the effect.Many ways to calculate

ES= [Mean of experimental group] – [Mean of control group]/Standard Deviation

R-squared,

Cohens

-DStandard deviation is how well the mean summarizes the data

Effect Size

Slide54

Slide55

Slide56

Over 800 Meta-analyseshttps://www.youtube.com/watch?v=

sng4p3Vsu7Y

Visible Learning

by John Hattie

Slide57

Implications for your practice- why you selected the article

Introduction- what was the aim of the article, what were the specific research questionsMethods- Describe:ParticipantsSetting of study

Target behaviors- dependent variables

How they measured the dependent variables

Materials used

Intervention procedures used- specifyWere the procedures replicable?

Article Review: Use questions as your guide

Slide58

Methods used for treatment integrity

This asks how well does the author measure the implementation of the interventionWhat was the design? Experimental, quasi-experimental, single-subject (specify what type of design…multiple baseline, etc.)

Results- give quantifiable information of how the intervention worked

Discussion- issues discussed, limitations?

Conclusion- did they answer the research question?

Reflection- commentary, questionsArticle Review: more questions

Slide59

Get together with a partner.

Practice completing an article review for one of the articles you read this week. Note the design of the study.

In-Class Activity

Slide60

A detailed explanation of the assignment is posted on the wiki

What should you be doing in your groups?At this point you should have a topic and start coming up with your framework for your research project (based on literature).

Start to draft your conceptual framework, research questions & identify your dependent and independent variables

You should walk away from your group time with a list of tasks to complete.

Proposal Assignment & Group Work

Slide61

Socially Important Issue:

2. Conceptual Model/Hypothesis:

3. Research Question(s):

4. Dependent Variable:

5. Dependent Variable Measure:

6. Independent Variable:

7. Independent Variable Measure:

8. Research Design:

Slide62

Self-evaluation & feedback