/
Understanding and Operationalizing Evidence-Based Practices within Multi-Tiered Systems Understanding and Operationalizing Evidence-Based Practices within Multi-Tiered Systems

Understanding and Operationalizing Evidence-Based Practices within Multi-Tiered Systems - PowerPoint Presentation

magdactio
magdactio . @magdactio
Follow
345 views
Uploaded On 2020-08-06

Understanding and Operationalizing Evidence-Based Practices within Multi-Tiered Systems - PPT Presentation

Allison Gruner Gandhi Lynn Holdheide Rebecca Zumeta Edmonds Chris Lemons Copyright 2016 American Institutes for Research All rights reserved April 15 2016 Lessen anxiety around EBPs ID: 800352

based evidence ebps instruction evidence based instruction ebps practices tier http org intervention students resources student progress monitoring research

Share:

Link:

Embed:

Download Presentation from below link

Download The PPT/PDF document "Understanding and Operationalizing Evide..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Understanding and Operationalizing Evidence-Based Practices within Multi-Tiered Systems of Support

Allison Gruner GandhiLynn HoldheideRebecca Zumeta EdmondsChris Lemons

Copyright © 2016 American Institutes for Research. All rights reserved.

April

15,

2016

Slide2

Lessen anxiety around EBPs!Make the case for flexibility in evidence criteria across MTSS tiersShare resources available through TA Centers

Core instructional practiceEvidence for interventionToday’s Presentation: Goals 2

Slide3

Every Student Succeeds ActRequires use of “evidence-based interventions” in low performing schools

Four categories of evidence:"strong evidence“: supported by at least one randomized study"moderate evidence“: supported by at least one quasi-experimental study"promising evidence“: at least one correlational study with pretests as

covariatesprograms with a rationale based on high-quality research or a positive evaluation that are likely to improve student or other relevant outcomesSchool improvement interventions must be supported by categories 1-3 

Role of EBPs in Federal Policy

3

Slide4

OSEP’s Results-Driven Accountability (RDA) Initiative States accountable for compliance AND results

State systemic improvement plans include strategies to achieve a “state-identified measurable result (SiMR)” A child-level (or family-level, for Part C) outcomeNot a process or system resultMay be a single result or a cluster of related resultsIdentified based on analysis of dataRole of EBPs in Federal Policy

4

Slide5

“The focus of Phase II is on building State capacity to support local educational agencies (LEAs) with the implementation of

evidence-based practices (EBPs) that will lead to measurable improvement in the State-identified Measurable Result(s) (SIMR) for children with disabilities”Source: OSEP. (2015). Part B State Systemic Improvement Plan (SSIP) Phase II OSEP Guidance and Review Tool. Retrieved from: https://osep.grads360.org/#communities/pdc/documents/8823 EBPs in the SSIP

5

Slide6

Improvement strategies must be “based on the best available research, aligned to data analysis and the infrastructure analysis, and support systemic change”

“The State should provide the evidence it used to make decisions on appropriate improvement strategies to improve the SIMR.” Source: http://ncsi.wested.org/wpcontent/uploads/2015/02/CompilationofOSEPGuidanceonSSIP_January2015_Release.pdf

EBPs in the SSIP6

Slide7

Inconsistent Terminology: Authors and organizations refer to EBPs in a variety of ways

Evidence-based practiceResearch-based practiceScientifically-based practicePromising practiceBest practice Challenge: They’re Everywhere, but What are They?7

Slide8

Inconsistent Criteria: Authors and organizations define criteria for EBPs in different ways

Quality of evidence (study design)Direction and magnitude of resultsQuantity of evidenceSource of evidenceSome combination of all these thingsChallenge: They’re Everywhere, but What are They?8

Slide9

Are we Taking EBPs a bit too Seriously?

9

I can tell you’re struggling with fractions, Tony, but there aren’t any EBPs for kids in your grade. Maybe check back in a few years?

Slide10

What are EBPs and when can I benefit from them?

Can I adapt an EBP?How do I know if it’s working?Common EBP Questions10

Slide11

ESSA and RDA

11Special

educators:We need EBPs to put in our SSIP for RDA!

What

practices have some good research

?

General

educators:

We

need EBPs to help our lowest performing

schools! What

practices were studied with an RCT?

Common

aim

:

Implement practices

to help lowest achieving

students

Slide12

Could MTSS Help Provide an Organizational Framework for Communicating about the Evidence Base?

12

In MTSS, the “practice” in “evidence-based practice” is different in each tier!

Wouldn’t this mean that criteria for evidence should differ too?

Slide13

Characteristics of Intervention Levels/Tiers

13

Primary (T1)

Secondary (T2)

Intensive (T3)

Instruction/

Intervention

Approach

Comprehensive

research-based curriculum

Standardized

, targeted

small-group instruction

I

ndividualized

,

based on student data

Group

Size

Class-wide

(with some small group instruction)

3

7

students

No more than 3 students (ideally)

Monitor

Progress

1x per term

At least

1x per month

Weekly

Population

Served

All

students

At-risk students

Students with significant

& persistent needs

Slide14

But What About Evidence of Effectiveness at Each Level?

14

Slide15

Role of Evidence

15

Primary (T1)

Secondary (T2)

Intensive (T3)

Instruction/

Intervention

Approach

Comprehensive

research-based curriculum

Standardized

, targeted

small-group instruction

I

ndividualized

,

based on student data

Evidence?

Comprehensive

coverage of critical content

Instructional practices and strategies with evidence of efficacy

Intervention

aligned to target skill(s)

Standardized

program with demonstrated efficacy

Individualization of intervention, embedding instructional strategies and supports based on student data

Frequent

, ongoing progress monitoring to determine impact

Slide16

Core Instructional Practice

16

Slide17

TerminologyEvidence-based practice

Research-based practiceScientifically-based practicePromising practiceBest practice High Leverage practices (HLPs)Mixed MessagesPreparation

Certification and Licensure StandardsEducator EvaluationProfessional Learning and Support

Instructional Practice

17

Slide18

What Does it Feel Like for Teachers?

18

Slide19

Result

19

Slide20

Shared ownership and accountability for the progress of all students

Access to the curriculum (e.g., standards)Instruction and practices that are effective at reaching the bulk of studentsScreening and progress monitoring to Determine effectiveness of instruction and practicesIdentify students that are struggling

Adjust/differentiate instructional strategies and practices accordingly

Solicit consultation, support, and services

Recommendations for Tier 2 support

Tier 1 Instruction: What is the Goal?

20

Slide21

Challenges

Variation in contextImplementation fidelity/qualityStudent characteristicsEnd ResultMissed opportunity for shared ownershipLack of alignment in expectations

Decreased confidence among general education teachers in supporting students with disabilities and struggling learnersMore students referred to tier 2 and 3Defining Evidence-based Practices at Tier 1

21

Slide22

What evidence should we look for?Gold Standard:

Rigorous experiments, quasi-experiments, or RDDs of a well-defined practice, strategy, or program (if available) Research-based: Meta-analyses or syntheses of research on instructional strategiesPromising practices: less rigorous, or correlational studiesExpert recommendations: Recommendations from reputable professional organizations (e.g., National Reading or Math Panel) Can/should There be a Happy Medium or Consensus in the Rigor of Evidence at Tier 1?

22

Alignment with grade level

standards

Slide23

Tier 1: Dimensions of Fidelity

23

Element Question

s Addressed

Schedule

Did instruction

occur as planned?

Content

coverage

Was

critical content covered in alignment with standards or recommendations from reputable professional organizations (e.g., National Reading Panel)?

Instructional practices and strategies

Were correct procedures followed

for implementing practices/strategies?

Screening procedures

Were correct screening administration,

and scoring procedures followed?

Slide24

What Works ClearinghouseIES Practice Guides Best Evidence Encyclopedia

IRIS Center: http://iris.peabody.vanderbilt.edu/ebp_summaries/Teaching Works: High Leverage Practices (Ball): http://www.teachingworks.org/work-of-teaching/high-leverage-practices CEEDAR Center: http://ceedar.education.ufl.edu/

Evidence-based Intervention Network: http

://ebi.missouri.edu

/

Resources to Support Identification of EBPs at Tier 1

24

Slide25

Evidence Base

CEEDAR Evidence Standards4 levels of evidenceStrong ModerateLimitedEmerginghttp://ceedar.education.ufl.edu/wp-content/uploads/2014/08/Evidence-Based-Practices-guide.pdf

(CEEDAR Center, 2014)

Slide26

CEEDAR Tools and Resources

Knowledge Development Papers

Innovation Configuration

Course Enhancement Modules

Scientifically Based Reading

Classroom Organization

& Behavior Management

Evidence-Based

Writing Instruction

Evidenced-based Math Instruction

Universal Design for Learning

Leadership

Technology

Inclusive Service

Significant

Disabilities

Sensory Impairments

Culturally Responsive Instruction

Secondary Transition

Linking

 Assessment & 

Instruction

Content Innovation Configuration

Slide27

CEEDAR Tools and Resources

Knowledge Development PapersInnovation Configuration

Course Enhancement Modules

http://ceedar.education.ufl.edu/cems

/

Slide28

Course Enhancement Modules

Writing

Leadership

Secondary Transition

Slide29

Evidence for Intervention

29

Slide30

Intervention

aligned to target skill(s)Standardized program with demonstrated efficacyBest aligned to current thinking about gold standard evidenceTier Two: Secondary Prevention

30

Slide31

Resources to Support Identification of EBPs at Tier 2

31http://www.intensiveintervention.org/chart/instructional-intervention-tools

Slide32

What Works Clearinghouse/IES Practice Guides:

http://www.ies.ed.gov/ncee/wwc/Publications_Reviews.aspx?f=All%20Publication%20and%20Product%20Types,3;#pubsearch Best Evidence Encyclopedia: http://www.bestevidence.org/index.cfm IRIS Center: http://iris.peabody.vanderbilt.edu/ebp_summaries/ EBI Network: http://ebi.missouri.edu

/ RTI Center: www.rti4success.org

Additional Resources

32

Slide33

Tier 2: Dimensions of Fidelity

33Element

Questions Addressed

Schedule

Did instruction

occur as planned?

Adherence to program

Did

the interventionist deliver all components of the program consistent with the way it was designed?

Progress monitoring procedures

Were correct progress

monitoring

administration

and scoring procedures followed?

Slide34

Limitations of Gold Standard Evidence…

34

Why isn’t this working?

Time and fiscal resources that delay movement from the research lab into schools may mean few options in certain areas.

Generalizability to other populations

Generally effective isn’t universally effective… Some kids need more.

Slide35

Individualization

of intervention, embedding instructional strategies and supports based on student data (e.g., precision or experimental teaching procedures) Targeted integration of academic and behavioral supports Frequent, ongoing progress monitoring to determine impact

Tier 3: Intensive Intervention35

Slide36

Data-Based Individualization

36

Slide37

Is the student making progress?

Student level responsiveness data from experimental or precision teaching approaches (e.g., Deno & Mirkin, 1977; Fuchs, Deno, & Mirkin, 1984; White, 1986) Assessment tools with evidence of validity and reliability

What’s the Evidence? 37

Slide38

Resources to Support Identification of EBPs at Tier 3

38Academic & Behavior InterventionProgress Monitoring

Slide39

National Center on Intensive Intervention http://www.intensiveintervention.org

/ Center on Instruction: http://www.centeroninstruction.org/intensive-interventions-for-students-struggling-in-reading-and-mathematics Additional Resources

39

Slide40

Tier 3: Dimensions of Fidelity

40

Element

Question

s Addressed

Schedule

Did instruction

occur as planned?

Adherence to the student’s individualized plan

Was

the student‘s program delivered as the intervention or IEP intended?

Were correct procedures followed in implementing intensification strategies?

Progress monitoring procedures

Were correct progress

monitoring

administration

and scoring procedures followed?

Slide41

Resources for Monitoring Fidelity of the Intensive Intervention System

41

http://www.intensiveintervention.org/resource/dbi-implementation-rubric-and-interview

Slide42

Resources for Monitoring Student-Level Fidelity

42

http://www.intensiveintervention.org/resource/dbi-implementation-rubric-and-interview

Slide43

Tier 3 Lack of complex packaged interventions that

target unique, multi-pronged problems Practitioner expertiseAvailability of appropriate progress monitoring tools Perceptions of “off-level” instruction and assessment Scale up Tier 2Limited availability in many grades and subject areas

Validity across contexts Transportability to different student populations Lack alignment to Tiers 1 and 3, or to student need Feasibility of implementation

Low-incidence populations

Challenges

43

Slide44

Discussion

44

Slide45

45

Connect to NCIISign up on our website intensiveintervention.org to receive our newsletter and announcementsFollow us on YouTube and TwitterYouTube Channel:

National Center on Intensive InterventionTwitter handle: @T

heNCII

Slide46

46

Disclaimer

This webinar was produced under the U.S. Department of Education, Office of Special Education Programs, Award

No. H326Q110005. Celia Rosenquist serves as the project officer.

The views expressed herein do not necessarily represent

the positions or polices of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this webinar is intended or should be inferred.

Slide47

Contact US

National Center on Intensive Intervention1000 Thomas Jefferson Street NWWashington, DC 20007-3835

www.intensiveintervention.org

ncii@air.org

@

TheNCII