Longitudinal - PowerPoint Presentation

Slide1

Longitudinal Analysis of Effects of Reclassification, Reporting Methods, and Analytical Techniques on Trends in Math Performance of Students with Disabilities

Yi-Chen Wu, Martha Thurlow, & Sheryl LazarusNational Center on Educational OutcomesUniversity of Minnesota

This paper was developed, in part, with support from the U.S. Department of Education, Office of Special Education Programs grants (#H373X070021and #H326G110002). Opinions expressed herein do not necessarily reflect those of the U.S. Department of Education or Offices within it.Slide2

NCEO Web site(http://www.cehd.umn.edu/nceo/) Slide3

OutlineBackgroundAchievement gapExplanationsYsseldyke and Bielinski (2002) study QuestionsMethodData source

Analytical TechniquesResultsConclusionsSlide4

Achievement gap

4

Focused on race/ethnicity or poverty.

Less attention on

achievement gaps

between SPED vs. Non-SPED

Research on Achievement Gap (

Chudowsky

,

Chudowsky

, &

Kober

, 2009a; 2009b)

Examined gaps for subgroups by proficiency rate & mean SS, but no comparison between SPED and Non-SPED

Examined the achievement over time for SWD, but not the gap between SWD vs. SWOD over timeSlide5

Explanations on gap increasing over time between SWD and SWOD

5

SPED drop out of school=high achievement (

McMillen

& Kaufman, 1997)

Tests given in higher grades are less valid for SWD (

Thurlow

&

Ysseldyke

, 1999;

Thurlow

,

Bielinski

,

Minnema

, & Scott, 2002)

Students with lower performance moved in SPED and students with higher performance move out SPED (

Ysseldyke

and

Bielinski

, 2002)Slide6

Ysseldyke and Bielinski (2002) study Explored the extent to which reclassification impacts the size of the achievement gap between GED and SPED across grades.to compare the effects of different reporting methods, and to examine the effects of reclassification

They argued that fair comparisons involved using clearly defined and consistent comparison groups, and that special education status complicates the reporting because status changes over time. Slide7

Ysseldyke and Bielinski (2002) study They used three methods to analyze trends in performance (cross-sectional, cohort-static and cohort-dynamic), and found that gap trends depended on the method usedexamined how the use of scaled scores and effect size could be used for reporting results.Slide8

PurposeThe Ysseldyke and Bielinski (2002) study did not use proficiency to examine the reporting resultsis now more than a decade old was completed prior to the implementation of ESEA 2001. There is a need to take a new look at how achievement gap trends are affected by the method used to calculate them. Slide9

Research QuestionsReporting Methods: How does the use of cross-sectional, cohort-static, and cohort-dynamic data analysis methods affect interpretation of trends in the performance of students with disabilities?Analytical Techniques: How does the score used in the analyses (proficiency level, scaled score, effect size) affect interpretation of trends and achievement gaps?Reclassification: To what extent do students move in and out of special education each year, and what are the achievement characteristics of those who do and do not move?Slide10

MethodData sourceused math assessment data for grades 3-8 from a midwestern stateCross-sectional2005-06 to 2009-10 305,819 recordsCohort2005-06 to 2009-10+ 2004-05 (G3-8)8,231 students with 6-yr recordsSlide11

Method- Methods Used to Measure GapCross-sectionalfive years of data were used to calculate the average performance to reduce year-to-year variations that might affect results if data from a single year were selected. Cohort-staticA cohort across six yearsGroup membership stayed the same across years.

Cohort-dynamicgroup membership was redefined every yearSlide12

Method- Analytical Techniques Slide13

Results—RQ1How does the use of cross-sectional, cohort-static, and cohort-dynamic data analysis methods affect interpretation of trends in the performance of students with disabilities?Using PF to show the trend over time among the three methods used to measure gapSlide14

Figure 1. Cross-sectional method: Percentage of students above proficiency level on math assessment by SPED and non-SPEDResults—Comparing reporting methods

21-->47Slide15

Results—Comparing reporting methodsFigure 2: Cohort-static method: Percentage of students above proficiency level on math assessment by SPED and Non-SPED

22->21Slide16

Results—Comparing reporting methodsFigure 3. Cohort-dynamic method: The percentage of students above proficiency level on math assessment by SPED and Non-SPED

22-->45Slide17

Results—Comparing reporting methodsQuit differentQuite similarSteady

Cohort-dynamic

Cohort-static

Cross-sectionalSlide18

Results—RQ2 How does the score used in the analyses (proficiency rate, scaled score, effect size) affect interpretation of trends and achievement gaps?Slide19

Figure 4. Percent proficient: Achievement gap (difference between non-SPED and SPED) in percent proficient on math assessmentResults—Comparing Analytical TechniquesSlide20

Figure 5. Scaled score: Achievement gap (difference between non-SPED and SPED) in mean scaled score on math assessmentResults—Comparing Analytical TechniquesSlide21

Figure 6. Effect size: Achievement gap (difference between non-SPED and SPED) in effect size on math assessmentResults—Comparing Analytical TechniquesSlide22

Results—Comparing analytical techniquesQuit differentQuite similarSteady

Effect size

Scaled Score

Proficiency LevelSlide23

Results—RQ3To what extent do students move in and out of special education each year, and what are the achievement characteristics of those who do and do not move?Slide24

Figure 7. Mean math scaled scores by special education status across yearsResults—Reclassification

Note: NS1 = Students who remained in non-special education in both of two consecutive years; NS2 = Students who moved from non-special education to special education in the second of two consecutive years; S1 = Students who remained in special education in both of two consecutive years; S2 = Students who moved from special education to non-special education in the second of two consecutive years.Slide25

Results—ReclassificationNon-SPED onlyStudents stayed in non-SPED for six yearsNon-SPED to SPEDStudents moved from non-SPED to SPED only once over six yearsSPED to Non-SPEDStudents moved from SPED to non-SPED only once over six

yearsBack and forthStudents moved between SPED and non-SPED more than once over six yearsSPED onlyStudents stayed in SPED for six yearsSlide26

Figure 8. The effect size between different reclassification groups in math assessment by using non-SPED only group as the reference groupResults—ReclassificationSlide27

Discussion and ConclusionDifferent methods of reporting data present different pictures of the gap between SPED and non-SPEDThis study was undertaken to update the work done more than a decade ago by Ysseldyke and Bielinski (2002)Replicated + proficiency levelConfirmed SuggestionsSlide28

Discussion and ConclusionSuggestionsThe choice of method affects what the results look like and the possible interpretation of findings.Tracking individual student performance provides a better indication of how well schools are educating their students than cross-sectional models where the grade remains the constant but the students change. Cross-sectional models should not be used when examining trends across grades. Cohort-static and cohort-dynamic methods enable educators to make comparisons among individual studentsSlide29

Discussion and ConclusionSpecific situation for each reporting methodIf the goal is to know how well students do yearly without considering changing students => cross-sectionalhttp://www.schooldigger.com/go/MN/schools/3243001386/school.aspxIf states and districts want to account with precision for the reclassification of students each year. => cohort-dynamicWhen the goal is to account for individual student performance over time without regard to the nature of services received=> cohort-static