US Department of Education March 16 2015 Copyright 2015 American Institutes for Research All rights reserved Agenda 900 915am Welcome and Overview 915 1105am Panel ID: 904619
Download The PPT/PDF document "Assessing the English Language Proficien..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Assessing the English Language Proficiency of English Learners With Disabilities
U.S. Department of EducationMarch 16, 2015
Copyright © 2015 American Institutes for Research. All rights reserved.
Slide2Agenda
9:00 – 9:15am Welcome and Overview
9:15
–
11:05am Panel Session 1: Background – Differentiating Language and Literacy Acquisition From Disability11:05 – 11:15am Break11:15 – 1:05pm Panel Session 2: Fostering Valid and Reliable Assessments for ELs With Disabilities1:05 – 2:00pm Lunch2:00 – 3:00pm Panel Session 3: Assessing ELs With Significant Cognitive Disabilities
2
Slide3Panelists
Panel Session 1: Background – Differentiating Language and Literacy Acquisition From Disability
Presenters: Aquiles Iglesias; Esther Geva; Sylvia Linan-Thompson; Alba Ortiz
Moderator: Diane August
Panel Session 2: Fostering Valid and Reliable Assessments for ELs With DisabilitiesPresenters: Martha Thurlow; Jamal Abedi; Phoebe Winter; Gary CookModerator: Robert LinquantiPanel Session 3: Assessing ELs With Significant Cognitive Disabilities Alternate ELP Standards and Assessments, and Growth and Attainment CriteriaPresenters: Martha Thurlow; Gary CookModerator: Brian Gong
3
Slide4Meeting Purpose
Build the knowledge base related to ELs with disabilities and inform future research and development in this area Inform the work of a panel constituted by the US Department of Education
Panel constituted to develop a peer review guide related to Title III assessment and accountability provisions.
Peer review guide will assist SEAs in preparing materials for the review and peer reviewers in conducting the review. Guide includes elements related to Title III assessment and accountability and evidence states must provide to show compliance with provisions of the law.
4
Slide5Background: Title III
Title III of ESEA provides federal funding to state and local education agencies to develop language instruction programs that assist ELs in acquiring English and meeting the same academic content standards as their English-proficient peers.
It also inaugurated important changes in assessment and accountability for EL students.
5
Slide6Background: Title III
Title III of the law requires states to establish state standards for English language proficiency (ELP) that correspond with state academic content standards required under Title I.
It requires an annually administered English language proficiency assessment based on those standards and measuring the four domains of reading, writing, listening, and speaking.
6
Slide7Background: Title III
Title III also instituted new accountability requirements for districts and states.
New EL accountability provisions require states to:
define criteria for progress in learning English
establish a performance standard for English language proficiency and academic content knowledgeset annually increasing performance targets for the number and percentage of ELs meeting these criteria
7
Slide8Background: Title III
The Annual Measurable Achievement Objectives include:AMAO 1
: annual increases in the number or percentage of ELs making progress in learning English;
AMAO 2
: annual increases in the number or percentage of ELs attaining English language proficiency (ELP) by the end of each school year as determined by a valid and reliable assessment of ELP; and AMAO 3: annual increases in the number or percentage of ELs making adequate yearly progress (AYP) on content area assessments.
8
Slide9Demographics
In 2012–13: ELs
comprised 9% of students enrolled in U.S. public
schools
There were 543,916 English learners with disabilities (this represents 8.5% of ELs and 13% of all students with disabilities)
9
Slide10Identification of
Speech-Language Impairment in ELsAquiles Iglesias
University of Delaware
March 16, 2015
10
Slide11Overview
What we know about language acquisition in ELsImplications for assessment
Identification of:
Students with True Language Impairment
Students with language learning difficulties not eligible for special servicesTypically developing students who are not making adequate progress
11
Slide12Knowledge Base and Implications for Assessment
Language acquisition Innate capacity to learn
Linguistic environment
Must assess:
Process of learningProduct of learning (innate capacity + linguistic environment)
12
Slide13Knowledge Base and Implications for Assessment
Growth trajectories of oral measures related to
literacy
for
TD K-3 ELs Must assess using language development norms, trajectories, and growth rates of similar studentsGenderBeginning L2 proficiency statusLanguage learning ability
13
Slide14Knowledge Base and Implications for Assessment
Children acquiring 2 languages
Knowledge acquired is distributed across 2 languages
Distribution is not always equal
14
Slide15Unequal Distributed Knowledge
VOCABULARY
15
Slide16Knowledge Base and Implications for Assessment
Children acquiring two
languages
Knowledge
acquired is distributes across two languages (not always equal)Total score in one language only provides partial information.Best measure of language ability takes into consideration distributed knowledge. Early identification and intervention results in better and more cost effective outcomes.Assess early
16
Slide17The Ideal Assessment Should:
Occur earlyAssess process and product
Account for distributed knowledge
Compare performance to similar students
StaticGrowth
17
Slide18Language Assessments for Spanish-English Speaking Children*
Preschool Computerized Language Assessment
(PCLA)
Funding: Institute of Education Sciences Grant R305A110284
Bilingual English Spanish Assessment (BESA)Funding: NIDCD N01-DC-8-2100Systematic Analysis of Language Transcripts (SALT)Partial Funding NIH and IES Grants HD59321 and R305U010001
*Disclosure: Aquiles Iglesias has financial interests in all three assessments
18
Slide19Our Task
19
Slide20Preschool Computerized Language Assessment (PCLA)
20
Slide21Our Task – Screening With PCLA
English Total Scores and Component Scores
Spanish Total Score and Component Scores
Best Score Total
21
Slide22Our Task – Screening With PCLA
English Total Scores and Component Scores
Spanish Total Score and Component Scores
Best Score Total
22
Slide23Our Task – Identification of True Language Impaired
23
Slide24Bilingual English Spanish Assessment (BESA)
Domains
Semantics
Morphosyntax
PhonologyLI Markers vary across languagesEnglish: Past tense, Plural NounsSpanish – Articles and Clitics
24
Slide25Bilingual English Spanish Assessment (BESA)
25
Slide26Our Task – Identification of True Language Impaired Older Students
Systematic Analysis of Language Transcripts (SALT)
Normed on over 4,000 English and Spanish Language Samples
Grades K-3, Ages 5 – 9
26
Slide27Elicitation Procedure
Model Retell Tell
27
Slide28Decision Making Process
Assess Language X
Compare to Language X Norms
WITHIN NORMAL LIMITS
Assess Language
Y
Compare to Language Y Norms
WITHIN NORMAL LIMITS
28
Slide29English
Spanish
29
Slide30Monitoring
Progress-Language Samples
30
Slide31Monitoring
Progress-Language Samples
31
Slide32Identification of Students
Students with True Language Impairment
Students with language learning difficulties not eligible for special services
Typically developing students that are not making adequate progress
32
Slide33References
Iglesias, A., & Rojas, R. (2012). Bilingual language development of ELLs: Modeling the growth of two languages. In B. Goldstein (Ed.),
Bilingual language development and disorders: Past, present, and future
(pp. 1-30). Baltimore, MD: Paul H. Brookes.
Miller, J. F., Heilmann, J., Nockerts, A., Iglesias, A., Fabiano, & Francis, D. (2006). Oral language and reading in bilingual children. Learning Disabilities Research and Practice, 2(1), 30-43. Peña, E. (2007). Lost in Translation: Methodological considerations in cross-cultural research. Child Development, 78, 1255-1264.Rojas, R., & Iglesias, A. (2009, March 3). Making a case for language sampling. The ASHA Leader, 14(3), 10-13.
Rojas, R., & Iglesias, A. (2013). The language growth of English language learners: Change over time, individual differences, and the impact of initial status on growth.
Journal of Child Development
,
84
(2), 630-646.
33
Slide34Assessments
Golinkoff, R., de Villiers, J., Hirsh-Pasek, K., Iglesias, A., & Wilson, M. (under development). Preschool Computerized Language Assessment.
Iglesias, A., & Miller, J. (2012). Systematic Assessment of Language Samples. SALT Software, LLC, Middletown, WI. (saltsoftware.com)
Peña, E., Gutierrez-Clellen, V., Iglesias, A. Goldsteim, B., & Bedore, L. (2014). Bilingual English Spanish Assessment. A-R Clinical Publications, San Rafael, CA.
34
Slide35Aquiles IglesiasCommunication Sciences and Disorders
University of DelawareAquiles @UDEL.edu
35
Slide36Assessing Disabilities in ELs Struggling With Phonological Awareness, Word Recognition, and Decoding Skills
Esther GevaUniversity of Toronto
March 16, 2015
36
Slide37Research Agenda
Clinical, assessment, and instructional issues related to:Over-identification (Cummins, 1991)
Under-identification (Limbos & Geva, 2001)
The challenge:
How do we tease apart L2 status from a possible learning disability?37
Slide38Strategies Guiding My Research on Language and Literacy in ELs
38
Slide39What Do the Data Show About Typical Development of Reading in ELs?
Longitudinal Design
Cross-sequential
(4
cohorts)A large task battery administered annuallyEach cohort tracked from Grade 1 to Grade 6 13 schools across 4 boards of education in a large metropolis mixed catchment area
first
generation
immigrants
communities
average poverty incidence =
23%
income below median for the metropolis
39
Slide40Test Battery
40
Slide41The Sample
Home languages spoken in the
EL group:
Punjabi
, Tamil, Urdu, Cantonese, Portuguese, SpanishGroupsNFemale
Male
Mean Age in Grade 4
EL
427
213
206
116.12
EL1
158
93
64
118.97
41
Slide42Receptive Vocabulary (PPVT): Average Growth Trajectories in EL and EL1
Grade1
Grade 2
Grade 3
Grade 4
Grade 5
Grade 6
37.40
57.68
77.96
98.25
118.53
Growth From Grade 1 to Grade 6
Vocabulary
EL1
ESL
(Farnia & Geva, 2011)
Persistent EL1-EL Difference
EL1 Slope/Rate of Growth
EL Slope (steeper in early years)
42
Slide43Basic Reading Skills: Average Growth Trajectories in EL and EL1
Development of Word Recognition Skills
Word Recognition
Development of
Decoding Skills
Decoding
Source: Geva & Farnia, LARCIC, 2009
43
Slide44Rapid Letter Naming (RAN): Average Growth Trajectories in EL1 and EL
Growth From Grade 1 to Grade 6
Rapid Naming
Source: Geva & Farnia, LARCIC, 2009
44
Slide45Phonological Awareness: Average Growth Trajectories in EL1 and EL
Growth From Grade 1 to Grade 6
Phonological Awareness
No EL-EL1 difference
Source: Geva & Farnia, LARCIC, 2009
45
Slide46Relationship between Phonological Awareness and Word Reading Skills
Grade 1
Grade 6
X = EL1
X = ESL
No EL-EL1 differences
46
Slide47Vocabulary - Strong Predictor of Reading Comprehension in Monolinguals and ELs
X = EL1
X = ESL
Source: Geva & Farnia, LARCIC, 2009
47
Slide48Reading Comprehension*- Average
Growth Trajectories in EL and EL1
Source: Farnia & Geva, J of Res in Reading, 2013
48
*Gates-MacGinitie Reading Test-ESS
Persistent EL-EL1 difference
Slide49Summary of Developmental Trends: EL1 vs. EL
Skills
EL vs. EL1
Rapid naming
= Phonological awareness=Word reading=Vocabulary<Reading comprehension
<
Farnia & Geva, 2013
49
Slide50Ways of examining “transfer” of skills between L1 and L2
Correlation between L1 and L2 skills (universal perspective
)
Effects of specific L1 features on L2 features (
positive/negative)
Geva, E. Written Language & Literacy, 2014
50
Slide51Correlations
Among English (L1) Predictors (Fall SK) and French Reading
a
Year Later
(Grade 1)
(Endler, M., Ph.D., 2007)
51
Slide52Spelling: EL1 Versus EL (Cantonese)
Spelling “th” in the word
thick
–
Gr. 1-Fall (T1) & Spring (T2)
Spelling “th” in the word
thick
–
Gr. 2 Fall (T3) & Spring (T4)
(Wang & Geva, 2003)
52
Slide53Research on ELs Who Might Have Learning Disabilities in the Following Areas:
Word Reading and Spelling (
focus for today
)
Reading FluencyReading Comprehension Written Expression Mathematics Calculations
53
Slide54Auditory Discrimination: Developing Sensitivity to New Phonemes
Pseudoword Auditory Discrimination Task
17 Pseudoword pairs
keathe-keev
thop-zopbish-bissshen-sen
“Did I say the same thing?”
54
Slide55Development of
Auditory Discrimination: At-
Risk
for
Dyslexia vs. No-Risk
Over time L1-EL differences disappear but
reading group differences persist
55
Slide56Profiles of At-Risk and Not-at-Risk EL1 and EL (End of Grade 2)
Geva, Yaghoub-Zadeh, & Schuster,
Annals of Dyslexia
, 2000
56
Slide57Spelling Growth by Language Group
In Grade 1, Punjabi and Portuguese groups had lower scores, however, their growth was steeper over time.
From Grades 4 to 6, home language was
not
a predictor
of initial status or growth.
Geva & Lafrance, 2011
57
Slide58Phonological Processing
Sub-GroupsPhonological Processing (PP) Risk Status:
Below 25%ile on either PA,
RAN,
or both in Grade 1 (PA-risk, RAN-risk, Double Deficit-DD, Typically Developing)
58
Slide59Spelling Growth by PP Risk Group
Risk Groups: Grades 1–3
Risk Groups: Grades 4–6
Initially, all risk groups had lower scores than the No Risk
group.
L2 status not a good predictor but Phonological Processing (PP) risk status
is.
59
Slide60What Predicts Longitudinally
Gr. 6 Spelling?
60
Slide61Conclusions
61
Slide62What Have We Learned?
The overall profiles of ELs and EL1s who have a learning disability are similar (despite differences in language proficiency)
Like monolinguals, ELs with persistent difficulties in word level skills have difficulties with:
processing factors (e.g., phonological awareness, RAN, memory, auditory discrimination)
accurate and fluent word reading and spellingthese difficulties impact reading comprehension and writing62
Slide63Food for Thought
Be mindful of current normsCompare relevant performance to reference group
What IS the relevant reference group?
Consider developmental patterns
Consider transfer from the L1(correlational; positive & negative) Assess evidence based known predictorsExpect same % of EL1 and EL with LD (systemic over- and under-identification are problematic)
63
Slide64Acknowledgments
Funding Support
Social Sciences and Humanities Research Council of Canada
Transfer Grants - Ontario Ministry of Education
Canadian Language and Literacy Research Network (CLLRNet) University of TorontoFormer/Current StudentsMahshid AzimiTodd CunninghamDana Shafman (David)Fataneh FarniaMaureen JeanEmiko KoyamaAdele LafranceKate Ndlovu (Herbert)Gloria RamirezBarbara SchusterZohreh Yaghoub-ZadehLesly Wade-WoolleyMin Wang
64
Slide65References
Farnia, F. & Geva, E., (2013). Growth and predictors of change in English language learners' reading comprehension. Journal of Research in reading
,
36
(4), 389–421. ISSN 0141–0423 DOI:10.1111/jrir.12003, Farnia, F., & Geva, E. (2011). Cognitive correlates of vocabulary growth in English language learners. Applied Psycholinguistics, 32(4), 711–738Geva, E. (2006). Second–language oral proficiency and second–language literacy. In D. August & T. Shanahan (Eds.), Developing literacy in second–language learners: A report of the National literacy Panel on language–Minority Children and Youth (123–140). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
65
Slide66References
Geva, E. (2014). The cross–language transfer journey – A guide to the perplexed. In E. Zaretsky & M. Schwartz (Eds.), Cross–linguistic transfer in reading in multilingual contexts – Recent research trends, (1–15). Boston, MA: University of Massachusetts.
Geva, E., & Farnia, F. (2012). Developmental changes in the nature of language proficiency and reading fluency paint a more complex view of reading comprehension in ELL and EL1. Reading and Writing: an Interdisciplinary Journal, 25(8), 1819–1845.
Geva, E., & Herbert, K. (2012). Assessment and interventions for English language learners with learning disabilities. In B. Wong & D. Butler (Eds.),
Learning about learning disabilities (4th ed.), (271–298). San Diego, CA: Elsevier Science.
66
Slide67References
Geva, E., & Massey–Garrison, A. (2013). A comparison of the language skills of ELLs and monolinguals who are poor decoders, poor comprehenders or normal readers. Journal of Learning Disabilities
,
46
(5), 387–401. Geva, E., & Yaghoub-Zadeh, Z. (2006). Reading efficiency in native English–speaking and English–as–a–second–language children: The role of oral proficiency and underlying cognitive–linguistic processes. Scientific Studies of Reading, 10(1), 31–57.Geva, E., Yaghoub–Zadeh, Z., & Schuster, B. (2000). Understanding individual differences in word recognition skills of ESL children. Annals of Dyslexia, 50(1), 121–154.Jean, M., & Geva, E. (2009). The development of vocabulary in English as a second language children and its role in predicting word recognition ability. Applied Psycholinguistics, 30(1), 153–185.
67
Slide68References
Limbos, M. & Geva, E. (2001). Accuracy of teacher assessments of second-language students at risk for reading disability. Journal of Learning Disabilities
,
34
(2), 136–151.Ndlovu, K. and Geva, E. (2008). Writing abilities in first and second language learners with and without reading disabilities. In J. Kormos & E.H. Kontra (Eds.), Language learners with special needs: An international perspective. Toronto, Canada: Multilingual Matters. Saiegh–Haddad, E. & Geva, E. (2008). Morphological awareness, phonological awareness, and reading in English–Arabic bilingual children. Reading and Writing: An Interdisciplinary Journal, 21(5), 481–504.
68
Slide69References
Wang, M., & Geva, E. (2003). Spelling performance of Chinese children: Lexical and visual–orthographic processes. Applied Psycholinguistics, 24,
1–25.
Wang, M., & Geva, E. (2003). Spelling acquisition of novel English phonemes in Chinese children. Reading and Writing: An Interdisciplinary Journal, 16(4), 325–348.
69
Slide70Esther GevaUniversity of Toronto
egeva@oise.utoronto.ca
70
Slide71A Comparison of Global and Discrete Measures When Assessing ELs’ Writing
Sylvia Linan-ThompsonUniversity of Texas at Austin
March 16, 2015
71
Slide72Rationale
Recently, there has been increased interest in writing.72% of fourth graders are below proficient level on the National Assessment of Educational Progress (2003).
Measures of writing that measure student progress and identify students who are struggling are available but ELs have not been included in much of the research.
72
Slide73Assessment
Curriculum Based Measures: Multiple probes of equivalent difficulty that are administered repeatedly (Deno, 1985; Jenkins, Deno, & Mirkin, 1979)
Progress monitoring: A nondiscriminatory RTI approach to track student progress
(Fuchs & Fuchs, 2006; Speece, Case, & Molloy, 2003)
73
Slide74Research
Words written, words spelled correctly, and correct letter sequence were:Sensitive to growth within year and across grades
Discriminated between student with and without disabilities
(Deno et al., 1982; Marston
& Deno., 1981; Marston, Deno, & Tindal, 1983)Analytic scores added to those measures discriminated between general education students and students with LD, at-risk, and low performance (Tindal & Hasbrouck, 1991).Percent of words spelled correctly Best screening tool (Parker et al., 1991b).
74
Slide75Research
CBMs in reading are not as reliable with ELs as they are for monolingual students.(Linan-Thompson, 2010; Linan-Thompson, Vaughn, Prater, &
Cirino, 2006).
75
Slide76Exploratory Study
To better understand the writing development of ELs in
English and
Spanish, we are using global and discrete measures to identify measures that best discriminate among three groups of second-grade students in dual language classrooms.
Students received reading instruction in Spanish; writing instruction was in both languages.Students had multiple opportunities to write: creative bilingual journals, independent writing, reader response.Teachers used a writers’ workshop model for writing instruction.Project ESTRE2LLA, US Department of Education, Office of Special Education ProgramsGrant # H326M110010
76
Slide77Data Sources
Journal samples Stanford English Language Proficiency
Stanford Spanish Language Proficiency
77
Slide78Variables
Total number of wordsCorrect word sequence
Correct word sequence without spelling
Number of correctly spelled words
78
Slide79Variables
Bilingual strategiesDiscourse level
Sentence/phrase level
Word level
(Adapted from Soltero-González, Escamilla, & Hopewell, 2011).Holistic rating1-5 scale Organization/contentCohesionComplexity
79
Slide80Findings: Group Means
Student
SELP WW
Pre
SELP WW post
SSLP WW
pre
SSLP WW post
Dyslexia
33.5
38.2
48.25
58
Low language proficiency
27.2
59.4
50
56
Average language proficiency
52
48.3
63.3
64.5
80
Slide81Findings: English Journal Samples
Student
WW
WW
WSC
WSC
Dyslexia
30
28
1
9
Low
83
30
23
13
Low
40
37
4
14
Average
46
54
42
45
81
Slide82Findings: English Journal Samples
Student
CWS
CWS w/o spelling
CWS
CWS w/o spelling
Dyslexia
0
24
2
26
Low
2
52
5
23
Low
0
35
6
26
Average
34
38
40
48
82
Slide83Findings: English Journal Samples
Student
Holistic
Holistic
Dyslexia
10
6
Low language proficiency
5
6
Low language proficiency
6
4
Average language proficiency
14
15
83
Slide84Findings: Bilingual Strategies
Student
Syntax
Literal Translation
Dyslexia
Mai tio hies
Low language proficiency
Average language proficiency
…rolacoster of water
…another one of ginger
84
Slide85Findings: Bilingual Strategies
Student
Language approximation
Phonetic Transfer
Dyslexia
Sow (so), abaut
Ple stesin, bak, flep
Pleis, ais cet
Resolors, febret
Low language proficiency
Flawr, gan (going), brouk
Aut sayd (outside), brader, lero, owis, jaga
Average language proficiency
Auncluse (uncles)
85
Slide86Summary and Next Steps
Generally, average performing students write more words than students in other groups.Average proficiency students are better spellers than students in the other two groups as measured by both TWC and CWS.
Average proficiency students have higher holistic scores than students in the other two groups.
Most bilingual strategies are at the word level.
We will continue to code writing to determine whether there are differences between low performing students and students with dyslexia in student growth on any measures.
86
Slide87Conclusion (With Limited Data)
Writing provides evidence of student development in the process of becoming bilingual/biliterate.Student writing may be an alternative to parallel monolingual assessments.
Writing may be easier to use and more sensitive than oral proficiency measures to monitor student language development.
87
Slide88References
Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52
, 219-232.
Fuchs, D., & Fuchs, L. S. (2006). Instruction to responsiveness-to-intervention: What, why and how valid is it?
Reading Research Quarterly, 41, 92-99.Linan-Thompson, S. (2010). Response to instruction, English language learners and disproportionate representation: The role of assessment. Psicothema 2010, 22(4), pp. 970-974.Linan-Thompson, S., Vaughn, S., Prater, K., & Cirino, P. T. (2006). The response to intervention of English language learners at-risk for reading problems. Journal of Learning Disabilities, 39, 390-398.
88
Slide89References
Marston, D., & Deno, S. L. (1981). The reliability of simple, direct, measures of written expression
(Vol. IRLD-RR-50). University of Minnesota, Institute of Research on Learning Disabilities.
Martson, D., Deno, S. L., & Tindal, G. (1983).
A comparison of standardized achievement tests and direct measurement techniques in measuring student progress (Vol. IRLD-RR-126). University of Minnesota, Institute of Research on Learning Disabilities.McMaster, K., & Epsin, C. (2007). Technical features of curriculum-based measurement in writing: A literature review. The Journal of Special Education, 41, 68-84.
89
Slide90References
Parker, R. I., Tindal, G., & Hasbrouck, J. (1991b). Progress monitoring with objective measures of writing performance for students with mild disabilities,
Exceptional Children, 58
, 61-73.
Soltero-Gonzalez, L., Escamilla, K., & Hopewell, S. (2011). Changing teachers’ perceptions about the writing abilities of emerging bilingual students: Toward a holistic bilingual perspective on writing assessment. International Journal of Bilingual Education and Bilingualism, 15, 71-94.Speece, D. L., Case, L. P., & Molloy, D. E. (2003). Responsiveness to general education education instruction as the first gate to learning disabilities identification. Learning Disabilities Research & Practice, 18, 147-156.
90
Slide91Sylvia Linan-ThompsonThe University of Texas at Austin
sylvialt@austin.utexas.edu
91
Slide92Multitiered
Systems of Support for English LearnersAlba A. Ortiz
University of Texas at Austin
March 16, 2015
92
Slide93Students Experience Learning Difficulties for a Variety of Reasons
Learning problems result from deficiencies in the teaching-learning environment.
Learning problems become more serious over time because instruction is not adapted to address identified needs or gaps (interplay of individual and environment).
Learning problems are associated with the presence of a disability (individual).
(Adapted from Adelman, 1992)
93
Slide94Multitiered System of
Support (MTSS)This school improvement framework reflects an interactional view of student success and failure.
It focuses on all aspects of the educational process, and those involved in this process, to ensure student success:
Effective schools
Prevention of school failureData-driven, multitiered systems of instructionCollaboration across programs, services, personnelProfessional developmentContinuous evaluation of the effectiveness of system-level and student-level initiatives and supports
94
Slide95MTSS Essential Components:
Positive School Environments
Well-implemented special language programs
Bilingual education and English as a Second Language
Shared knowledge base about ELs and about effective practices for their educationAppropriate screening, assessment, and progress monitoring systems for the native language (L1) and English (L2), depending on program modelLongitudinal record-keeping systems to track student progress within and across grades
95
Slide96MTSS Essential Components: Response to Intervention (RtI)
An essential component of a Multitiered System of Support, Response to Intervention is a multitier approach to providing high-quality instruction and intervention, matched to student needs (Elliott, 2008).
Progress is closely monitored and changes in instruction are based on data collected from ongoing assessment (National Association of State Directors of Special Education, 2005).
96
Slide97Elements of an Effective RtI Model for ELs
Universal screening in L1 and/or L2, depending on the program modelMeasures validated for ELs
Identification of students at risk of failure
97
Slide98Elements of an Effective RtI Model
for ELsTiered instruction
Core instruction (Tier 1)
Universal design principles
Language of InstructionIn the native language and in English in bilingual education programsIn English as a Second Language programs, English instruction with scaffolding across skill/content areas and settingsCulturally responsive principles in delivery of instructionContinuous progress monitoring
98
Slide99Elements of an Effective RtI Model
for ELsSupplemental instruction
(Tier 2 and Tier 3)
Increasingly intensive interventions for students experiencing learning difficulties
Research-based interventions validated for ELsIntervention consistent with the language of core instructionCulturally responsive principles in delivery of instructionProvided by educators with expertise specific to ELsContinuous progress monitoring in L1 and L2
99
Slide100Elements of an Effective RtI Model
for ELsData-based decision making
I
nstructional planning for all students
Grade, skill/content area, and school-level trends and issuesEffectiveness of curriculum and instruction; fidelity of implementationIdentification of students who are not meeting expectationsIdentification of students who may benefit from special education referral
100
Slide101Early Intervention for ELs
With Reading Difficulties (Tier 2)
Supplement
, do not replace, core
instructionAre based on screening, assessment, and progress monitoring data in L1 and/or L2Are differentiated to address identified language and literacy needsIncorporate strategies to build oral language skills in the native language and/or in English (with emphasis on vocabulary development).Provide systematic, explicit literacy instruction in identified areas of need, consistent with the language of core instruction.
(
Cavazos & Ortiz,
2014)
101
Slide102Problem-solving Teams
If specialized interventions do not resolve learning problems, the student’s case should be presented to a campus-based problem-solving team (PST). PSTs for ELs must include members with expertise specific to this population.
Team members identify factors that contribute to learning difficulties (system- and/or student-level).
102
Slide103Indicators That Support Referral
Multiple data sources corroborate student difficultiesParents concur with educators’ concerns
Significant life events have been ruled out as causal factors (e.g., medical history, accidents, family issues)
Native language skills are atypical of peers with similar backgrounds
ESL skills are atypical of peers with similar backgroundsCultural characteristics are atypical of peersNo evidence of interrupted education; no excessive absencesStudent has been enrolled in current placement long enough to benefit from interventionGeneral education interventions have not resulted in adequate progress
103
Slide104Linguistically and Culturally Responsive Special Education Processes
Referral committees include parents and other members with EL expertise
Full and Individual Evaluations using instruments and procedures that are valid and reliable for ELs
Multidisciplinary teams with EL expertise
Determine eligibilityDevelop IEPs that simultaneously address disability and language-related needsAssess results of special education intervention
104
Slide105Issues With EL/LD Identification
ELs with limited oral language proficiency in both the native language and in English have the highest special education placement rates ( Artiles, Rueda, Higareda, & Salazar, 2005).
ELs with specific learning disabilities (Ortiz, et al., 2011):
Were typically referred
in 2nd or 3rd grade; many had already been retained or socially promoted.Had limited oral language proficiency in the native language and in English at school entry and still had limited skills in both languages at the time of referral. Yet, oral language development was not targeted in early intervention efforts.
105
Slide106Issues With EL/LD Identification
Most states allow (or require) identification of specific learning disabilities based on significant discrepancies between intelligence and achievement.
The instruments used to establish IQ-achievement discrepancies are of questionable validity for ELs
.
Assessment personnel lack training in assessment of ELs and in interpretation of assessment results.
106
Slide107Issues With EL/LD Identification
Neither RtI nor special education processes provide appropriate data to determine the presence of a disability. Data are insufficient provide assurances that problems are not the result of:
Limited English language proficiency
Cultural differences
Economic disadvantageLack of appropriate instruction in reading and math107
Slide108Identification of LD
As many as 75% of ELs identified as having learning disabilities may be misclassified (Ortiz et al., 2011):Some have learning problems that can be explained by factors other than the presence of a disability (e.g., lack of access to appropriate instruction, lack of timely intervention, inappropriate assessment practices).
Some have disabilities, but not the one they have been assigned.
In examining special education representation patterns (overrepresentation, underrepresentation, and proportionate representation), it is important to ask:
Are we serving the right students?
108
Slide109Indicators of LD for ELs
In addition to the factors that supported referral:Full and individual evaluation corroborates reasons for referral.
Results of RtI data corroborate those of formal assessments.
The student’s performance is significantly different from that of EL peers.
The student exhibits behaviors typically associated with the suspected disability.Parents/family concur that problems manifest in home and community contexts (i.e., they exist 24 hours a day, not just in school).There are no competing hypotheses to explain student difficulties.
109
Slide110Research Infrastructure
Preparation of researchers with expertise in the interaction of native language and English language proficiency/ development and disability
Routine inclusion of ELs in research on students with disabilities or specific explanation for their inclusion
Application of relevant theories and frameworks in studies involving ELs
110
Slide111Research Needs
Documentation of special education representation patterns and factors contributing to disproportionate representation by language group
Criteria
to determine eligibility of ELs with disabilities for bilingual education and/or ESL/ELD programs and criteria for exit from these
programsDetermining who is a “true” peerProgress monitoring procedures for oral language development
111
Slide112Research Needs
Effectiveness of RtI for ELs, including effect on special education and placement ratesBest practices in native language and English assessments to identify disabilities
Guidance for distinguishing differences from disabilities
Effective instructional practices for core instruction, supplemental intervention, and special education instruction
By age, grade, language proficiency, academic achievement, disability/exceptionality, category, severity level
112
Slide113RtI Model Demonstration Projects
Model Demonstration Project for English Language Learners With or At-Risk of Having a Disability (84.326M)
3 Funded Projects
The University of Texas at Austin (2)
The University of Colorado at Boulder (1)Sponsor: U.S. Department of Education, Office of Special Education ProgramsFunding Period: 2012-2015
113
Slide114References
Adelman, H. S. (1992). LD: The next 25 years. Journal of Learning Disabilities
, 25, 17-22.
Artiles, A. J., Rueda, R., Salazar, J. J., & Higareda, I. (2005). Within-group diversity in minority disproportionate representation: English Language Learners in urban school districts.
Exceptional Children, 71(3), 283-300.Elliot, J. (2008, September). Response to Intervention: What and Why? School Administrator, 10-18.Cavazos, L. & Ortiz, A. A. (2014, February). Response to Intervention for English Language Learners with Reading Difficulties. Presented at the meeting of the National Association for Bilingual Education, San Diego, California.García, S. B., & Ortiz, A. A. (2008). A framework for culturally and linguistically responsive design of Response-to-Intervention models. Multiple Voices for Ethnically Diverse Exceptional Learners, 11(1), 24-41.
114
Slide115References
Linan-Thompson, S., & Ortiz, A. A. (2009). Response to intervention and English Language Learners: Instructional and assessment considerations. Seminars in Speech and Language
, 30(2), 105-120.
National Association of State Directors of Special Education. (2005).
Response to Intervention: Policy considerations and implementation. Alexandria, VA: Author.Ortiz, A. A., Robertson, P. M., Wilkinson, C. Y., Liu, Y., McGhee, B. D., & Kushner, M. I. (2011). The role of bilingual education teachers in preventing inappropriate referrals of ELLs to special education: Implications for Response to Intervention. Bilingual Research Journal, 34(3), 316-333.
115
Slide116Alba A. Ortiz, Professor EmeritusDepartment of Special Education
The University of Texas at Austinalba.ortiz@austin.utexas.edu
116
Slide117Moderated Discussion
117
Slide118Questions
118
Slide119Break
11:05 – 11:15 am
119
Slide120Assessing the English Language Proficiency of ELs With Disabilities:
Implications From Research and PracticeMartha L. Thurlow
National Center on Educational Outcomes (NCEO)
March 16, 2015
120
Slide121Relevant Evidence
Analyses of participation of ELs with disabilities in states’ assessments of English language proficiency
Policies
Public Reporting (including participation and performance)
Analyses of accessibility approaches and accommodations in English language proficiency assessmentsFocus group study on perspectives from the field on challenges and successes in including ELs with disabilities in ELP assessments
121
Slide122Framework
122
Slide123State Assessment Policies: Participation
Criteria for ELs With
Disabilities
49 of 50 states provide criteria for the participation of ELs with disabilities in their ELP assessments
123
Slide124Reporting on ELs With Disabilities ELP Participation and Performance
ELs With Disabilities Data in Public Reports for
2012-13 ELP Assessments
California
ColoradoLouisiana Minnesota New York
Texas
None of these states reported a participation
rate,
although 2 reported percent not tested
All of these states reported
percent proficient
(and all but 1 reported by performance level)
124
Slide125State Policies: Reporting
States rarely address what happens to the students’ scores when the students are not able to participate in all domains of the ELP assessment.
When states do indicate what happens to scores, it is generally to indicate either that a student’s non-participation in some domains did not count against school participation rates or that, if an alternative means of assessing the student was used, the score was considered invalid.
125
Slide126Performance Reported Publicly
6 states reported data publicly, but 1 only reported on its Alternate Assessment of ELP
126
GRADE
CA
ELSWD
CA
ELs
LA
ELSWD
LA
ELs
MN
ELSWD
MN
ELs
NY
ELSWD
NY
ELs
TX
ELSWD
TX
ELs
Grade 4
10%
40%
14%
35%
8%
31%
10%
21%
36%
78%
Grade 8
19%
56%
39% + **
48%
1%
8%
11%
16%
69%
86%
Grade 10
18%
48%
13% + **
41%
16%
30%
15%
16%
65%
82%
Slide127Implications
Better national and state data are needed on ELs with disabilities, including information on their disabilities, language background, assessment participation, and proficiency.
No student should be denied a score of language proficiency because of his or her disability.
127
Slide128State Policies: Accommodations
Decision-making criteria differ across states, with the most frequent criteria (cited by more than ½ of states) being:
IEP team decision – 46 states
Maintains validity – 41 states
Used in class – 39 statesMeets individual student needs – 39 statesAppropriate for domain – 28 statesWhether not recommended because of severity of disability – 27 statesSome criteria, mentioned by fewer than ½ of states, refer to specific disabilities:Student is proficient in braille or lip reading – 22 states
128
Slide129State Policies: Accommodations
Reading
Writing
Listening
SpeakingLarge Print (46)Proctor/Scribe (42)Braille (40)Magnification (36)Amplification (36)Directions-Repeat, Re-read, Clarify (36)Sign Interpret Directions (36)
Student Read Aloud (30)
Large
Print (46)
Braille (40)
Proctor/Scribe (38)
Magnification (38)
Computer/Machine (38)
Amplification (35)
Directions-Repeat, Re-read, Clarify (36)
Student Read Aloud (29)
Large
Print (44)
Braille (38)
Magnification (38)
Sign Interpret Directions (38)
Directions – Repeat, Re-read, Clarify (36)
Amplification (35)
Large
Print (42)
Braille (38)
Magnification (38)
Sign Interpret Directions (37)
Directions-Repeat, Re-read, Clarify (36)
Amplification (35)
129
Slide130More on Approaches for Deaf/HH
Selective participation for ELP assessment allowed for students who are deaf/hard of hearing in 26 statesLeast controversial accommodations – sign interpret directions and amplification equipment
Sign interpret questions
prohibited across domains in most states (30-32 states)
Sign response prohibited across domains in most states (26-27 states)Few state policies addressed visual cues (allowed in 4; prohibited in 1)
130
Slide131More on Approaches for Blind/VI
Selective participation for ELP assessment allowed for students who are blind/visually impaired in 24 states
Least controversial accommodations – braille, large print, magnification equipment
Few state policies addressed
read aloud directions; more likely to allow for writing than for listening and speakingRead aloud questions allowed in 30 states for writing domain; rarely allowed for other domainsBrailler prohibited for writing domain in 25 states
131
Slide132Implications
Accessibility and accommodations policies should be based on determinations about the construct being tested. More research and discussion may be needed on:
What does listening mean for a student who is deaf/hard of hearing?
What does reading mean for a student who is blind/visually impaired?
What does speaking mean for a student who has been deaf from birth or who has a speech impediment?What does writing mean for a student who has a significant motor disability?Based on decisions about constructs, plans need to be made for obtaining a total score for all students
132
Slide133Perspectives From the Field:
Challenges and SuccessesParticipants stated that the IEP process served primarily to make decisions about accommodations on content assessments, but less so for state ELP assessments and accommodations.
Participants described needs specific to ELs with disabilities for support and guidance from school and state education leaders on assessment and accommodations. The needs were for additional qualified staff and training, clear and consistent written assessment policies, and appropriate uses of state accountability test scores.
133
Slide134Implications
More training is needed on appropriate decision making for participation and accommodations for ELs with disabilities.
The IEP team must include professionals who know English language development.
The IEP team should make decisions about both language development and content development.
134
Slide135Information Resources
Christensen, L. L., Albus, D. A., Liu, K
. K
., Thurlow, M., & Kincaid, A. (2013).
Accommodations for students with disabilities on state English language proficiency assessments: A review of 2011 state policies. Minneapolis, MN: U of MN, NCEO. Christensen, L. L., Albus, D. A., Kincaid, A., Liu, K. K., & Thurlow, M. L. (2014). Including students who are deaf or hard-of-hearing in English language proficiency assessments: A review of state policies. Minneapolis, MN: U of MN, NCEOChristensen, L. L., Albus, D. A., Kincaid, A., Christian, E., Liu, K. K., & Thurlow, M. L. (2014). Including students who are are blind or visually impaired in English language proficiency assessments: A review of state policies. Minneapolis, MN: U of MN, NCEO
135
Slide136Information Resources
Guzman-Orth, D., Laitusis, C., Thurlow, M., & Christensen, L. (2014).
Conceptualizing accessibility for English learners and English learners with disabilities taking English proficiency assessments: What do we know and where do we go from here?
Princeton, NJ: ETS.
Liu, K., Goldstone, L., Thurlow, M., Ward, J., Hatten, J., & Christensen, L. Voices from the field: Making state assessment decisions for English language learners with disabilities. Minneapolis, MN: U of MN, NCEONCEO. (May 2014). Participation of ELLs with disabilities in ELP assessments (NCEO Brief #8). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
136
Slide137Training and Other Resources
Improving the Validity of Assessment Results for English Language Learners with Disabilities (IVARED) – Training Module:
http://www.ivared.info/training.html
.
Christensen, L., Shyyan, V., Rogers, C., & Kincaid, A. (2014). Audio support guidelines for accessible assessments: Insights from cognitive labs. Minneapolis, MN: U of MN, NCEO, GAAP Project.Shyyan, V., Christensen, L., Rogers, C., & Kincaid, A. (2014). Sign support guidelines for accessible assessments: Insights from cognitive labs. Minneapolis, MN: U of MN, NCEO, GAAP Project.
137
Slide138Training and Other Resources
Shyyan, V., Christensen, L., Touchette, B., Lightborne, L., Gholson, M., & Burton, K. (2013).
Accommodations manual: How to select, administer, and evaluate use of accommodations for instruction and assessment of English language learners with disabilities.
Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
138
Slide139Martha ThurlowNational Center on Educational Outcomes (NCEO)
THURL001@umn.edu
139
Slide140ELP Assessment Accommodations for ELs With Disabilities:
Relevance, Effectiveness, Feasibility, and ValidityJamal Abedi
University of California, Davis
March 16, 2015
140
Slide141Why Is It Important to Pay Special Attention to Assessment of ELP for ELs With Disabilities?
They are faced with the highest level of challenges in their academic career (the most at risk)They are often at the lowest level of ELP
In content-area assessments, they are about 1 to 1 ½ standard deviations below mainstream students
They may have the content knowledge but not the language capabilities to express it
The rate of misclassification of these students is extremely high141
Slide142What Types of Accommodations Are Used in the Assessment of ELs With Disabilities?
ELs with disabilities may receive two different types of accommodations:
1.
Accommodations due to their EL status2. Accommodations due to their disabilities (based on their IEP or 504 plans)States should provide evidence on the effectiveness and validity of these accommodations
142
Slide143Can accommodations that are used in
content-area assessments for ELs be used in ELP assessments for ELs with disabilities?
The most useful accommodations
for ELs in content-area assessments where the focal construct is not language are language-based accommodations
Examples: English and bilingual dictionaryEnglish and bilingual glossaryNative language assessmentsHowever, the focal construct in ELP assessments is language; therefore, language-based accommodations may not be valid for ELP assessments
143
Slide144Issues/Questions To Be Addressed
How can we make sure that accommodations and accessibility features used in ELP assessments are effective in making ELP assessments more accessible for ELs with disabilities (effectiveness)?
How can we make sure that accommodations used for ELs with disabilities do not alter the focal construct?
How can we judge the feasibility of accommodations used for ELs with disabilities?
144
Slide145How Can Accommodations for ELs With Disabilities Be Examined for Validity*?
Only through experimentally-controlled research where:
ELs
with disabilities and
non-ELs without disabilities are randomly assigned to the accommodated conditionsBoth ELs with disabilities and non-EL students are observed under accommodated and non-accommodated assessmentsAn accommodation is valid if it does not impact performance of non-ELs without disabilities. If it does impact the performance of non-ELs without disabilities, then the accommodation provides an unfair advantage to the recipients* Not altering the focal construct
145
Slide146How Can Accommodations for ELs With Disabilities Be Examined for Effectiveness?
Only through experimentally-controlled research where:
ELs
with disabilities are randomly assigned to either experimental or control groups
The experimental group receives an accommodationThe control group is tested under the standard condition with no accommodation providedA significant improvement in the performance of the experimental group indicates effectiveness
146
Slide147Are Accommodated and Non-Accommodated Assessments for ELs With Disabilities Comparable?
The way to establish comparability between accommodated and non-accommodated assessments for ELs with disabilities is to make sure that:
The accommodations used are valid (i.e., do not alter the focal construct (ELP))
The accommodations used are effective in making assessments more accessible for these students by controlling the construct-irrelevant sources.
147
Slide148Sample Accommodations Used for ELs With Disabilities
Accommodation
Examples
Presentation
Repeat directions, read aloud, large print, braille EquipmentCalculator, amplification equipment, manipulativesResponseMark answers in book, scribe records responseSettingStudy carrel, student's home, separate roomTiming/Scheduling
Extended time, frequent breaks
148
Slide149Issues/Questions To Be Addressed
Can accommodations that are used in content-area assessments for ELs be used in ELP assessments?
Can we assume accommodations that help reducing sources of construct-irrelevant variance in content-area assessments do the same in ELP assessments (validity)?
How can we make sure that accommodations used in ELP assessments do not alter the focal construct (validity)?
149
Slide150Sample Accommodations Used for ELs With Disabilities
Accommodation
Examples
Presentation
Repeat directions, read aloud, large print, braille EquipmentCalculator, amplification equipment, manipulativesResponseMark answers in book, scribe records responseSettingStudy carrel, student's home, separate roomTiming/Scheduling
Extended time, frequent breaks
However, not enough evidence on the effectiveness or validity of these accommodations for ELs with disabilities
Source:
NCEO
(
http://www.cehd.umn.edu/NCEO/TOPICAREAS/
Accommodations/Accomtopic.htm
150
Slide151How Are We Doing in Practice Nationally?
Are states and districts across the nation cognizant of these important criteria for selecting and using accommodations that are effective and valid for ELs with disabilities?
Are there any objective national criteria to help states to select appropriate accommodations for EL students?
Or, is the assignment of accommodations to these students based on temporary and subjective decisions?
151
Slide152An Evidence-based System
for Determining Appropriate Accommodations for ELs With Disabilities (Abedi & Ewers, 2013)
Use
:
Supported by existing research as being effective in making assessments more accessible and valid (i.e., does not alter the focal construct) for ELs with disabilities or supports the concept of differential boostUse/Low Evidence: No clear evidence that the validity assumption is violated; however, additional support would strengthen their acceptance in the fieldDo Not Use: There is enough consistent evidence suggesting an accommodation is not effective and alters the focal construct.
152
Slide153An Evidence-based System for Determining Appropriate Accommodations for ELs With Disabilities
Unsure
/
Low Evidence Needed
: Existing research-based evidence is supportive of the accommodation but not sufficient to make a judgment about its effectiveness and validityUnsure/Moderate Evidence: Existing research-based evidence is not quite sufficient to make a judgment about effectiveness and validity; some additional research-based evidence is neededUnsure/High Evidence: Existing research-based evidence neither supports nor rejects the effectiveness and validity of the accommodation, substantial research-based evidence is needed
153
Slide154Overall Decision on
Accommodation Use
If valid/any level of effectiveness, then
use
If valid or low evidence needed/any level of effectiveness, then use with minor riskIf validity is unsure with low evidence needed/any level of effectiveness, then use with minor riskIf validity is unsure with moderate evidence needed/any level of effectiveness, then use with moderate riskIf validity is unsure with high evidence needed/any level of effectiveness, then use with high risk
154
Slide155An Example of the Decision Process: Overall Decision – Use
EL Accommodation
Extra time within the testing day
Research Findings
This study indicated that extra time is both effective and valid for students in Grade 4 (Abedi et al., 2003b).
Both EL and non-EL students in Grade 8 are helped by this accommodation on a mathematics assessment of 35 released NAEP items (Abedi, Hofstetter, Baker, & Lord, 2001a).
Highly rated by a team of experts as helpful for the lowest English language proficient students. (Acosta et al., 2008).
Recommendation/
Validity
Use
Recommendation/
Effectiveness
Unsure/
Moderate Evidence
Overall Decision
Use -
Access
155
Slide156Conclusions and Recommendations
Accommodations
:
Must be relevant in addressing assessment issues for ELs with disabilities
Must be effective in reducing the performance gap between ELs with disabilities and those who are not ELs with disabilitiesShould not alter the construct being measured; under such condition, the accommodated results can be aggregated with the assessments under standard conditionsMust be feasible in national and state assessments
156
Slide157References
Abedi, J. (2013). Testing of ELL Students
(Chapter 101).
In K. F. Geisinger
, APA handbook of testing and assessment in psychology. Washington, DC: American Psychological Association.Abedi, J. (2012). Validity issues in designing accommodations. In G. Fulcher & F. Davidson, The Routledge handbook of language testing. London, UK: Routledge, Taylor & Francis.Abedi, J. (2007). English language learners with disabilities. In C. Cahlan-Laitusis & L. Cook (Eds.), Accommodating student with disabilities on state assessments: What works? Arlington, VA: Council for Exceptional Children.
Abedi, J. (2014). The
use
of
computer
t
echnology
in
designing
a
ppropriate
t
est
a
ccommodations
for English
language
l
earners
.
Applied Measurement in Education
.
Abedi, J., & Ewers, N. (2013).
Accommodations for English
language
l
earners
and
students
with
disabilities
: A
research-based
d
ecision
a
lgorithm
. Smarter Balanced Assessment
Consortium. Retrieved from
http://
www.smarterbalanced.org/wordpress/wp-content/uploads/2012/08/Accomodations-for-under-represented-students.pdf
157
Slide158Jamal AbediUniversity of California, Davis
jabedi@ucdavis.edu(530) 754-9150
158
Slide159Accessibility and Accommodations on ELPA21
Phoebe WinterIndependent Consultant
March 16, 2015
159
Slide160ELPA21
Developed by a multi-state consortium, USED EAGBased on the
English Language Proficiency (ELP) Standards
(CCSSO, 2013)
Correspondence to content area standards Computer-based assessmentUse of technology to improve measurement
160
Slide161Designed for Accessibility
Coordination among professionals with expertise inEnglish language acquisitionStudents with disabilitiesMeasurement
Evidence centered design
Test design
Item developmentResearchTest delivery161
Slide162Developing Accessible Items
Minimizing language-related barriers due to non-targeted domainsMinimizing barriers to measuring all students’ proficiency/performanceUniversal design
Designing with accessibility features and accommodation in mind
162
Slide163Building in Accessibility – Strategies
Use of pictures and graphicsMultiple modes of presentationNon-verbal response modesAPIP
compatibility
Alt text
Reviews during and after item development163
Slide164Sample Compare Pictures Set (Grade 4-5 Speaking)
Written directions and narrator audio: Look carefully at the two pictures. Describe what is the same and what is different. Use as many details as you can. Include at least three
things that are the same or different.
164
Slide165Sample Follow Instructions Set (
G1 Listening) Written directions and narrator audio: Look at this classroom. Listen to the teacher. Follow the teacher’s directions
.
(
Teacher audio): Put the ruler on the bookshelf.(Teacher audio): Choose the book about animals. Put it on the table by the window.
165
Slide166Sample Listen and Match
Word (G1 Listening)
Written directions and narrator audio: Listen to the word. Choose the picture that matches the word
.
(Narrator audio): balloon166
Slide167Conceptual Model
167
Slide168Blind and Visually Impaired Students
Assess the same standards as the general ELPA21Provide scores that are comparable in terms of acquisition of academic English
As parallel in structure as possible to the general
ELPA21
Minimize dependence on language skills other than those being assessedBe engaging and motivatingUse the computer as much as possible, while being amenable to paper-based administration168
Slide169Items
Item “twins” developedGoal is to have 1.5 X items needed for each formItem review for accessibilityGaps filled by twins at the task type levelTwins designed to measure same standards
Braille for reading items above K
Braille and audio for other domains
Manipulatives (realia) for response options and as stimuli Cognitive labs169
Slide170Sample Item Twins
Compare pictures Follow instructionsListen and matchRead and match
Word builder
Storyboard
170
Slide171Sample Compare Pictures Set General Assessment (Grade 4-5 Speaking
)Written directions and narrator audio: Look carefully at the two pictures. Describe what is the same and what is different. Use as many details as you can. Include at least
three
things that are the same or different.
171
Slide172Sample Compare Pictures Twin (Grade 4-5 Speaking)
Narrator audio and braille stimulusI’d like to hear about how playing at school and playing at home are the same and different.
Describe
some things that are the same about playing at school and playing at the home. Please give as many details as you can.
Now describe some things that are different about playing at school and playing at the home. Please give as many details as you can.172
Slide173Sample Follow Instructions Set General Assessment (G1 Listening)
Written directions and narrator audio: Look at this classroom. Listen to the teacher. Follow the teacher’s directions.
(Teacher): Put the ruler on the bookshelf.
(
Teacher audio): Choose the book about animals. Put it on the table by the window.173
Slide174Sample Listen and Match Word General Assessment (G1 Listening)
Written directions and narrator audio: Listen to the word. Choose the picture that matches the word.
(
Narrator audio):
balloon174
Slide175Sample Follow Instructions Twin (Grade 4-5 Listening)
Script: Listen to the instructions. Follow my directions.
Script: Put the plant in the box
.
Manipulatives: A small plant and a book; box, paper bag, backpack, vase Sample Listen and Match Twin (Grade 1 Listening)Script: Listen to the word. Choose the object that matches the word.Script: BalloonManipulatives: Ball, basket, balloon 175
Slide176Sample Read and Match Item General Assessment (G1 Reading)
Written directions [and narrator at lower grades]: Look at the picture. Choose the word that matches the picture.
dot
d
ogbug176
Slide177Sample Read and Match Twin (Grade 4-5 Reading)
Script: Touch the object. Read the words. Choose the word that matches the object.Braille book:
pail
paper
panparentManipulative: paper 177
Slide178Sample World Builder Item General Assessment (G1 Writing)
Written directions and narrator audio: Move the letter to complete the word.
(
Narrator audio):
bearb p w_ear178
Slide179Sample World Builder Twin(G1
Writing)Script: Listen to the word. Complete the word.
Script
: bear
Materials: Braille cards or tiles_earb p w 179
Slide180Sample Storyboard General Assessment (Grades 6-8 Writing)
Written directions and narrator audio: The four pictures below show a story about something that happened. What story do the pictures tell? Look at the pictures and prepare to write a paragraph. You may use the words in the Word Bank to help you. Type your story in the text box and click submit when you are finished.
Be sure to check your work. Make sure you…
tell a complete story from beginning to end
organize ideas in a logical wayuse the right style for the task and the audience
Word Bank
empty
litter
garden
clean
plants
180
Slide181Sample Storyboard Twin (
G4-5 Writing)Narrator audio and braille stimulus: Write a story about a time when you did something fun with your friends. You may use the words in the Word Bank to help you.
Be sure to check your work. Make sure you…
tell
a complete story from beginning to endorganize ideas in a logical wayuse clear language and complete sentencesWord Bank:friendsfun excitingplay 181
Slide182Considerations for Accessibility
Understanding clearly the constructs being assessedIncorporating an understanding of the students in test design and item developmentProviding general access to a suite of tools Carefully selecting/designing and considering the impact of other accessibility features – designated features and accommodations
Incorporating empirical research into development
182
Slide183Resources
English Language Proficiency Assessment for the 21st Century, ELPA21, is a consortium of states committed to supporting educators, member states, and members of the public as they adopt and implement the English language proficiency standards and and college- and career-ready standards.
The consortium
is developing an assessment system based on the ELP Standards
For more information see: http://www.elpa21.org/183
Slide184Phoebe Winter
phoebe.winter@outlook.comFor more information about ELPA21 in general: http://www.elpa21.orgFor information specific to accessibility and accommodations:
http://www.elpa21.org/assessment-system/features-benefits
184
Slide185WIDA Consortium, ACCESS 2.0Accommodations for ELs with disabilities
H Gary Cook, Ph.D. Associate Research Scientist
Research
185
Slide186Overview
The WIDA Consortium & ASSETS GrantACCESS 2.0 Accessibility Features and AccommodationsCreating Alternate Composites Scores for ELs with Disabilities
Research
186
Slide187ASSETS Grant Members (WIDA 2.0)
Research
187
Slide188ASSETS Grant
Next Generation ELD Standards & Assessments to support Carrier & College Ready Language Expectations
Research
188
Slide189ACCESS 2.0 Features
ACCESS 2.0 will be provided both in a paper & pencil & online formatSpeaking test computer delivered and scored remotely (online)Writing above grade 3 is provided online
Research
189
Slide190Categorization of Accommodations is Different: ACCESS for ELLs & ACCESS 2.0
What’s Different
Reframed the 40+ Access for ELLs accommodations into 3 categories:
More streamlined approach to accommodation
Added additional supports within online test
Expanded accessibility for
all ELLs
Research
190
Slide191Introduction of New “Middle” Category: Accessibility Features
Include tools and supports that are available to all
ELLs taking the tests, based on need or preference.
M
ay either be embedded in the computer-based test or provided to ELLs by test administrators on online or paper-based tests.
Research
191
Slide192UDL Principles Applied Throughout Test Items:
Balance Accessibility and UsabilityIncreased multimodality
Add supporting prompts with appropriate animations and graphics
Uncomplicated and predictable as possible, low extraneous processing demands
The test will look uncluttered and function intuitively for the test taker.
Sample Reading Item Online Layout
Research
192
Slide193Creating Alternate
Composite ScoresProblem
Some EL/SWD do not get composite scores (e.g., deaf or blind students)
Lack of composite scores effects AMAOs
SolutionsCreate conjunctive expectations based on administered testsCreate alternate composite scores
Research
193
Slide194A Proficient Performance Criterion
U.S. Dept. of Education
(2012). Prepared by Cook, Linquanti, Chinen, Jung.
Conversations
about proficiency
can start here.
Research
194
Slide195Conjunctive Expectations
Research
195
Slide196Alternate Composites
Research
196
Slide197Creating Alternate
Composite ScoresIt is possible to create composite score analogs which can be used to exit ELs who are SWD who cannot participate on all domain tests.
These methods show a predictive relationships to content assessments.
Research
197
Slide198Gary Cook
Wisconsin Center for Education ResearchUniversity of Wisconsinhcook@wisc.edu
198
Slide199Moderated Discussion
199
Slide200Questions
200
Slide201Lunch
1 – 2 pm
201
Slide202Alternate Assessments
of ELP for ELLs With Significant Cognitive Disabilities:Considerations From the Field
Martha Thurlow
National Center on Educational Outcomes (NCEO)
March 16, 2015202
Slide203Overview
What Have We Learned From Alternate Assessment Consortia?Who Are ELs With Significant Cognitive Disabilities?
What Else Do We Know From the Field?
What Might an Alternate Assessment of ELP Look Like?
203
Slide204Delphi Study Results
“Schools must address the language development needs of ELs with significant cognitive disabilities. This is a fundamental civil rights issues….they must have high-quality information about these students’ language skills and needs.”
204
Slide205DLM and NCSC Findings
Most students in the AA-AAS use oral speech, read sight words, and do math with a calculator, and can learn more than “rote academic skills.” Still there is a small group who do not.
Communication
systems are
essential for access to the general curriculum, including “language” development.It is important to develop a description of the student population and a theory of learning for these students – these should drive the nature of the assessment. 205
Slide206DLM and NCSC Findings
Many students with significant cognitive disabilities have not had access to the curriculum or to language development efforts.
It is important to take a three-pronged approach for these students – curriculum, instruction, and assessment.
206
Slide207Misperceptions About Students
Common Misperceptions
1
about who the students are who participate in the AA-AAS include:They function more like infants or toddlers than their actual age.They have life-threatening medical conditions or are not able to communicate.They can learn only rote academic
skills.
1
From Quenemoen, Kearns, Quenemoen, Flowers, & Kleinert (2010),
NCEO Synthesis Report 73.
207
Slide208Primary Disability Categories of Students in the AA-AAS
From: National Alternate Assessment Center (NAAC)
208
Slide209Research on Characteristics of Students With Significant Cognitive Disabilities
Most students (about 72%) in AA-AAS had expressive communication (used verbal or written words, signs, braille, or language-based augmentative/alternative communication system) –
symbolic language users
About
18% of AA-AAS students were emerging symbolic language usersAnother 10% of AA-AAS students were pre-symbolic language users1 From Kearns, Towles-Reeves, Kleinert, Kleinert, & Thomas. (2011). The Journal of Special Education.209
Slide210Meet Shelley
Shelley also has multiple disabilities and is using a switch to indicate that she wants “to go more.” It took about 8 trials in one session to get this consistent response.
210
Slide211Meet Jordan
Jordan, who has autism, participates in the AA-AAS. He uses symbolic language and reads sign words. In this clip, he is learning about idiomatic expressions.
211
Slide212Meet Bruce
Bruce is a student who was identified as having significant cognitive disabilities with no oral communication. After a communication system was identified for him, educators no longer believed he should be taking an AA-AAS.
212
Slide213Meet Axel and Thasya
These two students are highlighted on the NCSC website. They are both ELLs with significant cognitive disabilities.
The videos show how complex the identification of these students is and how determining the communication that works for each student is critical.
Find the videos under Multimedia at
http://www.ncscpartners.org213
Slide214Characteristics of ELLs With Significant Cognitive Disabilities
Category of Disability
Category of Disability
C
ountPercentMental retardation355959.2%
Autism
1038
17.3%
Multiple disabilities
599
10.0%
Other
health impairment
154
2.3
%
Speech/language impairment
91
1.5%
Hearing impairment
91
1.5%
Orthopedic
62
1.0%
Emotional disability
45
0.7%
Traumatic brain injury
36
0.6%
Deaf-Blind
26
0.4%
Visual impairment
16
0.3%
Other
246
4.1%
No data
45
0.7%
214
Slide215Characteristics of ELLs With Significant Cognitive Disabilities
Expressive Communication
Expressive
Communication
CountPercentUses symbolic language3675
61.2%
Uses intentional communication
1388
23.1%
Communicates primarily through cries, etc.
850
14.1%
No data
95
1.6%
215
Slide216Participation of ELLs With Significant Cognitive Disabilities in ELP Assessments
216
Slide217Instruction of ELLs With Significant Cognitive Disabilities
Language of instruction – almost exclusively English; special education teachers for the most part do not have backgrounds in language development; some have paraprofessionals who know other languages to provide needed scaffolding
Successful approaches – picture and photo cues, realia, identify objects and pictures in both English and student’s home language
More curricula and instructional strategies are needed
217
Slide218What Might an Alternate ELP Look Like? Some Hints from AA-AAS History
Early AA-AAS:Primarily portfolios, with teachers collecting evidence
Current AA-AAS:
Technology-based assessments, with allowances for teacher administration
Item-based assessmentsAssessments based on grade-level achievement standards218
Slide219Intercultural Considerations
Students and their families are characterized by diverse communication styles
Non-verbal communication misunderstandings are more likely to occur in intercultural settings
Disability is perceived differently across cultures
Students’ cultures of origin and prior intercultural experiences have implications for curriculum, instruction, and assessment decisionsEducators need intercultural competence skills219
Slide220Concluding Thoughts
An alternate assessment of ELP needs to be: Based
on
strong college and career ready standards of ELP
Carefully planned in terms of how it addresses the assessment of reading, writing, speaking, and listening domains, so that those with disabilities in one or more areas can still earn a score and demonstrate proficiencyDesigned so that ELs with significant cognitive disabilities can demonstrate proficiency in English, with solid exit criteria based on alternate performance criteria.
220
Slide221Resources
Kearns, J., Towles-Reeves, E., Kleinert, H., Kleinert, J., & Thomas, M. (2011). Characteristics of and implications for students participating in alternate assessments based on alternate academic achievement standards
.
Journal of Special Education, 45(1), 3-14. doi:10.1177/0022466909344223 Liu, K. K., Goldstone, L. S., Thurlow, M. L., Ward, J. M., Hatten, J., & Christensen, L. L. (2013). Voices from the field: Making state assessment decisions for English language learners with disabilities. Minneapolis, MN: U of MN, IVARED. NCEO. (2014). Exploring alternate ELP assessments for ELLs with significant cognitive disabilities (NCEO Brief 10). Minneapolis, MN: U of MN, NCEO.221
Slide222Resources
Quenemoen, R., Kearns, J., Quenemoen, M., Flowers, C., & Kleinert, H. (2010).
Common misperceptions and research-based recommendations for alternate assessment based on alternate achievement standards
(Synthesis Report 73). Minneapolis, MN:
U of MN, NCEO.Quenemoen, R. (2008). A brief history of alternate assessments based on alternate achievement standards (Synthesis Report 68). Minneapolis, MN: U of MN, NCEO.Thurlow, M. L., Liu, K. K., Ward, J. M., & Christensen, L. L. (2013). Assessment principles and guidelines for ELLs with disabilities. Minneapolis, MN: U of MN, IVARED.222
Slide223Martha ThurlowNational
Center on Educational Outcomes (NCEO)THURL001@umn.edu
223
Slide224What we’re learning about Alt ACCESS
H Gary Cook, Ph.D. Associate Research Scientist
Research
224
Slide225Overview
About Alt ACCESSWhat we’re learning about attainmentWhat we’re learning about growth
Research
225
Slide226What Is Alt ACCESS?
Alternate ACCESS for ELLs (Alt ACCESS)Assessment based on WIDA’s Alternate Model Performance Indicators (AMPIs) www.wida.us/assessment/alternateaccess.aspx
Meant to assess ELs with significant cognitive disabilities
Adaption of an assessment created from a 2008-2011 USDE Enhanced Assessment Grant
Research
226
Slide227What is Alt ACCESS?
Locally administered assessmentFour grade clusters: 1-2, 3-5, 6-8, 9-12Four domains: listening, speaking reading, writingMeasures social
instructional
language,
and the language of ELA, mathematics, and scienceMeasures five proficiency levels
Research
227
Slide228Who participates in Alt ACCESS?
Eligible to participate on Alt ACCESS
Research
228
Slide229Attainment on Alt ACCESS
What does it mean to be proficient on Alt ACCESS? What does language proficiency look like for ELs with significant cognitive disabilities?One idea…Conceptually, apply methods used to identify English language proficiency on general ELD assessments
Research
229
Slide230A Proficient Performance Criterion
US Dept. of Education
(2012). Prepared by Cook, Linquanti, Chinen, Jung.
Conversations
about proficiencycan start here.
Research
230
Slide231Alt ACCESS to State Alternate Assessment Proficiency (in 4 states)
Logistic Probability Curves
Domain
A1/A2
A2/A3
A3/P1
P1/P2
Overall
Composite
924
931
938
944
Research
231
Slide232What Do We
See About Attainment?This method seems to provide information consistent with a similar method used for ELD and content assessments.
This method defines the English proficient performance criterion with reference to the state’s alternate assessment.
Research
232
Slide233Overall Composite Score Level Change on Alt ACCESS
262 154 279 180 96 Total = 971
Research
233
Slide234Overall Composite Score Level Change on Alt ACCESS
337 227 448 550 292 Total = 1854
Research
234
Slide235Overall Composite Score Level Change on Alt ACCESS
198 113 283 394 383 Total = 1371
Research
235
Slide236Overall Composite Score Level Change on Alt ACCESS
160 119 322 338 319 Total = 1258
Research
236
Slide237What Do We See
About Growth?Proportionally, most students did not move levels (except for level A2)
Largest level gain occurs at A2 level across all grade clusters
Students in level A1 grew the least across all grade clusters (discounting level P2)
Scale Score growth (not shown) very small
Research
237
Slide238Some Questions
Given the large number of student not progressing in proficiency on Alt ACCESS….How long would it take to be proficient for these students?What should our growth expectations be vis-à-vis AMAO 1 for these students?Is there a point where a student’s lack of growth suggests that Title III services are no longer helpful?
Research
238
Slide239Gary CookWisconsin Center for Education Research
University of Wisconsinhcook@wisc.edu
239
Slide240Moderated Discussion
240
Slide241Questions
241
Slide242Diane AugustManaging Researcher
1000 Thomas Jefferson St. NWWashington, DC 20007daugust@air.org
242