SY 1617 Update 10 JUNE 21 2017 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION 1 Topics Operations TIDE Reporting Documents Parent Communication Data Reporting Timelines System Launches ID: 760351
Download Presentation The PPT/PDF document "Assessment Update Webinar" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Assessment Update Webinar
SY 16-17 Update #10JUNE 21, 2017
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
1
Slide2Topics
OperationsTIDEReporting DocumentsParent CommunicationDataReporting TimelinesSystem LaunchesTIDE Updates
Content DevelopmentTeacher Involvement/Development OpportunitiesDigital Library StorefrontELA ScoringNAEPParticipation/Field NotificationELPA21Screener and ReportsTechnology
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
2
Slide3Operations
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
3
Slide4TIDE Appeals
Online Testers: Appeals to Invalidate and Reset student online test results may be flagged in TIDE now through June 25.Paper Testers: Appeals to Invalidate and Reset student paper test results may be flagged in TIDE according to the schedule outlined below.
Test available under TIDE administration drop down AssessmentOnline Tests TIDE Appeals Window ClosesPaper Tests TIDE Appeals Window Spring 2017Smarter Balanced6/25Now-7/6 Spring 2017Off Grade Science 6/257/10-7/12Spring MSP 2017MSP 6/257/17/-7/19Spring EOC 2017Spring Math EOC N/A7/25-7/27Spring EOC 2017Spring Bio EOC N/A7/25-7/27
Only state-level users will be able to approve invalidation appeals.
4
Slide5Participation Codes in TIDE
Participation Window in TIDE closes July 3Within this application, districts can establish students attempt code (reason not tested). It also allows for districts to identify exemptions (NNEP, Home-based, Private school, F1 Visa). The state does not utilize the Reason not Tested and Exemptions data from TIDE. The Exemption calculations are collected from CEDARS.L2 Basic must be applied in TIDE, under Participation Codes, prior to the end of the window on July 3. The state utilizes this data from TIDE for reporting to ORS, ISRs, and the Report Card web site.
Data Required for State Use
Optional; District Use Only
5
Slide6Reporting Documents
All reporting documents, as outlined in the PIRG, are to be reported to the state. Test Incident Report: All test incidents must be reported by Friday, June 23Note whether students were able to complete their testing opportunity District Administration and Security Report: (Required) – Completed reports for each administration must be signed by the district Superintendent and submitted to the state within one week of the conclusion of the district’s test administration. Test Material Variance Form: Discrepancies in the quantity of materials received or returned to vendor, or damaged materials shall be reported immediately. Missing or damaged materials must also be notated in your District Administration and Security Report.
Do not send secure or confidential student information through email
6
Slide7Parent Communication
Spring 2017: Students took one or more of the Smarter Balanced, MSP, Off-Grade Level, and/or EOC assessments.Individual Score Reports (ISRs): Hard copy reports arrive in district August 17—18Shipment includes two copies of each ISR. One copy for parents and one copy for student folder.Support Documents: Materials in support of parent communication available the first of August (English versions)State Testing Parent Letter Templates Your Child’s Progress handoutGraduation in Washington Toolkit
Translated versions available the first of September
7
Slide8Testing Options SY 2017-18
School Year 2017-18 State Testing Schedule Calendar Posted: http://www.k12.wa.us/assessment/StateTesting/timelines-calendars.aspx School Year 2016-17 calendar available for referenceConsiderations for School Testing Schedules Paper-pencil tests for students with IEP designating paper as the testing platform Early release and late starts Holidays and Winter and Spring breaks PT 15 day expiration and CAT 45 day expiration
8
Slide9Data
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
9
Slide10Data – CEDARS/DDSM activities
Directory updates due by July 3Opening Schools, reopening schoolsGrade spansCategory/type changes LEA changesWork with DDSM for more informationEnd of year and start of year rollover activites
10
Slide11Data Reporting Timelines
Final enrollment data extract from CEDARS: July 10Complete scorefiles arriving at OSPI:First file to OSPI from AIR (MSP Science) July 17Last file to OSPI from AIR (EOC math and bio) July 24Final data all approved July 25Then, we merge with data from CEDARS, QC the data, and load the data to our applications:EDS applications (CAA/CIA, WAMS, Report Card Preview, WA Query): August 8 (target)Report Card Live (current schedule) August 15SGPs: tentatively late August or early September (tbd)
11
Slide12Systems Launches: 2017-18
TIDE: August 15TS GOLD: August 15eDirect: tbd (early September)WAMS Registrations open during week of Labor DaySpring WA-AIMHard-copy accommodated materialsWinter and Spring EOCELPA21 NOTE: specific registration deadlines vary by task
12
Slide13TIDE Update for 2017-18
Observations from several years of automated TIDE processesRoutine: “Student enrolled here, but the old district hasn’t exited”Routine: “Student isn’t enrolled but will be testing with us”Common: “Student enrolled with us all year is somewhere else now”Common: “Student has two records in TIDE, one with the correct SSID…”Rare: “Student took part of test under SSID 1 and another under SSID 2…”Rare: “Student showed up and MUST test before our next CEDARS submission”Rarest: “Student showed up today and MUST test today!”Conclusion 1: 5 minute files from issuance are creating as many problems as they solve. Conclusion 2: If a student could be in 2 schools at once in TIDE, we wouldn’t need so many workarounds
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
13
Slide14TIDE Update for 2017-18 (part 2)
TIDE able to support visibility of student records in multiple locationsCurrent proposal for data flow redesign:1) Nightly files continue based on full CEDARS enrollment recordsBUT during the lag between a student exiting and a student entering, a record is sent at new siteAll test settings follow student to new siteDemographics/identifiers update based on new siteBoth sites can see record until old site reports ‘exit’ 2) No more ‘5 minute files’ directly from issuanceCaused HUGE volume of SSID merges and deletes needed in TIDECreated a 3-points-of-contact solution to fully clean up records in TIDE
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
14
Slide15TIDE Update for 2017-18 (part 3)
TIDE able to support visibility of student records in multiple locations3) Add WAMS ‘Send Student Record’ screenOnly connects to TIDE (not AIM which requires SpEd flag)Leverages same SSID Issuance table as 5 minute files process Only sends records intentionally (no more ‘accidentally pulled all the skills center kids…’)4) OSPI Data Analysts retain add/upload student function“Last Resort” for students needing to test RIGHT NOW!!Will only add/upload based on real SSIDs – no temp records without SSIDs (except ELPA21 screener)
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
15
Slide16NAEP
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
16
Slide172018 NAEP and International Assessments Update
For 2018, Washington is participating in NAEP, TIMSS, and ICILS. Assessed grades and subjects have been posted to the NAEP website at http://www.k12.wa.us/assessment/NAEP/default.aspx .On June 16, 2017 OSPI notified superintendents and assessment coordinators of districts selected to participate in 2018 NAEP, ICILS, or TIMSS.Selected schools will be notified between June 23-June 26, 2017. School notifications will include the estimated sample size, the school’s assessment date, and space and time requirements. District assessment coordinators will be included in these notifications. Schools should notify OSPI by July 10, 2017 to request a new assessment date if there is a conflict. The NAEP administration will occur during Winter 2018, TIMSS and ICILS administrations will occur during Spring/Summer 2018.
17
Slide18Content Development
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
18
Slide19Science Assessment Development OpportunitiesFall 2017—Summer 2018
Ongoing Educator Workgroup Assessment Development TimelineSeptember 2017: Content Review with Data Review student performance data for every field tested item and make recommendations for the operational item bank.October 2017: Content Review Review the products of Cluster Development to assure every stimulus, item, and rubric is scientifically accurate and gathers evidence about student mastery of the Washington State 2013 K-12 Science Learning Standards (NGSS).
19
Slide20Science Assessment Development OpportunitiesFall 2017—Summer 2018 (part 2)
Ongoing Educator Workgroup Assessment Development Timeline (continued)Spring 2018: Item Cluster Work Groups Write items aligned to the Washington State 2013 K-12 Science Learning Standards (NGSS) for science stimuli. Summer 2018: Pilot Rangefinding and Rubric Validation of field-test itemsPilot Range Finding: Review rubrics and student responses for hand-scored constructed-response field test items; finalize decisions about how every student answer on these items will be scored Rubric Validation: Review rubrics and student responses for machine-scored technology-enhanced field test items; finalize decisions about how every student answer on these items will be machine-scored.
20
Slide21Science Assessment Opportunities for new NGSS test, only in 2017—2018
Fall/Winter 2017: Achievement Level Descriptors (ALDs) Produce draft ALDs, descriptions of the knowledge and skills students demonstrate in each performance level.Winter/Spring 2018: Alignment StudyEvaluate the alignment of the science tests to the Washington State 2013 K-12 Science Learning Standards (NGSS).Winter/Spring 2018: Contrasting Groups StudyUse the Achievement Level Descriptors (ALDs) to predict how students will perform on the science assessment. Online ALD training is provided prior to this study.Late Summer 2018: Achievement Level SettingDevelop threshold score recommendations for student performance levels (Level 1, Level 2, Level 3, Level 4) on the state science tests.
21
Slide22Science Assessment Listserv
Interested in receiving email/text updates from Science Assessment?Go to the Subscribe page for GovDelivery. Choose a subscription type. Enter your email address/wireless number. On the Subscriptions page, select Content Areas > Science, then select the grade band(s) for which you would like to receive information. If you are interested in receiving updates from other OSPI departments, you can select those areas as well. Follow the directions until you have completed registration.
22
Slide23Teacher Involvement Update:Smarter Balanced
In March, we had over 200 educators apply for a wide range of Smarter Balanced activities through the end of the year.Recently, two more opportunities came open for work in October.About 100 educators applied; application closed June 15.OSPI is notifying all applicants this week of statusSmarter Balanced will make the final selection based on representation of grade levels, content, and demographic data across statesEducators will be contacted directly by Smarter Balanced or vendors managing the projectsInterested in receiving notices about activities like this? Subscribe for Math, ELA, and/or Digital Library email/text alerts.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
7/7/2017
23
Slide24Teacher Involvement Update:WA Specific
In addition to Smarter Balanced opportunities…ELA and Mathematics content teams will be doing “Member-managed” item developmentWashington educators will work with OSPI staff to write and review assessment items that will be used on Smarter Balanced assessmentsOriginally scheduled for AprilRescheduled for JulyInterested in receiving notices about activities like this? Subscribe for Math and/or ELA email/text alerts.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
7/7/2017
24
Slide2525
Smarter Balanced Digital Library:New Storefront
Smarter Balanced has released a public-facing Digital Library Storefront web page: SBDigitalLibrary.orgGives examples of the ideas, lessons, and activities you’ll find in the Digital LibraryProvides videos showing what teachers have to say about the Digital LibraryContains videos showing some of the new features of the Digital LibraryIncludes links to the Digital Library log-in page and a link for educators in Washington to register for their free account.Please share this page with educators, parents, community partners, etc.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
7/7/2017
Slide26ELA Performance Task Full Write Scoring
Questions:How can students earn 0 points on the rubric and still be at/near or above standard in their writing claim?Has the scoring of the full write items changed from previous years?What kinds of responses earn 0 points?Are more students earning 0 points this year compared to last year?Where in ORS can districts find data about writing mode and scores?How can teachers help students prepare for full write items?
26
Slide27Full Write Scoring and Writing Claim
How can students earn 0 points on the rubric and still be at/near or above standard in their writing claim?There are 11 items that contribute to the writing claim. The Full Write item is only 1 of those items.The other 10 items are included in the CAT.Examples of the other items can be found in the “LangVocab,” “Edit,” “Revise,” and “Brief Write” IABs.See the WCAP Portal for more information about:Claim scores in the Understanding Smarter Balanced Assessment Scores Audio presentation in the Test Coordinators > Modules folder.Writing targets in the ELA Summative Blueprint in the Test Administrators > General Information folder.
27
Slide28Full Write Scoring Process
Has the scoring of full write items changed from previous years?No, the processes and procedures for training the professional scorers have not changed from previous years.In addition, the processes and procedures for scoring the full write items have not changed. All the rubrics, training materials, and training processes have been approved by Smarter Balanced.We continuously track the statistics used to ensure the quality of scoring.Washington uses human readers to professionally score all constructed response items on state exams.
28
Slide29Causes of 0 points on Full Write
What kinds of responses earn 0 points?BlankWritten in a language other than EnglishOff TopicOff Purpose (Mode)Insufficient
29
Slide30Causes of 0 points (continued)
Off Purpose (Mode): The response does not follow the mode of writing given in the prompt.Example: When told to write a narrative based on the information in the sources, the student writes an informational text.Insufficient: The response is too brief to score.The response is not meaningful. For example, “IDK” and “I don’t care.”The student has not provided enough of their own, original writing to determine whether or not they are on purpose (mode). The response may include an abundance of text copied verbatim from the prompt or from the sources.
30
Slide31Statewide Data
Are more students earning 0 points this year compared to last year?No, the percent of non-scorable responses (full write items earning zeros) decreased from Spring 2016 to Spring 2017.What trends can be seen in the data?Insufficient responses are the cause of most zeros.Off-purpose (mode) responses are more common in the Narrative mode than in the other two modes.
31
Slide32Statewide Data (percent non-scorable)
Grade2016 mean % non-scorable2017* mean % non-scorable328.923.2424.314.8517.58.5616.110.3715.98.6813.27.8HS12.67.2
*2017 stats as of May 12, 2017
32
Slide33Statewide Data (trends in grades 3-5)
Grade-Mode2016 mean % Insufficient2017* mean % Insufficient2016 mean % Off-Purpose (Mode)2017* mean % Off-Purpose (Mode)3-Informational20.419.30.50.33-Opinion16.111.80.00.13-Narrative15.313.622.917.54-Informational18.813.10.70.54-Opinion6.34.31.50.24-Narrative7.37.225.211.95-Informational13.24.51.80.25-Opinion4.0 3.50.90.35-Narrative4.25.217.26.9
*2017 stats as of May 12, 2017
33
Slide34Statewide Data (trends in grades 6-8)
Grade-Mode2016 mean % Insufficient2017* mean % Insufficient2016 mean % Off-Purpose (Mode)2017* mean % Off-Purpose (Mode)6-Explanatory12.57.00.00.06-Argumentative10.15.90.00.06-Narrative13.77.47.79.67-Explanatory12.65.70.40.17-Argumentative14.36.80.20.07-Narrative9.45.811.98.88-Explanatory12.26.70.00.08-Argumentative8.73.80.10.08-Narrative9.15.710.99.4
*2017 stats as of May 12, 2017
34
Slide35Statewide Data (trends in high school)
Grade-Mode2016 mean % Insufficient2017* mean % Insufficient2016 mean % Off-Purpose (Mode)2017* mean % Off-Purpose (Mode)HS-Explanatory13.07.70.00.0HS-Argumentative10.45.90.20.1
*2017 stats as of May 12, 2017
35
Slide36District data in Online Reporting System
Where in ORS can districts find data about writing mode and scores?After logging in, select Retrieve Student ResultsIn the Step 1 box: Report Type: select “Student Data”Tested Grade: you can leave as “All Grades” but we suggest selecting a specific levelDownload Format: leave as “Excel”In the Step 2 box: School: leave as “All” or select a specific schoolThen select “Download”
36
Slide37Columns AE through AM
3 columns per mode (Informational/Explanatory, Opinion/Argumentative, Narrative)Organization/purpose = 4 points possibleEvidence/elaboration = 4 points possibleConventions = 2 points possible1 row per studentDigits are the scores which show in the online ISRs. Non-scorable responses earn 0’s.N/A = the student’s PT was not in this mode
Informational Organization/PurposeInformational Evidence/ElaborationInformational ConventionsOpinion Organization/PurposeOpinion Evidence/ElaborationOpinion ConventionsNarrative Organization/PurposeNarrative Evidence/ElaborationNarrative Conventions111N/AN/AN/AN/AN/AN/AN/AN/AN/A221N/AN/AN/AN/AN/AN/AN/AN/AN/A000
37
Slide38Guidance for teachers
How can teachers help students prepare for full writes?Clarify all the modes of writing and have students write for different purposes using resource textsGrades 3-5: narrative, opinion, informationalGrades 6-8: narrative, argumentative, explanatoryGrades HS: argumentative, explanatoryShow students how to use informational texts to inspire narrativesTeach students to develop their own thoughts first, then add quotations from the sources to support as evidence, and follow up with explanations of the text evidence to make connections.
38
Slide39ELPA21
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
39
Slide40ELPA21 Update: Screener
Transition Timeline ChangesOSPI has made a partial revision to the transition timeline for the ELPA21 Screener. This revision affects only in-coming kindergarten students. The change was made in response to communication that districts were unable to plan for the onset of a new school year with unknown elements around proctoring the screener (test duration, proctor-student ratios, etc.)In-coming kindergarten students can take the WELPA through September 30, 2017. Once the ELPA21 screener is launched (August 15) for statewide access, districts will have the discretion to continue with use of the WELPA for kindergartener screening or may opt to use the ELPA21 screener.
40
Slide41ELPA21 Update: Screener
Scoring TurnaroundOSPI has heard concerns around the 7-day turnaround needed for third party scoring. We are exploring ways to lessen the time it takes for scoring. For the coming school year, no changes will be made to the scoring process. Our vendor is committed to meeting the 7-day turnaround and is working to identify ways to streamline scoring to minimize the time it takes. What you can do to ensure scores are timely1. Screen potential ELs as soon as possible after enrollment.2. Make sure screeners are submitted, not paused, at the end of testing. 3. Check ORS for scores regularly after submission of screeners.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
41
Slide42ELPA21 Update: Score Reports
Sending Score Reports to FamiliesOSPI has become aware of an issue with K-1 test booklets that led to students receiving “not attempted” in the writing domain. This problem is limited to Kindergarten and first grade ELs. Districts have been contacted depending on their status as “affected” or “not affected” by this scoring error. All districts can disseminate score reports for all second grade and older ELs. “Not affected” districts can disseminate all score reports (including K-1 reports).“Affected” districts should pull score reports with “not attempted” reported for the writing domains of K-1 students and hold until given further direction.Students with actual impacted results will receive updated score reports. These are scheduled to come in August prior to the start of school.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
42
Slide43Technology
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
43
Slide44Technology Update: 17-18 Secure Browser
Updated Secure Browsers for use in 2017-18 are expected to be released by AIR on July 18, and will be available for download from WCAP portal Updated Windows Secure Browser (v.10) will no longer support VistaUpdated Macintosh Secure Browsers (v.10 for OS X 10.9 and higher; v.9.5 for OS X 10.7 and 10.8) will no longer support OS X 10.6Updated Chrome Secure Browser will block ChromeVoxIf updates to iOS client is needed, it will be available at the same time (note iOS 8.x will no longer be supported)
44
Slide45Contacts for Technology, System, or Program Issues
Test Administrators, School Test Coordinators, and District Test Coordinators contact the WCAP Help Desk (TAs and School Test Coordinators must also alert the District Test and Technology Coordinators of issues). The WCAP Help Desk is open Monday–Friday from 6:00 a.m. to 6:00 p.m. PT (except holidays or as otherwise indicated on the Washington Comprehensive Assessment Program Portal) and can be reached by:Toll-Free Phone Support: 1-844-560-7366 Email Support: wahelpdesk@air.org
45
Slide46USER SUPPORT
Questions regarding information in the Test Administration Manuals Test Administrators and School Test Coordinators, contact your District Test Coordinator. District Test Coordinators contact the OSPI Assessment Operations Team at: Assessment@k12.wa.us Policy or Test Administration Questions Test Administrators and School Test Coordinators, refer to the Test Administration Manual. If additional support is needed, contact your District Test Coordinator. District Test Coordinators contact the OSPI Assessment Operations Team at: Assessment@k12.wa.us
46
Slide47Contacts
AIR Help Desk for Technical, Network, and password issuesAIR Help Desk: wahelpdesk@air.org 1-844-560-7366OSPI Assessment Analysts for student data issuesOSPI Assessment Analysts: assessmentanalysts@k12.wa.us 360-725-6109OSPI Assessment Operations for assessment policy and test materialsAssessment Operations Assessment@k12.wa.us (360) 725-6348 or (800) 725-4311, press option 3Smarter Balanced has updated its website. Check out the revamped and more user-friendly site at smarterbalanced.org
47
Slide48Thank you!
48