Summer Scoring Training - PowerPoint Presentation

Slide1

Summer Scoring TrainingSmarter Balanced Mathematics

Deborah J. BryantSeptember 18, 2015Slide2

Purposes and Objectives

• Learn how the summative assessments are scored• Practice using standard handscoring

tools and

processes

• Experience methods of applying standardized

criteria

to the evaluation of student work

• Use scoring information to improve instruction in

the

classroomSlide3

How Do Educators Benefitfrom Scoring Training?

Enhanced assessment literacy – Deeper understanding of Smarter Balanced

claims

and

targets

–

Better understanding of

handscoring

procedures

–

Opportunity to take a closer look at rubrics

–

Opportunity to see examples of student work at

a range

of score points

• Instructional Improvement

–

Ideas for improving instruction and evaluation of

student

work in the classroomSlide4

Standard

Handscoring Materials

Provided by Smarter Balanced

– Standards for Mathematical Practice/Targets from the Mathematics Summative Assessment Blueprint

Ensure

raters have the tools to apply the scoring

criteria

consistently

Include

the following:

–

PT sets: stimuli and prompts

–

Scoring rubrics and exemplars

–

Samples: responses with scores and

annotations

–

Practice set: sample responses for you to

score

Slide5

Summative Scoring at ETS

• Training—Summative scoring training is similar to the summer scoring training.

•

Certification—

Prospective raters must take and

pass a certification test

on a set of

prescored

responses

.

•

Calibration—

Before raters start scoring, they must take and

pass a calibration set to demonstrate that they are ready to

score accurately

.

•

Double Scoring—

10% of responses are double scored to

verify that raters are scoring consistently with the rubric

.

•

Backrating

—

Scoring Leaders “read behind” raters and score

responses raters have scored to provide another check.Slide6

Summative Scoring (cont.)

Scoring Leaders: – Provide feedback to raters for additional content

support

; help prevent score drift;

serve

to

maintain scoring

pace.

–

Mentor struggling raters and/or direct

them to preselected

sample sets for

retraining

.Slide7

Scoring Terms

Stimulus: Short scenario with useful information needed to solve problems.Prompts:

The questions to which students are

responding, also

called “items.”

Rubric

:

A standardized approach to score

student responses

. Math rubrics are item-specific.

Exemplar

:

An example of a correct answer. It is

important to

note that equivalent answers are always accepted

unless

specific

notes in the rubric indicate that only a certain set

of answers

will be considered as correct.

Samples

:

Student responses that are

prescored

and annotated

.

Annotations

are often just as important

and insightful

as the rubric.Slide8

Mathematics Assessment

Performance Task (PT): A task with six prompts based on a common stimulus or scenario

; some items will be scored based on

responses

to other prompts in the same PT.

•

Non-PT (CAT):

A single task with a single

prompt

.

•

Other:

Some machine-scored prompts need

human

scoring when administered on paper.Slide9

Sample Annotations

An annotation is the rationale given for why a response receives full, partial, or no credit.The annotations written for samples provide

further

insight into how student

responses

were scored. It is important to read the rubric

and annotations carefully.Slide10

Scoring Dependent Items

Item 1 Response

Item 2 Response

Item 3 Response

Item 4 Response

Item 5 Response

Item 6 Response

A

Dependent Item

is an item that depends on the response to a previous item(s) in the PT. We call these previous items

Parent Items

or

Related Items.

If the reasoning is correct on item 5, but there is a follow-through error from items 3 and/or 4, award full credit.Slide11

Scoring Dependent Items (cont.)

How can the dependent item receive full credit if theparent item received partial or no credit?

If

the student uses the answers he or she gave to the parent

item and correctly applies them to find a solution to the

dependent item, he or she can earn full credit for the answer

on the dependent item

.

It is also possible that while answering the dependent item,

the student realized his or her errors in the parent item and

may correctly answer the dependent item without relying on

the answer to the parent item.Slide12

Assessment Claims and Targets forMathematics Summative Assessment

Assessment Claims are broad evidence-based statements about

what students know and can do as demonstrated

by their

performance on the assessments

.

Assessment Claims

1

. Concepts &

Procedures-40%

2

. Problem Solving

40% 3

. Communicating

Reasoning-20%

4

. Modeling Data

Analysis

Within each claim there are several assessment targets which really define the nature of the performance expectations.Slide13

SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample

A TRIP TO THE ZOOAnna and her family go to the zoo. The gift store prices are shown in the table.

Gift Store Prices

Gift

Price

Stuffed panda bear

$9

Zoo

magnet

$4

Pack of 4 pens

$6

Photo frame

$8Slide14

SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample

ContinuedSlide15

SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued

Read the rubric and the exemplar. Think about the different types of responses that a student may provide to correctly answer the prompt. Discuss with your group or partner what information the student will need to answer the prompt.

For this item, a full-credit response (2 points) includes

-stating

correct gifts that Anna and Ray can

buy based

on the

student’s

response to item

217

and

the

restrictions in the stem

AND

-explaining

why there is enough money for

the gifts based

on

the student’s

response to

item

217

.

For example: “Anna and Ray can buy a stuffed panda bear and a pack of 4 pens which will cost $13. There is $25 dollars left to buy gifts. $13 dollars is less than $25 dollars.” Slide16

SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued

For this item, a partial-credit response (1 point) includes: -stating correct gifts that Anna and Ray can

buy based

on student’s response to item

217

and the

restrictions

in the stem

.

For example: “They can buy a stuffed panda bear and a zoo magnet.”

For this item, a no-credit response (0 points) includes none of the features of a full- or partial-credit response

.

For example: “They can buy 4 stuffed panda bears.”

This item is not graded on spelling or grammar.

Slide17

SBAC Practice Test Scoring Guide Grade 4 Performance Task Sample Continued

-Individually read and score Practice Set 1.-Discuss the practice papers with your group, providing evidence from the rubric and samples when you assign your scores. Try to reach consensus on a single score for each student response.

-They would share the score that SBAC assigned to the practice papers along with the reasons why.

A TRIP TO THE ZOO

A TRIP TO THE ZOO

Your Score

Group Consensus

Score

SBAC Score

Comments

Practice 1

Practice 2

Practice 3Slide18

Closing Activity

Reflect on today’s session, and discuss with your table group:Strategies you learned that are relevant to classroom instruction and assessmentIdeas you will take back to the classroom

Other thoughts about scoring training and professional development

Share ideas with larger group!Slide19

Interested in Scoring?/Questions?