A Balanced Approach: Preparing for Smarter Balanced ELA Assessments

A Balanced Approach:Preparing for Smarter Balanced ELA Assessments

Ken Hermens, Oregon Department of Education

Regional ESD Workshops, Updated 2015

Getting AcquaintedWhat, in your experience, are the valued learning outcomes in the Common Core State Standards (CCSS)? What have you and your school site

done to transition to

the Common Core?

Four Corners1. Common Core instructional shifts

2. Smarter Balanced in general

3. Smarter Balanced claims, targets, and rubrics

4. Smarter Balanced performance tasks

HistoryState Board of Education adopts the Common Core State Standards for ELA and Math in October 2010

Standards reflect the expectations of college and career readiness by the end of high school

Development led by CCSSO and NGA, included teachers, parents, administrators, and content experts from across the country

Smarter Balanced Assessment System adopted by the State Board of Education in May 2013

Smarter Balanced Membership

Moving Forward2013-14

2014-15 and beyond

OAKS Reading and Writing

Smarter

Balanced ELA

OAKS Math

Smarter

Balanced Math

OAKS

Science and Social Sciences

OAKS Science and Social Sciences

Extended

Assessment

Extended AssessmentELPAELPAKindergarten AssessmentKindergarten Assessment

New Portal Address for OAKS: http://oaksportal.org

Testing Windows**Testing shall not begin until at least 66% of a school’s annual instructional days have been completed in grades 3-8, and 80% of annual instructional days in High School

Smarter Balanced Math

March 10

th

to June 12

th**

Smarter

Balanced ELA

March

10

th

to June 12th** Science and Social SciencesJanuary 6th to May 28th 12th Grade Retest Math and Reading

January 6th to May 28th12

th

Grade Retest Writing

January 6

th

to March

13th

ELPA

January

6

th

to April 15

th

Kindergarten Assessment

August

11

th

to October 23

rd

How do the CCSS define college and career readiness? The Common Core State Standards: Are aligned with college and career expectationsInclude rigorous content and application of knowledge through higher order skillsBuild upon strengths and lessons of current state standards

Are informed by top-performing countries, so that all students are prepared to succeed in our global economy and society

Are evidence- and/or research-based

Handout: What a College-and-Career-Ready Student Looks LikeSkim the handout on college and career readiness. As you read, think about:the major shifts in instruction the learning opportunities that students will need in order to meet these shifts

College and Career Ready They demonstrate independenceThey build strong content knowledge.They respond based on audience, task, purpose.They comprehend as well as critique.They value evidence.They use technology/digital media strategically and capably.

They understand other perspectives and cultures.

Oregon Common Core ShiftsIncreased reading of Informational TextsIncreased emphasis on Text Complexity

Academic Vocabulary

Text-based Answers

Increase of Writing from Sources

Literacy Instruction in all Content Areas

Oregon

CCSS Toolkit

http://www.ode.state.or.us/go/CommonCore

Common Core State Standards—ELA & Literacy Resources

http://www.ode.state.or.us/search/page/?id=3359

Development Timeline

Smarter Balanced Assessments

Smarter Balanced Test DocumentsTest BlueprintsNumber of itemsScore points (weighting)Depth of knowledgeContent SpecificationsList of all Assessment Targets

Identify which CCSS each Target assesses

Item Specifications

Broken out by grade level, Claim, and Target

Contains Task Models

Appropriate Stems

Components of the Smarter Balanced Summative TestCAT Test (Computer Adaptive Test)

Item Types:

Selected Response (Multiple Choice)

Technology-Enhanced/Enabled Items

Short Constructed Response

Performance

Task

Two-day Event

Classroom Activity

Short Answer to provide scaffolding

Multiple

Resources

http

://www.smarterbalanced.org/smarter-balanced-assessments/

Smarter Balanced Test Times

Smarter Balanced AssessmentsThe overall claim for grades 3-8: “Students can demonstrate progress toward college and career readiness in English language arts and literacy.”

The overall claim for grade 11:

“Students can demonstrate college and career readiness in English language arts and literacy.”

Claims, Targets, and Standards

3

rd

Grade ELA

CCSS

Target

Content

Category

Claim

Overall

Claim

Smarter Balanced Claims

SB Item SpecificationsFor each of the different Claims, Smarter Balanced has developed assessment

Targets

specifying the content of items that will be on the summative assessments.

Targets are generally associated with

one CCSS

, but some Targets address more than one

There are

multiple models

for each Target

Models are

scenarios or descriptions

of what an item or activity might look like to assess any given

Target

SB Targets and ModelsEach Target has anywhere from two to six potential models; these are identified on the Item Specifications Tables

The

Models

each indicate the

assigned item type

(SR, CR, or TE) and DOK

For

each item

, item writers choose

ONE of the Models

and use the description provided to write the type of item that has been identified.

Generally, there

is at least one Model for each Target

that addresses SR items and at least one addressing CR items.

Sample Blueprint Table (Grade 11)

Table ActivityUsing the ELA Test Blueprints…Familiarize yourself with the documentNote where content categories are being assessed: in the CAT portion, the PT portion, or both

Note the number and types of items for the different claims

What is DOK measuring?

What is and is not measured in the PT?

Targets for Claim 1: Reading

Targets for Claim 2: Writing

Targets for Claim 3: Listening

Targets for Claim 4: Research

Claim #4—Research/InquiryWhy is a research claim called out separately from reading and writing? What is the purpose of this claim?

Discuss with your group.

GRADEWORD COUNT RANGE(short text)WORD COUNT RANGE

(long text)

3

200 – 487

488

– 650

4

450 – 562

563

– 750

5

450 – 562

563 – 7506

650

– 712

713

– 950

7

650

– 712

713

– 950

8

650

– 712

713

– 950

11

800 – 825

826

– 1100

Grade

“Stretch”

Lexile

Band

3

420 – 820

4 – 5

740 – 1010

6 – 8

925 – 1185

9 – 10

1050 – 1335111185 – 1385SB Word Counts and Lexile Targets

Smarter Balanced Item Types

Variations on Selected ResponseMultiple Choice, single correct responseMultiple Choice, multiple correct response (4-8 options, 2-4 correct)Two-part Multiple Choice (Part A: conclusion or inference, Part B: select evidence)

Hot Text, select text (“click on” number, line, sentence, paragraph)

Hot Text, re-order text (drag and drop)

Matching Table

Sample Two-Part Multiple ChoicePart A

Click on the statement that

best

describes what information in the first paragraph of Pong reveals about the author’s point of view.

[The author appreciates advancements in technology.]

[The author celebrates the revolutionary nature of Pong.]

[The author prefers old style games like Pong over current video games.]

[The author questions the connection of current technology to technology of the past.]

Part B

Click on the sentence from the text that best supports your answer in Part A.

[On any smartphone you pick up, you will find some kind of video game.] [People have access to games so sophisticated that there are worldwide competitions to determine the best players.] [Game design is so advanced that the graphics look almost real.] [Long before any of this, however, people played a low-tech game that sported paddles made of short lines on the edges of the screen, a dotted centerline to mark opposing sides, and square dot designed to act like a ball.] [That game was I.]

Sample Matching Table

Argument/Claim

Source 1

Source 2

Both Sources

Neither Source

Roundabouts save money









Roundabouts save lives









Roundabouts reduce accidents









Roundabouts are popular with locals









Roundabouts harm pedestrians









Roundabouts ease traffic congestion









Click on the boxes to reflect which source, if either, makes the following claims:

Sample Brief Response Reading Item

Constructed ResponseBrief Writes (both CAT and Performance AssessmentDraft introductions, conclusionsAdd a topic sentence or descriptive detailProvide transitions, dialogue

Revise to eliminate conventions errors

Full Writes (Performance Assessment)

Short Text Constructed Response (CAT Test)

Identify themes, main ideas

Analyze structures, characters

Compare and contrast two versions

Smarter Balanced:Training and Practice TestsTraining TestShorter TestIncludes tutorials on item typesNot intended to approximate content coverage

Reflects full range of supports and tools

Practice Test

Somewhat longer than training test

Shows more of the breadth of content

Reflects full range of supports and tools

Oregon Accessibility Manual

Qualitative measures

– levels of meaning, structure, language conventionality and clarity, and knowledge demands often best measured by an attentive human reader.

Reader and Task considerations

– background knowledge of reader, motivation, interests, and complexity generated by tasks assigned often best made by educators employing their professional judgment.

Quantitative measures

– readability and other scores of text complexity often best measured by computer software.

Determining Text Complexity

Text complexity is determined by:

Qualitative

Quantitative

Reader and Task

Determining Text Complexity

Measures such as:

Word length

Word frequency

Word difficulty

Sentence length

Text length

Text cohesion

Step 1: Quantitative Measures

Qualitative

Quantitative

Reader and Task

Step 2: Qualitative Measures

Measures such as:

Levels of meaning

Levels of purpose

Structure

Organization

Language conventionality

Language clarity

Prior knowledge demands

Determining Text

Complexity

Qualitative

Quantitative

Reader and Task

Step 3: Reader and Task

Considerations such as:

Motivation

Knowledge and experience

Purpose for reading

Complexity of task assigned regarding text

Complexity of questions asked regarding text

Determining Text

Complexity

Qualitative

Quantitative

Reader and Task

Smarter Balanced Performance TasksRequire student-initiated planning, management of information and ideas, interaction with a variety of other materials.

Require production of extended responses, such as oral presentations, exhibitions, and other

scorable

products, including more extended writing responses which might be revised and

edited.

Reflect a real-world task and/or scenario-based problem; tasks are multi-stepped and allow for reflection and

revision.

Smarter Balanced Performance TasksAllow for multiple approaches to developing and organizing ideas

Measure capacities such as depth of understanding, research skills, complex analysis, and identification/providing of relevant evidence

Represent content that is relevant and meaningful to students

Table Activity: Performance TasksAt your table, look at the Smarter Balanced Performance Task packet, focusing on the one most appropriate for your grade level assignment.After, individual exploration, discuss as a group what skills a student would need in order to successfully complete a Performance Task.Select a recorder to capture the most critical skills on chart pack.Report out to large group one or two skills you identified.

Smarter Balanced Performance TasksPart 1:

Consulting Resources and Scaffolding

Read/review/reflect upon

resources

related to the central topic; sources

represent

a variety of perspectives and viewpoints

Students respond

to several

(usually 2

or 3) scaffolding questions relating

to the research claim which

can

provide scaffolding for the culminating essay or “full write.”If time, students begin planning/drafting of the “full write” essay.

Smarter Balanced Performance TasksPart 2: Final Essay or “Full Write”

Students

have

continued access to the source materials through the writing process

but NOT to

their responses to the

research questions

.

Although students

are given

a general topic or theme, they

develop

their own thesis or topic statement.

Essays should be multi-paragraph and draw directly from at least two or more of the source materials. Source attributions should be made within the student work.

Combination of Claims and Targets Assessed by GradeGrades 3-5

Writing (

Narrative, Informational/Explanatory, Opinion

)

+ Research

+ Possibly Reading

Grade 11

Writing (

Informational/Explanatory, Argumentative

)

+ Research

+ Possibly Reading

Grades 6-8

Writing (Narrative, Informational/Explanatory, Argumentative)+ Research+ Possibly Reading

Stimuli Used in a PTGrade Span

# of Sources*

3

2

4

– 5

3

6

– 8

3-4

11

4-5

*each has 2-5 depending on grade level

Total Word Counts for PTs

Grade SpanMaximum Word Count3

1000

4

– 5

1400

6

– 8

2400

11

3400

Smarter Balanced Performance Tasks

Three Attributes Purpose/Organization

(

4-point scale)

Evidence/Elaboration

(

4-point scale)

Conventions

(

2-point scale)

Individual attribute scores

contribute to the overall score on the summative assessment.

ELA Performance Tasks:

Smarter Balanced Rubrics

Smarter Balanced Rubrics:Purpose/OrganizationKey Elements:

Strong main idea or claim, and focus is maintained appropriate to

audience

and

purpose

Variety of transitions

used to clarify relationships between and among ideas

Introduction and conclusion are present

Logical progression

with strong connections; “syntactic variety”

For argumentative mode,

opposing arguments are acknowledged or

addressed

Smarter Balanced Rubric: Evidence/ElaborationKey Elements:

Effective use of

sources, facts and details

as support for ideas or

claims

Effective elaboration of ideas using

precise language

(elaboration may include the use of personal experiences relevant to the main idea)

References to sources are relevant and specific and effectively

integrated into the essay

Vocabulary is

appropriate

for the

audience and purposeStyle enhances content

Smarter Balanced Rubrics: ConventionsKey Elements:

Correct sentence formation

Correct capitalization

Correct grammar/usage

Correct

spelling

Scoring is affected by:

Variety or range of errors

Severity of errors (Basic errors are more heavily weighted than higher-level errors)

Density of errors (Proportion of errors to the amount of writing/length of the essay

)

Skills for SuccessCiting: Using evidence within a text to support a claim, explain a concept, justify an analysis)Quoting: Knowing when to insert direct quotations for maximum impact and effect

Attributing:

Giving credit for ideas or information taken from source materials either formally or informally

Skills for SuccessParaphrasing: Putting information into your own words, complete with attributionIntegrating: Inserting information from sources seamlessly into the paper

Transitioning

:

Moving smoothly from idea to idea within and between paragraphs

Skills for SuccessCountering: Addressing counter claims in Argumentative Writing to acknowledge they exist; refute them or minimize their significanceSynthesizing: Pulling together material from multiple sources, drawing comparisons, noting contrasts in style, content, and point of view.

Highlighting:

Regularly engage students in highlighting text for different purposes (e.g., main ideas, descriptive details, unfamiliar vocabulary, etc.)

Skills for SuccessElaborating: Narrative: Include details, description and dialogueExplanatory: Include examples, explanations, definitions, statistics, factual details

Argumentative: Include a stated claim that is maintained, integrated source material, logical reasoning, problem solving, counter claims.

Table Activity: Skills for SuccessEach table will be given a card with a critical skill or skills listed.Brainstorm as a group lessons or activities you could structure to address the skills on your table’s card to facilitate student success.Capture your ideas/strategies on chart pack.Report out to large group.

Student Perspectives

“Typing was hard but I thought it [test] was more different and cool. Writing is good because I can write down my thoughts. I have good explanations that I want someone to hear.”

–Jacklyn, 5

th

Grade

“Tell them they need to prepare for not just clicking an answer but wording it [responses] in a way that makes sense…work on typing and work on how to answer in words.”

–Sicily, 6

th

Grade

“Practice typing.” –

Darbi, 5

th

Grade

“…Good to teach us [students] how to go more in-depth with essay, paragraph, and sentence structure.”

–Ella, 6

th

Grade

“Practice typing because there’s a lot of typing, and practice essays…how to do them…how to write them.” –

Van, 4

th

Grade

Strategies for Classroom PracticeClose ReadingTake the Practice Testhttp://www.smarterbalanced.org/practice-test/

Practice Performance Tasks

Create “cousin items” (e.g., Matching Tables)

http://www.smarterbalanced.org/smarter-balanced-assessments/

Close ReadingNumber the stanzas or paragraphs

(helpful when calling attention to specific information)

Chunk the text

(Draw lines to separate logical divisions in the text

, e.g.

¶ 1-3, 4-6, 7-9. Do it for them at first, later have them do it themselves)

Underline and circle…with a purpose

(Identify what you want them to do: underline the claims or main idea, circle unfamiliar vocabulary, circle examples of figurative language.

Close ReadingMargin Notes

Left

margin:

Summarize what the author is saying in 10 words or less.

Right margin:

Dig deeper into the

text; use “power verbs” to describe what the author is doing (describing _______, illustrating_________, arguing_________, comparing__________). Be complete!

Links to ResourcesSmarter Balanced Assessments

http://www.smarterbalanced.org/

Text Complexity

http://www.ccsso.org/Navigating_Text_Complexity.html

http://achievethecore.org/page/642/text-complexity-collection

Room for Debate: New York Times

http://www.nytimes.com/roomfordebate

Close Reading

http://iteachicoachiblog.blogspot.com/2012/06/five-simple-close-reading-strategies.html