for Music Teachers A Pennsylvania Perspective Created by O David Deitz for 20122013 PMEA Mentoring Program Webinar Series there are still a lot of questions to be asked and answered ID: 668626
Download Presentation The PPT/PDF document "Measuring Educator Effectiveness" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Measuring Educator Effectiveness for Music Teachers
A Pennsylvania Perspective
Created by O David
Deitz
for
2012-2013 PMEA Mentoring Program Webinar SeriesSlide2
(there are still a lot of questions to be asked and answered
!)?Slide3
Big Idea:
An effective
teacher in every classroom.Slide4
House Bill 1901
(B) FOR PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL
EMPLOYES WHO SERVE AS CLASSROOM TEACHERS, THE FOLLOWING
SHALL
APPLY:
(1) BEGINNING IN THE 2013-201 4 SCHOOL YEAR, THE EVALUATIO N
OF THE EFFECTIVENESS OF PROFESSIONAL EMPLOYES AND TEMPORARY
PROFESSIONAL EMPLOYES SERVING AS CLASSROOM TEACHERS SHALL GIVE
DUE CONSIDERATION TO THE FOLLOWING:
(I) CLASSROOM OBSERVATION AND PRACTICE MODELS THAT ARERELATED TO STUDENT ACHIEVEMENT IN EACH OF THE FOLLOWING AREAS:(A) PLANNING AND PREPARATION.(B) CLASSROOM ENVIRONMENT.(C) INSTRUCTION.(D) PROFESSIONAL RESPONSIBILITIES.(II) STUDENT PERFORMANCE, WHICH SHALL COMPRISE FIFTY PERCENTUM (50%) OF THE OVERALL RATING OF THE PROFESSIONAL EMPLOYEOR TEMPORARY PROFESSIONAL EMPLOYE SERVING AS A CLASSROOM TEACHERAND SHALL BE BASED UPON MULTIPLE MEASURES OFSTUDENT ACHIEVEMENT.
Race to the TopSlide5
5Slide6
6Slide7
Charlotte Danielson’s Framework for Teaching
Four Domains 22 Components 50%
Observation/Evidence
Danielson
Framework Domains
Planning and Preparation
Classroom
Environment
Instruction
Professional ResponsibilitiesSlide8
8
Domain Focus—
Adapted from Danielson’s
Framework for Teaching
8
Planning and Preparation
Classroom
Environment
Instruction
Professional
Responsibilities
What a teacher knows and does in preparation for teaching.
All aspects of teaching that lead to a culture for learning in the classroom.
Professional responsibilities and behavior in and out of the classroom
.
What a teacher does to engage students in learning.Slide9
Multiple Measures of Teacher Effectiveness – Observation/Evidence and the Danielson Framework
Classroom observations by Principal/supervisor, including evidence that demonstrates behaviors associated with improving student achievement:
Planning and preparation
,
including selecting standards-based lesson goals and designing effective instruction and assessment;
Classroom environment
,
including establishing a culture for learning and appropriate classroom management techniques that maximize instructional time;
Instruction
, including the use of research-based strategies which engage students in meaningful learning and utilize assessment results to make decisions abut student needs; and Professional responsibilities, including using systems for managing student data and communicating with student families9Slide10
10
10
The Framework for Teaching
The Framework for Teaching Charlotte Danielson
Domain 1: Planning and Preparation
a. Demonstrating
Knowledge of Content and
Pedagogy
b. Demonstrating Knowledge of Students
c. Setting Instructional Outcomes
d. Demonstrating Knowledge of Resources
e. Designing Coherent Instruction
f. Designing Student Assessments
Domain 2: The Classroom Environment
a. Creating
an Environment of Respect and
Rapport
b. Establishing a Culture for Learning
c. Managing Classroom Procedures
d. Managing Student Behavior
e. Organizing Physical Space
Domain 4: Professional Responsibilities
a. Reflecting on Teaching
b. Maintaining Accurate Records
c. Communicating with Families
d. Participating in a Professional Community
e. Growing and Developing Professionally
f. Showing Professionalism
Domain 3: Instruction
a. Communicating With Students
b. Using Questioning and Discussion
Techniques
c
.
Engaging Students in Learning
d. Using Assessment in Instruction
e. Demonstrating Flexibility and
ResponsivenessSlide11
Danielson Framework, Domain 3: Instruction
Component
1. Failing
2. Needs Improvement
3. Proficient
4. Distinguished
3a: Communicating with students
Expectations for learning, directions and procedures, and explanations of content are
unclear or confusing
to students. Teacher’s use of language
contains errors
or is
inappropriate
to students’ cultures or levels of development
Expectations for learning, directions and procedures, and explanations of content are
clarified after initial confusion
; teacher’s use of language is
correct but may not be completely appropriate to students’ cultures or levels of development.
Expectations for learning, directions and procedures, and explanations of content
are clear to students
. Communications are
appropriate
to students’ cultures and levels of development.
Expectations for learning, directions and procedures, and explanations of content
are clear
to students. Teacher’s oral and written communication is
clear and expressive, appropriate
to students’ cultures and levels of development, and
anticipates
possible student misconceptions.
3b: Using questioning and discussion techniques
Teacher’s questions are
low-level or inappropriate, eliciting limited student participation,
and recitation rather than discussion.
Some of the teacher’s questions elicit a thoughtful response, but
most are low-level, posed in rapid succession.
Teacher’s
attempts to engage all students
in the discussions are only partially successful.
Most of the teacher’s questions elicit a thoughtful response,
and the teacher allows sufficient time for students to answer.
The students are engaged and participate
in the discussion, with the teacher stepping aside when appropriate.
Questions reflect high expectations and are culturally and developmentally appropriate. Students formulate many of the high-level questions and ensure that all voices are heard.
11Slide12
Building Level Data
Building Level Data
PSSA Achievement
PVAAS Growth
Graduation Rate
Promotion Rate
Attendance Rate
AP Course Participation
SAT/PSAT
Building Level Data
PSSA Achievement
PVAAS Growth
Graduation Rate
Promotion Rate
Attendance Rate
AP Course Participation
SAT/PSAT
15 %Slide13
Teacher Specific/Elective Data
Teacher Specific Data
PSSA Achievement
PVAAS Growth
IEP Growth
Locally Developed District Rubrics
Elective Data/SLOs
District Designed Measures and Examinations
Nationally Recognized Standardized Tests
Industry Certification Examinations
Student Projects Pursuant to Local Requirements
Student Portfolios Pursuant to Local Requirements
15%
20%
35%
For
M
USIC:
ElectiveSlide14
What are some ways to develop evidence of student achievement in non-tested grades and subjects?Slide15
Student Learning Objectives
Other Assessments
Measures of Collective Performance
Three Approaches to the ChallengeSlide16
Measures of Collective Performance
School-wide growth measures (like the PA Report Card)
Standardized assessments used to measure the performance of groups of teachers (like program evaluation)Slide17Slide18
Student Learning Objectives
Content- and grade/course-specific measurable learning objectives that can be used to document student learning over a defined period of time
Can be written to measure growth or
masterySlide19
NY, IN, MA, GA, UT, RI, NH, New Haven CT, Austin TX
Student Learning Objectives
FL, DE, Pittsburgh PA
Other Assessments
KY, Pittsburgh PA
Measures of Collective Performance
What are other states/schools using for non-tested grades and subjects?Slide20
What is PA planning to do?
Student Learning Objectives with Model AssessmentsSlide21
Why this approach?
The “other assessment” approach causes narrowing of curriculum and is not financially sustainable.The “measures of collective performance” are already included in the formula (required by Race to the Top) and do not offer NTGS teachers an opportunity to be evaluated on what they actually teach.SLOs that don’t describe the assessment protocol tend to lack a true learning focus.Slide22
SLO Concepts
S
tudent achievement can be measured in ways that reflect authentic learning of content standards.
Educator
effectiveness can be
measured
through use of student achievement measures.Slide23
Educator Effectiveness: SLO
>
www.education.state.pa.us
Standards Based Education
Measurement of Student Achievement
Measurement of Educator EffectivenessSlide24
What knowledge and skills might be needed to develop an SLO?Slide25
Know and understand the
Standards
PA Standards
Common Core Standards
Professional/Technical
Standards
2. Assessment Tasks
Authentic to the grade or courseAligned to Standards3. Assessment ScoringCan describe levels of student achievement toward standards based learning objectivesSlide26
THE PA SLO TEMPLATE &
PROCESSWhat it is supposed to be
:
What it is not supposed to be
:
More paperwork for teachers that has no meaning or purpose
More testing for students
A weak substitute for PVAAS or other standardized testing data
More paperwork for teachers that has no meaning or
purposeMore testing for studentsA weak substitute for PVAAS or other standardized testing dataA format to inform strong instructional practice and strong student achievementA way to measure teacher effectiveness based on student achievementAn opportunity for teachers to define, describe and present data on student achievement in the content area that they teachSlide27
PA Educator Challenge
To develop and implement an appropriatelyrigorous measure of teacher effectiveness based
on student achievement in your content area
through the use of the PA SLO Template.
>
www.education.state.pa.us
Educator Effectiveness: SLOSlide28
The Approach
Provide a method for teachers to write SLOs.Provide a method for teachers to design assessments appropriate to the SLO.Provide piloted and peer-reviewed models on the PDE SAS website.Provide professional development appropriate to SLO development.
The Methodology:
AN SLO TEMPLATESlide29
STUDENT LEARNING OBJECTIVE TEMPLATE
A format to provide a measure of teacher effectiveness based on student achievement of content standards, as applicable to
the “Elective Data” portion of the Teacher Effectiveness System
in Act 82 (HB 1901).
Educator Effectiveness: SLO
>
www.education.state.pa.usSlide30
Looking at the Components of
PA’s SLO Template
Teacher Information
Content
Area
Student Learning Objective
Data and Targets Used to Establish the SLO
Assessment/Performance Task
Administration of the Assessment/Performance Task
Evidence of Individual
Student
Achievement
Strategies/Actions to Achieve
the SLO
Implementation Timeline
30
Teacher Effectiveness
Measure (Rating)Slide31
…one piece at a time.
Teacher Information
Teacher Name
School Name
District Name
2. Content Area
Course Title or Content Area
PA Standards Subject Area
Grade Level(s)
Brief Description of the course/ content area
Number of students per class/session
Frequency of classes/sessions
Number of minutes per class/session
Total number of classes/sessions
Slide32
9. Teacher
Effectiveness Measure
Classroom Objective
How will the aggregated scores of the “
Evidence
of Individual Student Achievement” results be used to define teacher effectiveness?
Failing
: few students achieve content mastery or growth
Needs Improvement
: less than a significant number of students achieve content mastery or growth
Proficient
: A significant number of students achieve content mastery or growth
Distinguished
: An exceptional number of students achieve content mastery or growthSlide33
Grade 8 General Music
Less than 60% of students will score 42-50 on a minimum of 2 assessments, with scores lower than 37 on the other two assessments.
60%-70% of students will score 42-50 on a minimum of 2 assessments, with scores lower than 37 on the other two assessments.
70%-84% of students will score 42-50 on a minimum on 2 out of the four assessments, and score no lower than 37 on the other two assessments.
85% of students will score 42-50 on a minimum of 3 out of the four assessments, and score no lower than 37 on a fourth assessment.
HS Instrumental
Music
Mastery:
Less than 50% of students achieve proficient or advanced levels in 5 out of 6 components on the second assessment.
Growth
: Less than 50% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components.
Mastery:
50% -74% of students achieve proficient or advanced levels in 5 out of 6 components.
Growth
: 50% -74% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components.
.
Mastery:
70% -84% of students achieve proficient or advanced levels in 5 out of 6 components
Growth
: 70% -84% of students achieve improved scores in 2 out of 6 components, with no decrease in scores in the other components
Mastery:
85% -100% of students achieve proficient or advanced levels in 5 out of 6 components
Growth
: 85% -100% of students achieve improved scores in 2 out of 6 components, with no decrease in the scores in other componentsSlide34
Student
Learning
Objective
Guiding Questions:
Why is this objective important and meaningful to your students
?
Sample Statements
:
Grade
8 General Music: Students will utilize musical notation as both a performer and composer.HS Instrumental Music
:
Students develop independent performance skills appropriate to positive musical contribution in an instrumental performing ensemble.
SLO (Student Learning Objective)
Learning objectives that can be validly measured to document student learning over a defined period of time.Slide35
SLO Rationale
Guiding Questions:
How is your student learning objective measurable and clearly understood by students in this grade/course?
Based upon this objective, how will students independently demonstrate their learning?
Sample Statements:
Grade 8 General Music
:
Students will independently demonstrate their learning by performing individually on the piano and guitar and by creating an individual musical composition based on the structural designs found in the music of the classic period (1750-1825).
HS Instrumental Music
: The quality and musicality of an instrumental ensemble is dependent on each individual as a contributor to the collective whole. Individual assessments inform each student of his/her progress and contribution as an independent musician.
SLO (Student Learning Objective)
Learning objectives that can be validly measured to document student learning over a defined period of time.Slide36
Curriculum Framework Connection
What are the big ideas, essential questions, concepts and competencies (as identified by PA’s Curriculum Framework) to be measured?
www.pdesas.org
Essential Questions
Concepts
Competencies
Big Ideas
Professional/Industry Standards
Common Core Standards
PA Standards
Statements from this section should be cut and pasted from
:Slide37
Student Preparedness/
Baseline Data
Guiding Questions:
What, if any, are the course or grade prerequisites?
What do you know about the students in this class at the beginning of the learning process?
What may typical and not-so-typical student progress look like on the way to achieving the learning objective?
Sample Statements:
Grade 8 General Music
:
There are no specific prerequisites. They will have had some instruction in music history, composition, guitar, and keyboard in grades six and seven. Large lapses of time between instruction periods (up to a year) will cause a need for extensive review of concepts and competencies previously experienced. Most students will be successful at manipulating the instruments and writing music notation. Some students will struggle with conceptualizing ideas regarding music composition. A pretest of retained knowledge of music notation is administered on the first day of the course to inform instruction.
4. Data
and Targets Used to Establish the SLOSlide38
4. Data
and Targets Used to Establish the SLO
Targets
Guiding Questions:
What is the expected outcome or goal by the end of the designated instructional period (i.e., year, semester, course length, etc.)?
Using student baseline data, are differentiated targets/goals needed? If yes, what are those targets/goals?
Are the targets ambitious yet realistic, and how will you monitor progress along the way?
Sample Statements:
HS Instrumental Music
:
Each student will demonstrate an ability to perform his/her instrumental part independently in an unaccompanied setting, exhibiting mastery and/or growth in such components as tone quality, intonation, rhythm, musicality, and technique appropriate to the student’s specific instrument.Slide39
5.
Assessment/Performance
Task
Name of the Assessment/
Performance Task
Sample Statements:
Grade 8 General Music
:
Classical Time Period Assessment
Composition Assessment
Keyboard Performance Assessment
Guitar Performance Assessment
HS
Instrumental Music
:
HS Instrumental Music-Individual Student PerformanceSlide40
5.
Assessment/Performance
Task
Description
of
the
Assessment/
Performance
Task
Guiding Questions:
Who is the developer of the assessments/performance task used (e.g., teacher-made, district-developed, commercial, etc.)?
What is a description of the assessment/performance task that will be used to measure the student learning objective (SLO)?
Are there any products or artifacts that will be gathered as part of the data collection process?
Sample Statements:
HS Instrumental Music
:
Individual Performance Assessments (
IPA
):
The IPA will occur two times during the year, near the mid-point of the first and fourth marking periods. Students will usually submit the assessment electronically, a variety of submission formats will be available.. Material for this assessment will be drawn from both scales and ensemble music currently being rehearsed in class. Specific technical exercises taught and rehearsed in class may be included as well.
IPA Requirements, First Marking Period 2012
Wind Symphony Slide41
Growth or Mastery
Does this assessment
/
performance
task align with the student
achievement goals
for the SLO?
Check one:
Growth (change in student achievement across two or more points in time) Mastery (attainment of a defined level of achievement
)
Growth
and Mastery
>
www.education.state.pa.us
Educator Effectiveness: SLOSlide42
6. Administration
of the Assessment/Performance Task
Frequency of
Assessment
/
Performance
Task
Administration
Guiding Questions:How often and when is this assessment/performance task administered?
If measuring growth, are multiple assessment windows in place?
Sample Statements:
Grade
8 General Music
:
Each of the four project
assessments is administered once throughout the course. Formative assessments are inherent in the instructional design. A pretest is administered on the first day of class that assesses prior knowledge applicable to the composition and keyboard assessments.
HS Instrumental Music
:
Students will be assessed at the midpoint of the first and fourth quarters.Slide43
6. Administration
of the Assessment/Performance Task
Resources Required
Guiding Questions:
What unique or specific equipment, technologies, or resources are needed to complete this assessment/performance task?
Sample Statements:
Grade 8 General Music
:
Keyboards, guitars, audio-playback equipment.
HS Instrumental Music
:
Video/audio recording technology and/or methodologies for students to create online submissions that will require computers and appropriate software.Slide44
All Classes:
IEP and 504 accommodations will be implemented.
6. Administration
of the Assessment/Performance Task
Adaptations for Diverse Learners and/or Students with Disabilities
Guiding Questions:
What assessment/performance task adaptations are needed to assist diverse learners and/or students with disabilities?
Sample Statements:
Grade 8 General Music
:
Students struggling with keyboard or guitar playing skills can perform simple chord structures.
Students can use a computer notation program to create the composition.
HS
Instrumental Music
:
IEP adaptations are made (additional time allotted, alternative print formats, teacher-directed instruction to help students feel more comfortable about being assessed individually, etc.)Slide45
Most Classes:
The assessments can be administered by an equivalent peer.
Some Classes:
Student use of equipment needed to complete the authentic assessment could be monitored by an equivalent peer, but should probably be monitored by the class instructor (i.e. athletic, career tech, chemistry, biology equipment).
6. Administration
of the Assessment/Performance Task
Personnel
Guiding Questions:
Can this assessment/performance task be administered by an equivalent peer (educator in a similar content area)? If not, please explain.
Does a district policy exist with regard to assessment/performance task administration?
Sample Statements
:
An equivalent peer can administer this assessment. Many students will self-administer this assessment.Slide46
7. Evidence
of Student Achievement
Rubrics/
Scoring
Scales
Guiding Questions:
How will individual student growth or mastery be determined (defined and scored) using this assessment/performance task? Include the specific rubric/scoring scale that will be used.
Does the rubric and/or scoring scale correlate with the assessment/performance task?
Sample Statements:
HS Instrumental Music
:
See next slide………….
OR LOOK AT SCOTT SHEEHAN’S PMEA WEBINAR PRESENTATION Slide47
7. Evidence
of Student Achievement
Category
Advanced
Proficient
Basic
Below Basic
10-9
8-7
6-5
4-0
Tone Quality
Tone quality is controlled, full, rich, and characteristic in all registers
Tone quality is characteristic in the normal playing range of the instrument, but distorts in a few passages or extreme registers (occasional lapses of control), these lapses rarely hinder the performance.
Tone quality exhibits some flaws in production in the normal range of the instrument, extreme registers are usually uncontrolled, tone quality occasionally detracts from performance.
Tone quality has significant flaws in basic production in all registers of the instrument and consistently detracts from the performance.
Intonation
Intonation is accurate in all ranges and registers
Intonation is mostly accurate; the few problem pitches are adjusted to an acceptable standard during the performance.
Intonation is somewhat accurate but consistently includes out-of-tune notes; these inaccuracies occasionally detract from the performance.
Intonation is consistently inaccurate and hinders the quality of performance.
Rhythmic Accuracy
Rhythms are accurate and precise throughout the performance.
Rhythms are nearly accurate; occasionally, rhythms lack precise interpretation.
Most rhythm patterns are accurate, but errors in precision are present and occasionally detract from the performance. Many rhythms performed incorrectly or inconsistently, major errors are present and detract from the performance. Pitch Accuracy (melodic accuracy)
Pitches are consistently correct. Occasional incorrect pitches are played, but do not detract from the performance
Most pitches are correct, but errors are present and occasionally detract from the performance.Many pitches are performed incorrectly; this inaccuracy consistently seriously hinders the performance.
Slide48
7. Evidence
of Student Achievement
Data Collection
Guiding Questions:
In what format will data be collected (e.g., database, graphed, portfolio, etc.)
Is a pre-post test being used? (If so, please describe.)
How frequently will data be collected?
Scores from the rubric will be collected on a database. Data will be collected at the mid-point of the first and fourth marking periods.Slide49
7. Evidence
of Student Achievement
Scoring Student
Progress/
Preparation
Guiding Questions:
How was baseline data collected? (If baseline data was not collected, please explain.)
Can baseline data be compared with the results of this assessment/performance task?
Sample Statements:
Grade
8 General Music
:
Baseline data is collected through a pretest of retained knowledge; there is no assessment of authentic skills prior to instruction.
HS Instrumental Music
:
Scoring from the first marking period assessment will be used as baseline data for describing growth. Scoring from the fourth marking period assessment will be used to describe growth and/or mastery.Slide50
7. Evidence
of Student Achievement
Data Presentation
Guiding Questions:
What evidence will be presented to principal/evaluator to support the teacher effectiveness measure?
How will data be presented to the principal/evaluator (e.g., database, graphed, portfolio, individual student artifacts, etc.)?
Sample Statements:
Grade 8 General Music
:
Evidence will be presented through aggregated scores from the database.
HS Instrumental Music
: An
ipad
database program created by the teacher will be used to present scoring information.Slide51
All Grades and Courses:
The assessment can be scored by an equivalent peer.
All Grades and Courses:
The assessment is a commercially produced assessment that has a history of validity and reliability.
Data Analysis and Interpretation
Guiding Questions:
How can the assessment/performance task results be interpreted in the same way across equivalent peers?
Is there a reliable and valid scoring and interpretive process (i.e., state developed, district-based, commercial, standardized, etc.) that is associated with the assessment/performance task? If so, please describe.
Sample Statements:
An
equivalent peer can use the rubrics to assess the projects.
7. Evidence
of Student AchievementSlide52
8. Strategies/Actions
to Achieve the SLO
Assessment for Learning
Guiding Questions:
What formative assessment information lets you know if your instructional practices will lead to successful completion of the SLO?
Sample Statements:
Grade
8
General Music
: Rubric will
be provided to the students as a part of progress monitoring throughout the instructional period.
HS Instrumental Music
: Interim formative
assessments during group lessons will be used to focus instruction toward successful completion of the SLO.Slide53
8. Strategies/Actions
to Achieve the SLO
Alignment with the Danielson Framework for Teaching
Guiding Questions:
Based upon reflection, what instructional practices would you like to change or strengthen?
What professional learning and/or other type of support will help you to achieve this SLO?
Sample Statements:
Grade
8 General Music
:
Based on the formative assessment information
provided by student composition and performance
tasks, students will be provided with supplemental materials to either address music reading difficulties or enhance current abilities to go beyond the targets. (Danielson 3E)
HS Instrumental
Music: Based on weekly formative performance assessments, student seating within the band will be uniquely rearranged periodically to provide students the opportunity to develop higher levels of performance independence. (Danielson 2 E)Slide54
Educator Effectiveness: SLO
SLO Resources
http://nassauboces.org/Page/1667
http://www.riseindiana.org/sites/default/files/files/RISE%201.0/Student%20Learning%20Objectives%20Handbook%201%200%20FINAL.pdf
http://www.ride.ri.gov/EducatorQuality/EducatorEvaluation/SLO.aspx
http://www.gadoe.org/School-Improvement/Teacher-and-Leader-Effectiveness/Documents/SLO%20Manual.pdf
Assessment Development Resource:
beta.ctcurriculum.orgSlide55
The Elective Portion of PA’s Educator Effectiveness Design
Student Learning Objectives: Understanding and Completing the SLO Template
An Instructional Webinar
Link:
http://vclass.cciu.org/play_recording.html?recordingId=1249208167245_1348758027178Slide56
Educator Effectiveness: SLO
Contact:
O. David
Deitz
SLO Project Lead, Educator Effectiveness
oddeitz@comcast.net
THANK YOU!