Can high-stakes state assessments be based on
Description: Can high-stakes state assessments be based on schools own assessments ? Should they be? Paul Black Kings College, London UK 2 19989 Task Group on Assessment and Testing report to Kenneth Baker recommendations about teachers role 14. The
Related Topics
Download Presentation
"Can high-stakes state assessments be based on" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Can high-stakes state assessments be based on schools’ own assessments ? Should they be? Paul Black King’s College, London UK<br>
slide2. 2 1998/9 Task Group on Assessment and Testing report to Kenneth Baker recommendations about teachers’ role 14. The national assessment system should be based on a combination of moderated teachers' ratings and standardised assessment tasks. (Paragraph 63)
15. Group moderation should be an integral part of the national assessment system. It should be used to produce the agreed combination of moderated teachers' ratings and the results of the national tests. (Paragraph 77)
17. The final reports on individual pupils to their parents should be the responsibility of the teacher, supported by standardised assessment tasks and group moderation. (Paragraph 80)<br>
slide3. 3 New Minister - new national assessment tasks (The original tasks ) were made a little too complicated and we have said we will simplify them. . . The complications themselves were largely designed in the first place in an attempt to pacify opponents who feared above all else 'paper and pencil' tests. . . This opposition to testing and examinations is largely based on a folk memory in the left about the old debate on the 11-plus and grammar schools. ( Clarke 1991)<br>
slide4. Margaret Thatcher’s opinion Ken Baker warmly welcomed the report. Whether he had read it properly I do not know: if he had it says much for his stamina. Certainly I had no opportunity to do so before agreeing to its publication . . .that it was then welcomed by the Labour party, the National Union of Teachers and the Times Educational Supplement was enough to confirm for me that its approach was suspect.
Pp.594-5 in THATCHER, M. (1993) The Downing Street Years (London, Harper Collins). 4<br>
slide5. 5 Purposes of assessment 1 Formative - develop pupils’ as learners
2 Summative for individuals
3 Accountability - of teachers, of schools, of nations
Synergies and tensions :
3 separate methods ?
or multi-purpose?<br>
slide6. 6 Validity Validity is achieved when
inferences that users of the results need to make
are justified by the evidence
The users are:
The student
Those who teach that student next year
Or in the next school
And all involved with the student after schooling completed<br>
slide7. 7 Our Present Situation
from RS report
Higher education, employers, and students themselves, are not given valid information
In our present national systems there are neither Opportunities nor Motivation for any agency to develop the quality of teachers’ summative assessments<br>
slide8. KOSAP King’s-Oxfordshire-Summative-Assessment Project
Investigate & strengthen teachers’ assessments to include open-ended tasks as well as formal tests
English & Mathematics departments
Year 8 (13 year olds)<br>
slide9. Participants and Activities 3 schools’ departments of maths. and science
12 teachers : 2 in each of the school’s departments
Formulating a number of assessment activities for Y8 students
Develop action plans in each school for broader summative assessments for all teachers and for all Y8 pupils in English and mathematics.<br>
slide10. Research Questions Audit the present summative assessment practices
How do the processes of moderation support teacher learning about students’ progress?
How can these valid assessment systems be implemented within departments ?<br>
slide11. Summative Assessment Purposes Decisions about teaching sets
Information for the next teacher
Reporting to Senior Management Team
Reporting to Parents
Target setting<br>
slide12. Problems in the current practice and culture The teacher’ existing practices lacked the rigour and uniformity required
There was a general acceptance of the tests and tasks that they already do, despite their concerns that these assessment tools may be unfair, invalid, and unreliable in measuring the capabilities of their students.
Uncritical attitude that the teachers had towards the task of making summative judgments<br>
slide13. Maths SA Practices Maths teachers used end-of-topic tests.
Derived questions from external examinations & textbooks
Teachers had not considered quality of tests nor how appropriate these tests where at assessing specific skills and conceptual understanding.<br>
slide14. 14 Validity of Assessments What does it mean to be good at - -- ? The project made me think more critically about what exactly I was assessing. The first question I remember being asked (‘what does it mean to be good at English?’) gave me a different perspective on assessment. I find myself continually returning to this question.
Teacher in King’s project<br>
slide15. Learning Through Investigations . . . at the end of units, they’d all be really nice investigations to do that would allow them to think about things that they’ve covered in class, but also, you know, be exposed to the investigations of how to do them, what to think about. Because I was quite surprised at the beginning how they couldn’t ... they didn’t know how to do it. Mathematics teacher 15<br>
slide16. Moderation Meetings . . . and we’ve had moderation meetings, we were together with the other schools, teachers in other schools looked at how rigorous our assessment would be and they criticised what, you know, our marking criteria (are). And we changed it, which is all being very positive .
Mathematics teacher 16<br>
slide17. Moderation: teaching and learning conversations I think its quite a healthy thing for a department to be doing because I think it will encourage people to have conversations and it’s about teaching and learning. . . . it really provides a discussion hopefully as well to talk about quality and you know what you think of was a success in English. Still really fundamental conversations.
Teacher in King’s project 17<br>
slide18. 18 Teachers’ Summative Assessment Confidence But I think if all the teachers had more, possibly more ownership of what we are actually doing in terms of summative assessment then you would have more confidence in saying to parents, which I think is one of the biggest things I find with lower school.
Mathematics teacher<br>
slide19. Moderation: Collaborative Professional Development Similarly, teachers who examined student data together and worked out as a group what its implications were for deciding how best to help those under-achieving, difficult-to-move students, had higher achieving students than those schools where such a collective examination, diagnosis and problem-solving cycle did not operate.
New Zealand : Parr and Timperley, 2008, p.69 19<br>
slide20. 20 Validity in the future? … the teacher is increasingly being seen as the primary assessor in the most important aspects of assessment. The broadening of assessment is based on a view that there are aspects of learning that are important but cannot be adequately assessed by formal external tests. These aspects require human judgment to integrate the many elements of performance behaviours that are required in dealing with authentic assessment tasks.
p.31 in Stanley, G., MacCann, R., Gardner, J., Reynolds, L. & Wild, I. (2009). Review of teacher assessment: what works best and issues for development. Oxford University Centre for Educational Development; Report commissioned by the QCA.<br>
slide21. 21 Two National Systems Queensland : all by moderated teacher assessment in local clusters of schools. State tests to inter-calibrate
Sweden: state tests fix the mean and spread of each school, teachers decide results of individual pupils<br>
slide22. 22 Royal Society 2014 Report : Vision
for science and mathematics education
Teacher assessment and moderation are more common and trusted components of assessment for publications (10 years time)
Teachers are responsible for assessment of public qualifications. Teacher assessment is trusted, understood and by parents, the public, and employers (20 years time).<br>
slide23. References 1 Assessment for learning
Black, P., Harrison, C., Lee, C., Marshall, B. & Wiliam, D, (2003) Assessment for Learning– putting it into practice. Buckingham: Open University Press.
Black, P. & Wiliam, D. (2009) Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
Summative Assessment studies
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2010) Validity in teachers’ summative assessments. Assessment in Education 17(2) 215-232.
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2011) Can teachers’ summative assessments produce dependable results and also enhance classroom learning? Assessment in Education. 18(4), 451-469.
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2013) Inside the Black Box of Assessment: Assessment of learning by teachers and schools. London: GL Assessment. In press
continued
con
con 23<br>
slide24. References 2 Black, P. (2013) Formative and Summative Aspects of Assessment: Theoretical and Research Foundations in the Context of Pedagogy. p.167-178 in McMillan, J.H. (ed.) Sage Handbook of Research on Classroom Assessment. In press
Klenowski, V. & Wyatt-Smith, C. (2013) Assessment for Education: A guide for Students, Teachers and Researchers. London: Sage. In Press.
Self Theories and Mind-Set
Dweck, C. S. (2000). Self-theories: their role in motivation, personality and development. Philadelphia, PA: Psychology Press.
Dweck, C. S. (2006) Mindset: the new psychology of success. New York:Random House.
Group work
Lyn Daws, Neil Mercer and Rupert Wegerif (2003) Thinking Together. Published by Imaginative Minds Ltd.
Ed Baines, Peter Blatchford and Peter Kutnick (2009) Promoting Effective Group Work in the Primary Classroom London: Routledge 24<br>
slide2. 2 1998/9 Task Group on Assessment and Testing report to Kenneth Baker recommendations about teachers’ role 14. The national assessment system should be based on a combination of moderated teachers' ratings and standardised assessment tasks. (Paragraph 63)
15. Group moderation should be an integral part of the national assessment system. It should be used to produce the agreed combination of moderated teachers' ratings and the results of the national tests. (Paragraph 77)
17. The final reports on individual pupils to their parents should be the responsibility of the teacher, supported by standardised assessment tasks and group moderation. (Paragraph 80)<br>
slide3. 3 New Minister - new national assessment tasks (The original tasks ) were made a little too complicated and we have said we will simplify them. . . The complications themselves were largely designed in the first place in an attempt to pacify opponents who feared above all else 'paper and pencil' tests. . . This opposition to testing and examinations is largely based on a folk memory in the left about the old debate on the 11-plus and grammar schools. ( Clarke 1991)<br>
slide4. Margaret Thatcher’s opinion Ken Baker warmly welcomed the report. Whether he had read it properly I do not know: if he had it says much for his stamina. Certainly I had no opportunity to do so before agreeing to its publication . . .that it was then welcomed by the Labour party, the National Union of Teachers and the Times Educational Supplement was enough to confirm for me that its approach was suspect.
Pp.594-5 in THATCHER, M. (1993) The Downing Street Years (London, Harper Collins). 4<br>
slide5. 5 Purposes of assessment 1 Formative - develop pupils’ as learners
2 Summative for individuals
3 Accountability - of teachers, of schools, of nations
Synergies and tensions :
3 separate methods ?
or multi-purpose?<br>
slide6. 6 Validity Validity is achieved when
inferences that users of the results need to make
are justified by the evidence
The users are:
The student
Those who teach that student next year
Or in the next school
And all involved with the student after schooling completed<br>
slide7. 7 Our Present Situation
from RS report
Higher education, employers, and students themselves, are not given valid information
In our present national systems there are neither Opportunities nor Motivation for any agency to develop the quality of teachers’ summative assessments<br>
slide8. KOSAP King’s-Oxfordshire-Summative-Assessment Project
Investigate & strengthen teachers’ assessments to include open-ended tasks as well as formal tests
English & Mathematics departments
Year 8 (13 year olds)<br>
slide9. Participants and Activities 3 schools’ departments of maths. and science
12 teachers : 2 in each of the school’s departments
Formulating a number of assessment activities for Y8 students
Develop action plans in each school for broader summative assessments for all teachers and for all Y8 pupils in English and mathematics.<br>
slide10. Research Questions Audit the present summative assessment practices
How do the processes of moderation support teacher learning about students’ progress?
How can these valid assessment systems be implemented within departments ?<br>
slide11. Summative Assessment Purposes Decisions about teaching sets
Information for the next teacher
Reporting to Senior Management Team
Reporting to Parents
Target setting<br>
slide12. Problems in the current practice and culture The teacher’ existing practices lacked the rigour and uniformity required
There was a general acceptance of the tests and tasks that they already do, despite their concerns that these assessment tools may be unfair, invalid, and unreliable in measuring the capabilities of their students.
Uncritical attitude that the teachers had towards the task of making summative judgments<br>
slide13. Maths SA Practices Maths teachers used end-of-topic tests.
Derived questions from external examinations & textbooks
Teachers had not considered quality of tests nor how appropriate these tests where at assessing specific skills and conceptual understanding.<br>
slide14. 14 Validity of Assessments What does it mean to be good at - -- ? The project made me think more critically about what exactly I was assessing. The first question I remember being asked (‘what does it mean to be good at English?’) gave me a different perspective on assessment. I find myself continually returning to this question.
Teacher in King’s project<br>
slide15. Learning Through Investigations . . . at the end of units, they’d all be really nice investigations to do that would allow them to think about things that they’ve covered in class, but also, you know, be exposed to the investigations of how to do them, what to think about. Because I was quite surprised at the beginning how they couldn’t ... they didn’t know how to do it. Mathematics teacher 15<br>
slide16. Moderation Meetings . . . and we’ve had moderation meetings, we were together with the other schools, teachers in other schools looked at how rigorous our assessment would be and they criticised what, you know, our marking criteria (are). And we changed it, which is all being very positive .
Mathematics teacher 16<br>
slide17. Moderation: teaching and learning conversations I think its quite a healthy thing for a department to be doing because I think it will encourage people to have conversations and it’s about teaching and learning. . . . it really provides a discussion hopefully as well to talk about quality and you know what you think of was a success in English. Still really fundamental conversations.
Teacher in King’s project 17<br>
slide18. 18 Teachers’ Summative Assessment Confidence But I think if all the teachers had more, possibly more ownership of what we are actually doing in terms of summative assessment then you would have more confidence in saying to parents, which I think is one of the biggest things I find with lower school.
Mathematics teacher<br>
slide19. Moderation: Collaborative Professional Development Similarly, teachers who examined student data together and worked out as a group what its implications were for deciding how best to help those under-achieving, difficult-to-move students, had higher achieving students than those schools where such a collective examination, diagnosis and problem-solving cycle did not operate.
New Zealand : Parr and Timperley, 2008, p.69 19<br>
slide20. 20 Validity in the future? … the teacher is increasingly being seen as the primary assessor in the most important aspects of assessment. The broadening of assessment is based on a view that there are aspects of learning that are important but cannot be adequately assessed by formal external tests. These aspects require human judgment to integrate the many elements of performance behaviours that are required in dealing with authentic assessment tasks.
p.31 in Stanley, G., MacCann, R., Gardner, J., Reynolds, L. & Wild, I. (2009). Review of teacher assessment: what works best and issues for development. Oxford University Centre for Educational Development; Report commissioned by the QCA.<br>
slide21. 21 Two National Systems Queensland : all by moderated teacher assessment in local clusters of schools. State tests to inter-calibrate
Sweden: state tests fix the mean and spread of each school, teachers decide results of individual pupils<br>
slide22. 22 Royal Society 2014 Report : Vision
for science and mathematics education
Teacher assessment and moderation are more common and trusted components of assessment for publications (10 years time)
Teachers are responsible for assessment of public qualifications. Teacher assessment is trusted, understood and by parents, the public, and employers (20 years time).<br>
slide23. References 1 Assessment for learning
Black, P., Harrison, C., Lee, C., Marshall, B. & Wiliam, D, (2003) Assessment for Learning– putting it into practice. Buckingham: Open University Press.
Black, P. & Wiliam, D. (2009) Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
Summative Assessment studies
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2010) Validity in teachers’ summative assessments. Assessment in Education 17(2) 215-232.
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2011) Can teachers’ summative assessments produce dependable results and also enhance classroom learning? Assessment in Education. 18(4), 451-469.
Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N. (2013) Inside the Black Box of Assessment: Assessment of learning by teachers and schools. London: GL Assessment. In press
continued
con
con 23<br>
slide24. References 2 Black, P. (2013) Formative and Summative Aspects of Assessment: Theoretical and Research Foundations in the Context of Pedagogy. p.167-178 in McMillan, J.H. (ed.) Sage Handbook of Research on Classroom Assessment. In press
Klenowski, V. & Wyatt-Smith, C. (2013) Assessment for Education: A guide for Students, Teachers and Researchers. London: Sage. In Press.
Self Theories and Mind-Set
Dweck, C. S. (2000). Self-theories: their role in motivation, personality and development. Philadelphia, PA: Psychology Press.
Dweck, C. S. (2006) Mindset: the new psychology of success. New York:Random House.
Group work
Lyn Daws, Neil Mercer and Rupert Wegerif (2003) Thinking Together. Published by Imaginative Minds Ltd.
Ed Baines, Peter Blatchford and Peter Kutnick (2009) Promoting Effective Group Work in the Primary Classroom London: Routledge 24<br>