Item Analysis What makes a question good??? Answer
Description: Item Analysis What makes a question good??? Answer options? Multiple Choice Questions Elements of a good multiple choice question???? Item Analysis How can we determine if it is a good question????? Distractor power Item difficulty Item
Related Topics
Download Presentation
"Item Analysis What makes a question good??? Answer" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Item Analysis What makes a question good???
Answer options?<br>
slide2. Multiple Choice Questions Elements of a good multiple choice question????<br>
slide3. Item Analysis How can we determine if it is a good question?????
Distractor power
Item difficulty
Item discrimination<br>
slide4. Distractor power Strength of each distractor (should be equal)
# answering incorrect
# of distractors<br>
slide5. E.G. There are 4 possible answers
Option a is correct
if 9 people choose option b (9/3 = 3)
and 12 people choose option c (12/3 = 4)
and 15 people choose option d (15/3 = 5)
then option d is the strongest distractor<br>
slide6. Item Difficulty = p (percent of people passing) p = number answering correctly
total number answering
If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question?
p1 = 34/100 = .34
For 65 correct p would be 65/100 = .65
SO… the higher the p, the easier the question
(score must be between 0 and 1)<br>
slide7. Item Difficulty is... A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparison<br>
slide8. In construct based tests Item difficulty is similar to item endorsement
Number agreeing with an item<br>
slide9. Item endorsement also... A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparison<br>
slide10. Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly?
(or on a psychological test, those scoring high or low on the construct)<br>
slide11. Distribution To determine level
upper (U) = top 25 to 33 % of scores
lower (L) = bottom 25 to 33 % of scores<br>
slide12. Determining item discrimination d = U-L
n
where d = item discrimination
U = number in the upper group who get question correct
L = number in the lower group who get question correct
n = number in one group (U or L)<br>
slide13. Examples of d With 33 in U and L
if all in U and all in L get it correct
33 - 33 / 33 = 0
if all in U get it correct, and none in L
33 - 0 / 33 = 1
if none in U get it correct, and all in L
0 - 33 / 33 = -1
if 30 in U and 10 in L get it correct
30 - 10 / 33 = .61<br>
slide14. Value of d Range for d = -1 to 1
higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group)
negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)<br>
Answer options?<br>
slide2. Multiple Choice Questions Elements of a good multiple choice question????<br>
slide3. Item Analysis How can we determine if it is a good question?????
Distractor power
Item difficulty
Item discrimination<br>
slide4. Distractor power Strength of each distractor (should be equal)
# answering incorrect
# of distractors<br>
slide5. E.G. There are 4 possible answers
Option a is correct
if 9 people choose option b (9/3 = 3)
and 12 people choose option c (12/3 = 4)
and 15 people choose option d (15/3 = 5)
then option d is the strongest distractor<br>
slide6. Item Difficulty = p (percent of people passing) p = number answering correctly
total number answering
If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question?
p1 = 34/100 = .34
For 65 correct p would be 65/100 = .65
SO… the higher the p, the easier the question
(score must be between 0 and 1)<br>
slide7. Item Difficulty is... A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparison<br>
slide8. In construct based tests Item difficulty is similar to item endorsement
Number agreeing with an item<br>
slide9. Item endorsement also... A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparison<br>
slide10. Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly?
(or on a psychological test, those scoring high or low on the construct)<br>
slide11. Distribution To determine level
upper (U) = top 25 to 33 % of scores
lower (L) = bottom 25 to 33 % of scores<br>
slide12. Determining item discrimination d = U-L
n
where d = item discrimination
U = number in the upper group who get question correct
L = number in the lower group who get question correct
n = number in one group (U or L)<br>
slide13. Examples of d With 33 in U and L
if all in U and all in L get it correct
33 - 33 / 33 = 0
if all in U get it correct, and none in L
33 - 0 / 33 = 1
if none in U get it correct, and all in L
0 - 33 / 33 = -1
if 30 in U and 10 in L get it correct
30 - 10 / 33 = .61<br>
slide14. Value of d Range for d = -1 to 1
higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group)
negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)<br>