Answer options Multiple Choice Questions Elements of a good multiple choice question Item Analysis How can we determine if it is a good question Distractor power Item difficulty Item discrimination ID: 630871
Download Presentation The PPT/PDF document "Item Analysis What makes a question good..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Item Analysis
What makes a question good???
Answer options?Slide2
Multiple Choice Questions
Elements of a good multiple
choice question????Slide3
Item Analysis
How can we determine if it is a good question?????
Distractor
power
Item difficulty
Item discriminationSlide4
Distractor power
Strength of each distractor (should be equal)
# answering incorrect
# of distractorsSlide5
E.G.
There are 4 possible answers
Option a is correct
if 9 people choose option b (9/3 = 3)
and 12 people choose option c (12/3 = 4)
and 15 people choose option d (15/3 = 5)
then option d is the strongest
distractorSlide6
Item Difficulty = p
(percent of people passing)
p
=
number answering correctly
total number answering
If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question?
p
1
=
34/100 = .34
For 65 correct p would be 65/100 = .65
SO… the higher the p, the easier the question
(score must be between 0 and 1)Slide7
Item Difficulty is...
A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparisonSlide8
In construct based tests
Item difficulty is similar to
item endorsement
Number agreeing with an itemSlide9
Item endorsement also...
A behavioral measure
Based on the current group
Allows internal comparison
Also allows external comparisonSlide10
Item discrimination = d
Does the question discriminate between those who do well overall and those who do poorly?
(or on a psychological test, those scoring high or low on the construct)Slide11
Distribution
To determine level
upper (U) = top 25 to 33 % of scores
lower (L) = bottom 25 to 33 % of scoresSlide12
Determining item discrimination
d =
U-L
n
where d = item discrimination
U = number in the upper group who
get question correct
L = number in the lower group who
get
question correct
n = number in one group (U or L)Slide13
Examples of d
With 33 in U and L
if all in U and all in L get it correct
33 - 33 / 33 = 0
if all in U get it correct, and none in L
33 - 0 / 33 = 1
if none in U get it correct, and all
in L
0 - 33 / 33 = -1
if 30 in U and 10 in L get it correct
30 - 10 / 33 = .61Slide14
Value of d
Range for d = -1 to 1
higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group)
negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)Slide15Slide16