Kumpulan 7 SAMLINAH BINTI JUPILEH 262400 NUR ALIA
Description: Kumpulan 7 SAMLINAH BINTI JUPILEH 262400 NUR ALIA BINTI MD RASHID 262704 SITI SOLEHAH BINTI ZULKEFLE 264366 RABBANIYAH BINTI MOHAMED 264387 ITEM ANALYSIS TOPIC OUTLINE In this slide, we will cover about Item Analysis in terms of: What is
Related Topics
Download Presentation
"Kumpulan 7 SAMLINAH BINTI JUPILEH 262400 NUR ALIA" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Kumpulan 7 SAMLINAH BINTI JUPILEH 262400
NUR ALIA BINTI MD RASHID 262704
SITI SOLEHAH BINTI ZULKEFLE 264366
RABBANIYAH BINTI MOHAMED 264387 ITEM ANALYSIS<br>
slide2. TOPIC OUTLINE In this slide, we will cover about Item Analysis in terms of:
What is Item Analysis
Calculate, interpret and compare item difficulty index
Calculate, interpret and compare item discrimination index
Distractor analysis
Improving the test #1<br>
slide3. ITEM ANALYSIS Item analysis is an examination of a test after its administration (Remmers et al., 1967). The examination of individual items on a test, rather than the test as a whole, for its difficulty, appropriateness, relationship to the rest of the test and etc. Item analysis is useful in helping test designers to determine which items to keep, modify or discard on a given test and how to finalize the score for a student. #2<br>
slide4. CONT. There two (2) methods to conduct Item Analysis:
We are going to present this topic by using method (1). 1) Using two groups of students which is high achieving group vs low achieving group by using their responses on the question and calculate using formula (manually or Excel). 2) Using responses of the question from All students and calculate using formula (SPSS or Excel). #3<br>
slide5. CONCEPT OF ITEM DIFFICULTY INDEX VS ITEM DISCRIMINATION INDEX ITEM DIFFICULTY INDEX
The proportion of test takers who answer an item correctly.
The bigger the value of the difficulty index, the easier the item, and vice versa.
ITEM DISCRIMINATION INDEX
The ability of an item to differentiate among students based on how well they know the material being tested.
This index is the equivalent to a Point-biserial Coefficient (Item-total Correlation Coefficients). #4<br>
slide6. ITEM DIFFICULTY INDEX Difficulty Index (Dif I) or Facility value or p value
Using the formula p = [H + L] / N #5 H = number of students answering the item correctly in the high achieving group
L = number of students answering the item correctly in the low achieving group
N = Total number of students in the two groups (including non-responders)<br>
slide7. CONT. Difficulty index describes the proportion of students who answered the item correctly.
It ranges between 0 and 1 for dichotomous item and 0 to J for polytomous item, where J = maximum score point allocated.
In general, items with a p value between .3 to .7 are considered as acceptable. Amongst these, items with p value between .5 to .6 are ideal (depending on the number of options for MCQ item).
In general, items with p value less than .3 (too difficult) and more than .7 (too easy) are not acceptable and need modification. #6<br>
slide8. ITEM DISCRIMINATION INDEX A calculation that determines the difference between test takers who score well and who score poorly on the test.
The discrimination index range is between -1 and +1.
The closer the index is to +1, the more effectively the item distinguishes between the two group of students. #7<br>
slide9. GUIDELINES FOR EVALUATING D VALUES #8<br>
slide10. FORMULA OF ITEM DISCRIMINATION INDEX D = (U correct – L correct)
U Expressed as a decimal fraction D = Discrimination Index
U correct = Total Number of Upper Group Students Answering Correctly
L correct= Total Number of Lower Group Students Answering Correctly
U = Total Number of Upper Group Students #9<br>
slide11. item a (low difficulty, moderate discrimination) For example : Item A using formula
for item difficulty index
H = 12 , L =10, N = 30 p = [H + L] / N p = [12 + 10] / 30
p = 0.73 For example : Item A using formula
for item difficulty index
H = 12 , L = 10 , N = 30 D = (U correct – L correct)
U D = (12 – 10)
15
D = 0.13 The answer is C #10<br>
slide12. item b ( moderate difficulty, high discrimination ) The answer is C P = 0.50
D = 0.33 #11 #TIME FOR SOME EXERCISES#<br>
slide13. item c (moderate difficulty, low discrimination) The answer is D P = 0.37
D = 0.07 #12<br>
slide14. item d ( high difficulty, moderate discrimination) The answer is D P = 0.23
D = 0.20 #13<br>
slide15. DISTRACTOR ANALYSIS Distractor analysis is an extension of item analysis, using techniques that are similar to item difficulty and item discrimination.
A distractor must be plausible but incorrect and originate from the text.
If there is only one best response, then all other response options are distractors.
A perfect test item would have 2 characteristics:
Everyone who knows the item gets it right
People who do not know the item will have responses equally distributed across the wrong answers. #14<br>
slide16. example item a STEM OPTION DISTRACTOR #15<br>
slide17. When the number of persons choosing a distractor significantly exceeds the number expected, there are two possibility:
It is possible that the choice reflects partial knowledge
The item is a poorly worded trick question
Unpopular distractors may lower item and test difficulty because it is easily eliminated
Extremely popular is likely to lower the reliability and validity of the test. #16<br>
slide18. EXAMPLE ITEM B: p = [H + L] / N D = (U correct – L correct)
U #17<br>
slide19. ANALYSIS DISTRACTOR ITEM B 9 students from low achieving group choose distractor A (RM 179.20) because they directly multiplied RM 12.80 with 14 (12.80 x 4) before dividing RM 12.80 with 4. The correct solution is [(12.80/4) x14].
Only two serves as the good distractors which is distractor A and D.
No one choose distractor B as his/her answer.
Distractor B can be changed to RM 230.40 in which low ability students conduct this operation (14 + 4) x 12.8 and Distractor D can be changed into RM 716.8 in which low ability students might incorrectly multiply 4 with RM 12.8 instead of dividing it before multiplying again with 14 (4 x 12.8 x 14).
p is 0.50, which means that the item has moderate difficulty (not too difficult, not too easy)
D is 0.33. It shows that this item has high discrimination because of total D is more than 0.30.
In conclusion, the item is good enough and should be retained in the test. This item has moderate difficulty and high discrimination. #18<br>
slide20. Improving the test Teacher can decide to drop, modify or keep a question depends on the p-value (difficulty index) and D (discrimination index)of the item.
p-value is to make sure the difficulty is within the range.
If the item is in moderate range (p-value), but the discrimination index (D) is close to zero or negative, the item should be revised.
Distractor analysis is also important because it gives information about how many students choose a particular answer and it will show whether the distractor is good or bad. If it is bad or too hard, teacher might replace the distractor. #19<br>
slide21. If the difficulty index is high but the discrimination index is negative (more student in lower group get the right answer) it indicates that the item is problematic.
Items with very low (or zero) discriminations can be removed.
Substantial negative discrimination usually indicates an error in coding. Item with negative discrimination need to be revise.
The need to improve item is to make sure that item used in a test have quality and can achieve the target of the assessment. #20<br>
slide22. To decide if an item should be revise or not, item writer should consider the p-value and D. For example, p = [H + L] / N D = (U correct – L correct)
U -item is quite hard -low D which indicates the
ability to discriminate is low. #21<br>
slide23. summary Item difficulty is about the ability to know the level of a question based on what percentage / number of students from each grade level answered that question correctly.
item discrimination is the ability of this item to discriminate between students based on their understanding of the questions / items given to them during the test / assessment.
Distractor analysis is an extension of item analysis, using techniques that are similar to item difficulty and item discrimination.
Item difficulty, item discrimination and distractor analysis give information about students traits and that information can be used to improve the item. #22<br>
slide24. REFERENCES https://www.slideshare.net/RonaldQuileste/item-analysis-discrimination-and-difficulty-index
https://languagetestitemanalysis.weebly.com/content/item-discrimination
https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/#:~:text=Item%20discrimination%20refers%20to%20the,low%20scoring%20groups%20of%20students. #23<br>
NUR ALIA BINTI MD RASHID 262704
SITI SOLEHAH BINTI ZULKEFLE 264366
RABBANIYAH BINTI MOHAMED 264387 ITEM ANALYSIS<br>
slide2. TOPIC OUTLINE In this slide, we will cover about Item Analysis in terms of:
What is Item Analysis
Calculate, interpret and compare item difficulty index
Calculate, interpret and compare item discrimination index
Distractor analysis
Improving the test #1<br>
slide3. ITEM ANALYSIS Item analysis is an examination of a test after its administration (Remmers et al., 1967). The examination of individual items on a test, rather than the test as a whole, for its difficulty, appropriateness, relationship to the rest of the test and etc. Item analysis is useful in helping test designers to determine which items to keep, modify or discard on a given test and how to finalize the score for a student. #2<br>
slide4. CONT. There two (2) methods to conduct Item Analysis:
We are going to present this topic by using method (1). 1) Using two groups of students which is high achieving group vs low achieving group by using their responses on the question and calculate using formula (manually or Excel). 2) Using responses of the question from All students and calculate using formula (SPSS or Excel). #3<br>
slide5. CONCEPT OF ITEM DIFFICULTY INDEX VS ITEM DISCRIMINATION INDEX ITEM DIFFICULTY INDEX
The proportion of test takers who answer an item correctly.
The bigger the value of the difficulty index, the easier the item, and vice versa.
ITEM DISCRIMINATION INDEX
The ability of an item to differentiate among students based on how well they know the material being tested.
This index is the equivalent to a Point-biserial Coefficient (Item-total Correlation Coefficients). #4<br>
slide6. ITEM DIFFICULTY INDEX Difficulty Index (Dif I) or Facility value or p value
Using the formula p = [H + L] / N #5 H = number of students answering the item correctly in the high achieving group
L = number of students answering the item correctly in the low achieving group
N = Total number of students in the two groups (including non-responders)<br>
slide7. CONT. Difficulty index describes the proportion of students who answered the item correctly.
It ranges between 0 and 1 for dichotomous item and 0 to J for polytomous item, where J = maximum score point allocated.
In general, items with a p value between .3 to .7 are considered as acceptable. Amongst these, items with p value between .5 to .6 are ideal (depending on the number of options for MCQ item).
In general, items with p value less than .3 (too difficult) and more than .7 (too easy) are not acceptable and need modification. #6<br>
slide8. ITEM DISCRIMINATION INDEX A calculation that determines the difference between test takers who score well and who score poorly on the test.
The discrimination index range is between -1 and +1.
The closer the index is to +1, the more effectively the item distinguishes between the two group of students. #7<br>
slide9. GUIDELINES FOR EVALUATING D VALUES #8<br>
slide10. FORMULA OF ITEM DISCRIMINATION INDEX D = (U correct – L correct)
U Expressed as a decimal fraction D = Discrimination Index
U correct = Total Number of Upper Group Students Answering Correctly
L correct= Total Number of Lower Group Students Answering Correctly
U = Total Number of Upper Group Students #9<br>
slide11. item a (low difficulty, moderate discrimination) For example : Item A using formula
for item difficulty index
H = 12 , L =10, N = 30 p = [H + L] / N p = [12 + 10] / 30
p = 0.73 For example : Item A using formula
for item difficulty index
H = 12 , L = 10 , N = 30 D = (U correct – L correct)
U D = (12 – 10)
15
D = 0.13 The answer is C #10<br>
slide12. item b ( moderate difficulty, high discrimination ) The answer is C P = 0.50
D = 0.33 #11 #TIME FOR SOME EXERCISES#<br>
slide13. item c (moderate difficulty, low discrimination) The answer is D P = 0.37
D = 0.07 #12<br>
slide14. item d ( high difficulty, moderate discrimination) The answer is D P = 0.23
D = 0.20 #13<br>
slide15. DISTRACTOR ANALYSIS Distractor analysis is an extension of item analysis, using techniques that are similar to item difficulty and item discrimination.
A distractor must be plausible but incorrect and originate from the text.
If there is only one best response, then all other response options are distractors.
A perfect test item would have 2 characteristics:
Everyone who knows the item gets it right
People who do not know the item will have responses equally distributed across the wrong answers. #14<br>
slide16. example item a STEM OPTION DISTRACTOR #15<br>
slide17. When the number of persons choosing a distractor significantly exceeds the number expected, there are two possibility:
It is possible that the choice reflects partial knowledge
The item is a poorly worded trick question
Unpopular distractors may lower item and test difficulty because it is easily eliminated
Extremely popular is likely to lower the reliability and validity of the test. #16<br>
slide18. EXAMPLE ITEM B: p = [H + L] / N D = (U correct – L correct)
U #17<br>
slide19. ANALYSIS DISTRACTOR ITEM B 9 students from low achieving group choose distractor A (RM 179.20) because they directly multiplied RM 12.80 with 14 (12.80 x 4) before dividing RM 12.80 with 4. The correct solution is [(12.80/4) x14].
Only two serves as the good distractors which is distractor A and D.
No one choose distractor B as his/her answer.
Distractor B can be changed to RM 230.40 in which low ability students conduct this operation (14 + 4) x 12.8 and Distractor D can be changed into RM 716.8 in which low ability students might incorrectly multiply 4 with RM 12.8 instead of dividing it before multiplying again with 14 (4 x 12.8 x 14).
p is 0.50, which means that the item has moderate difficulty (not too difficult, not too easy)
D is 0.33. It shows that this item has high discrimination because of total D is more than 0.30.
In conclusion, the item is good enough and should be retained in the test. This item has moderate difficulty and high discrimination. #18<br>
slide20. Improving the test Teacher can decide to drop, modify or keep a question depends on the p-value (difficulty index) and D (discrimination index)of the item.
p-value is to make sure the difficulty is within the range.
If the item is in moderate range (p-value), but the discrimination index (D) is close to zero or negative, the item should be revised.
Distractor analysis is also important because it gives information about how many students choose a particular answer and it will show whether the distractor is good or bad. If it is bad or too hard, teacher might replace the distractor. #19<br>
slide21. If the difficulty index is high but the discrimination index is negative (more student in lower group get the right answer) it indicates that the item is problematic.
Items with very low (or zero) discriminations can be removed.
Substantial negative discrimination usually indicates an error in coding. Item with negative discrimination need to be revise.
The need to improve item is to make sure that item used in a test have quality and can achieve the target of the assessment. #20<br>
slide22. To decide if an item should be revise or not, item writer should consider the p-value and D. For example, p = [H + L] / N D = (U correct – L correct)
U -item is quite hard -low D which indicates the
ability to discriminate is low. #21<br>
slide23. summary Item difficulty is about the ability to know the level of a question based on what percentage / number of students from each grade level answered that question correctly.
item discrimination is the ability of this item to discriminate between students based on their understanding of the questions / items given to them during the test / assessment.
Distractor analysis is an extension of item analysis, using techniques that are similar to item difficulty and item discrimination.
Item difficulty, item discrimination and distractor analysis give information about students traits and that information can be used to improve the item. #22<br>
slide24. REFERENCES https://www.slideshare.net/RonaldQuileste/item-analysis-discrimination-and-difficulty-index
https://languagetestitemanalysis.weebly.com/content/item-discrimination
https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/#:~:text=Item%20discrimination%20refers%20to%20the,low%20scoring%20groups%20of%20students. #23<br>