Is the Test Appropriate Useful and Meaningful Properties of Validity Does the test measure what it purports to measure Validity is a property of inferences that can be made Validity should be established over multiple inferences ID: 376897
Download Presentation The PPT/PDF document "Validity" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Validity
Is the Test Appropriate, Useful, and Meaningful?Slide2
Properties of Validity
Does the test measure what it purports to measure?
Validity is a property of inferences that can be made.
Validity should be established over multiple inferences.Slide3
Evidence of Validity
Meaning of test scores
Reliability
Adequate standardization and norming
Content validity
Criterion-related validity
Construct validitySlide4
Content Validity
How well does the test represent the domain?
Appropriateness of items.
Completeness of items.
How the items assess the content.Slide5
Do the Items Match the Content?
Face validity
Is the item considered part of the domain
Curriculum match
Do the items reflect what has been taught
Homogeneity with the test
Is the item positively correlated with the test score? (Minimum .25)
Point bi-serial r
xxSlide6
Are the Total Items Representative?
Are items included representing various parts of the domain?
Are different parts differentially represented?
How well are the parts represented?Slide7
How Is the Content Measured?
Different types of measures
Multiple choice, short answer
Performance-based vs. Factual recall
Measures should match the type of content taught
Measures should match HOW the content was taughtSlide8
Criterion-related (Cr)validity
How well does performance on the test reflect performance on what the test purports to measure?
Expressed as r
xx
Based on concurrent CR validity
And predictive CR validitySlide9
Concurrent Criterion-related Validity
How well does performance on the test estimate knowledge and/or skill on the criterion measure?
Does the reading test estimate the students current reading performance?
Compares performance on one test with performance with other similar tests.
KTEA w/ Woodcock-JohnsonSlide10
Predictive Criterion-related Validity
Will performance on a test now be predictive of performance at a later time?
Will a score derived now from one test be as accurate as a score derived by another and later test ?
Will a student’s current reading scores accurately reflect reading measured at a later time by another test?Slide11
Criterion-related Validity Should
Describe criterion measures accurately
Provide rationales for choices.
Provide sufficient information to judge adequacy of the criterion
Describe adequately the sample of students used.
Include analytical data used to determine predictivenessSlide12
Criterion-related Validity Should
Basic statistics should include
Number of cases
Reasons for eliminating cases
Central tendency estimates
Provide analysis of the limits toward generalizability of the test
What kind of inferences about the content can be madeSlide13
Construct Validity
How validly does the test measure the underlying constructs it purports to measure?
Do IQ tests measure intelligence?
Do self-concept scales measure self concept?Slide14
Definition of Construct
A psychological or personality trait
E.G. Intelligence, learning style
Or
A psychological concept, attribute, or theoretical characteristic
E.G. Problem solving, locus of control, or learning disabilitySlide15
Ways to Measure Construct Validity
Developmental change: determining expected differences among identified groups
Assuming content validity
Assuming reliability
Convergent /divergent validity: high correlation with similar tests and low correlation with tests measuring different constructsSlide16
Ways to measure Construct Validity
Predictive Validity: High scores on one test should predict high scores on similar tests.
Accumulation of evidence FAILING TO DISPROVE: If the concept or trait tested can be influenced by intervention, intervention effects should be reflected by pre and posttest scores.
If the test score should not be influenced by intervention, changes should not be reflected in pre and posttest scores.Slide17
Factors Affecting Validity
Unsystematic error
Lack of reliability
Systematic error
BiasSlide18
Reliability Effects on Validity
The validity of a measure can NEVER exceed the measure’s reliability
Reliability measures error
Validity measures expected traits (content, constructs, criteria)
r
xy
= r
x(t)y(t)
r
xx
r
yySlide19
Systematic Bias
Method of Measurement
Behaviors in testing
Item selection
Administration errors
NormsSlide20
Who is Responsible for Valid Assessment?
The Test Author
Authors are reponsible for ensuring and publishing validation.
The Test Giver
Test administrators are reponsible for following procedures outlined in administration guidelines.Slide21
Guidelines for Giving Tests
Exact Administration
Read the administration instructions.
Note procedures for establishing baselines.
Note procedures for individual items.
Practice giving the test.
Appropriate Pacing
Develop fluency with the test.