Validity and testing for validity Reliability and testing for reliability Group task for today Bias is a systematic error in the design conduct or analysis of a study that results in errors when calculating measures of association between risk factors and outcomes ID: 776689
Download Presentation The PPT/PDF document " OUTLINE OF PRESENTATION Systematic(bias..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
OUTLINE OF PRESENTATION
Systematic(bias)error in epidemiological studies
Validity and testing for validity
Reliability and testing for reliability
Group task for today
Slide2Bias is a
systematic
error in the design, conduct, or analysis of a study that results in errors when calculating measures of association between risk factors and outcomes
It is thus always one-sided, either underestimating or over-estimating risk
It is not usually a subjective error, and is often unavoidable, but must be recognised as a limitation of the study to allow for correct interpretation
Slide3Types
of bias in epidemiological studies
Selection
bias: In
selection bias study participants are selected in one locality of convenience and are therefore not representative of the target population
e.g. Selection
of a sample for high prevalence of alcoholic foetal syndrome in western cape is not representative of prevalence of alcoholic foetal syndrome in
SA.
Information
bias /Recall
bias:
Recall
bias result because those affected by a disease often recall their exposures better than those who are unaffected
e.g. Those
who were exposed to polio virus will remember better than those who were
unexposed
Non-response
bias:
leads
to a lack of information on non-responders. Thus if the non-response rate is high, the study may be severely biased. Sometimes called volunteer bias.
Misclassification bias:
Slide4Validity of a test
The
validity of a test is defined as its ability to distinguish between those who have the disease(Sensitivity )and those who do not have the disease(Specificity).
Factors
affecting validity
:
Incorrect calibration and or uncalibration of the measuring device e.g. Sphygmomanometer(internal validity)
Unstructured interviews or questionnaires in a study (internal validity
Unrepresentativeness of the sample from the population(external validity.
Components of validity
Validity of test has therefore two components namely sensitivity and specificity.
Sensitivity
Sensitivity of a test is defined as the ability of a test to identify
correctly
those who have the disease
S
pecificity
.
Specificity of a test is defined as the ability of a test to identify
correctly
those who do not have the disease.
True characteristic in the population
Test Have the Do not have Result disease the disease Positive True positive(TP) False Positive(FP) Have the disease Do not have the disease and test positive but test positive False negative(FN) True Negative(TN) Negative Have the but Do not have the Test negative and test negative Sensitivity = TP Specificity = TN TP +FN TN+FP
Slide6Types of validity testsGold standard testScreening testGold standard test refers to a diagnostic test or benchmark that is the best available under reasonable conditions. It is referred to as the most accurate test possible without restrictionsA hypothetical ideal of gold standard test has a sensitivity of 100% with respect to the presence of the disease (it identifies all individuals who have the disease; it does not have any false-negative results) Sensitivity = a OR TP a +c TP +FN
Slide7Also
a hypothetical ideal of gold standard test has a specificity of 100% with respect to the absence of disease (it identifies all individuals who do not have the disease.it does not have any false-positive results). Specificity = d OR TN d +b TP +FP
Slide8Example:
Suppose we have a hypothetical population of 1000 people, the gold standard will yield a sensitivity of 500 with respect to the presence of disease with no false negative test results and 500 with respect to the absence of disease with no false positive test results.
See below as shown.
Slide9True characteristics in the population
Test Have the Do not Results disease have the disease Positive 500 0 Negative 0 500 Total 500 500 Sensitivity= 500 Specificity= 500 500+0 500+0 = 100% = 100% In practice, there are sometimes no true "gold standard" tests.
Slide10SCREENING VALIDITY TEST
Screenings tests are tests that look for diseases before you have symptoms. Screening tests can detect diseases early and thus start treatment Detectable Preclinical Clinical phase phase ClinicalNo disease outcome Symptoms disease Therapy Biologic first diagnostic given onset of appear test disease Disease detectable by screening
Slide11Hypothetical example:
Suppose we have a hypothetical population of 1000 people of whom 100 have a certain disease and 900 do not. In this instance we use a screening test to distinguish persons with the disease from those who do not.
The results obtain by applying the test to this population of 1000 people are shown below
Slide12True
characteristics in the population Test Have the Do not Results disease have the disease Positive 80 100 Negative 20 800 Total 100 900 Sensitivity= 80 Specificity= 800 80 +20 800 +100 = 80% = 89%
Slide13The question we now ask is how good is the screening test compared to the gold standard.
First, the test indicates that of the 100 people with the disease,80 were correctly identified as positive and a positive identification was in 20 and thus the sensitivity of the test which is defined as the proportion of diseased people were correctly identified as positive by the test is 80/100 or 80%
Secondly, the test indicates that of the 900 people who did not have the ,the test correctly identified 800 people as negative and thus the specificity f the test, which is defined as the proportion of non diseased people who are correctly identified as negative by the test is therefore 800/900 or 89%
Slide14Why is the issue of false positive important?
The issue of false positive is important because:
All people who screened positive are brought back for sophisticated and more expensive test which becomes a burden on the health care system.
Another is the anxiety and worry induced in persons who have been told that they tested positive and this labelling is not completely erased even if the results of subsequent evaluation are negative.
Slide15Why is the issue of false
negative important
?
The issue of false positive is important because
:
If for example, the disease is a type of cancer that is curable only in its early stages, a false negative result could represent a virtual death sentence.
Thus the result depends on the nature of the severity of the disease being screened for effectiveness of available intervention measure and whether the effeteness is greater if the intervention is administered early in the natural history of the disease
Slide16Predictive value of a test
We have so far asked the question "
How good is the
test(gold
std
)
at identifying people with the disease and people without the disease”.
Put in another way "If we screen a population, what proportion of people who have the disease will be correctly identified?”
These question is clearly for
public health consideration.
Positive predictive value of a test
In the clinical setting, a different question for the clinician would be "
If the test results are positive in this patient, what is the probability that this patient has the disease
"This is called the positive predictive value(PPV) In other words the question is “what proportion of patients who test positive actually have the disease in question?
Slide17Negative predictive
value of a
test
Also a parallel question can be asked about negative test results as follows
“
If the test results is negative ,what is the probability that this patient does not have the disease "This is called negative predictive
value(NPV) of the test.
Slide18Calculation of both PPV and NPV
PPV of a test is calculated by dividing the number of true positive(TP) by the total number of people who tested positive, i.e. TP divide by TP plus FP
NPV of a test is calculated by dividing the number of true negatives(TN) by the total number of people who tested negative i.e. TN divide by TN plus FN
Slide19True characteristics in the population Test Have the Do not Totals Results disease have the disease Positive 80 100 180 PPV=80/180=44% Negative 20 800 820 NPV=800/820=90% A Positive predictive value of 44% means that of the 180 tested only 80 have the disease A Negative predictive of 90% means that of the 820 tested, only 800 do not have the disease
Slide20Reliability(Repeatability) of a diagnostic or screening tests
Reliability is the ability of a measure(both the subject and the observer to produce a repeatable result .
This
does not mean that the result is accurate, it just means that if the test is repeated, a similar result will be
obtained.
Clearly ,regardless of the sensitivity and specificity of test, if the test results cannot be reproduced, the value of and usefulness of the test are minimal
The factors contributing to the variability or non reliability of a test are listed.
Intra-subject variation( variation within individual subject0
Intra-observer
variation
(variation in reading of test result by same person
Inter-subject
variation(variation of test results between or among individual
Intra-observer variation(variation of test results between observers
Slide21Number of patients
Observer 1Observer 21002013104005106007118009101011
n=10 0=Negative 1=Positive
Slide22Positive for test 1 and
2Patients
7 and 10= 2 positive’s
Negative for 1 and
2Patient
1, 4, 6, 8 =4 negative’s
Positive for 1 and negative for
2Patient
3, 5, and 9=3
positives
Negative
for 1 and positive for
2
Patient
2, =1 negative
Observer 1
Positive Negative Total
Observer 2 Positive 2 1 3
Negative 3 4 7
Total 5 5 10
Calculate the expected the total expected proportion of agreement (
P
e
)
Calculate the expected the total expected proportion of
agreement(P
o
)
The (kappa) statistic
Slide23Use the 2x2 table on the next slide for calculating the kappa statistic (strength of agreement):
The total
expected
proportion of agreement, P
e
,
is given by :P
e
=
[
(
a+c)
*
(a+b)
]
+ [
(
b+d
)
*
(
c+d
)
]
N N N N
The total observed proportion of agreement, P
o
is
given by: P
o
= (
a+d)
N
The (kappa) statistic is given by:
P
o
- P
e
1-
P
e
where
Po-Pe
represents the proportion of the observed
agreement in excess of chance, and 1-
P
e
represent the
maximum proportion of agreement in excess of chance.
Slide24Value of
kappa
Strength
of agreement
<0.2
Poor
Poor
0.21-0.4
Fair
0.41-0.6
Moderate
0.61-0.8
Good
0.81-1.0
Very
good
Slide25Read chapters
………………………….
Elect 2 students to act as Doctor X and Doctor Y
The rest of the students will be patients, half of which will go to Dr X, the other half to Dr Y
The doctors will interview the patients, and decide whether they are eating a healthy diet or should be referred to a nutritionist
The recommendation for each patient will be recorded on the data collection sheet by each doctor.
Then the doctors will swop patients and repeat the interviews.
No contamination must take place, i.e. the doctors and patients should not share with each other anything that was said by their counterpart in
the previous interview.
The 2 doctors will make 10 copies of their data collection sheets and
distribute these to the 10 class groups.
The groups will summarise the data using the data summary sheet.
Each group will calculate the kappa statistic, and the results will be presented to the class.