Assessing the Risk of Bias - PowerPoint Presentation

344 views
Uploaded On 2020-06-16

Assessing the Risk of Bias - PPT Presentation

Critical appraisal of medical literature 2010 9 고려대학교 의과대학 안 형 식 What is bias Systematic error or deviation from the truth may overestimate or underestimate biased studies lead to misleading results ID: 779526

outcome bias data selection bias outcome selection data study studies outcomes control comparability effect adequate controls case risk follow

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/779526" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download The PPT/PDF document "Assessing the Risk of Bias" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

Assessing the Risk of Bias Critical appraisal of medical literature

2010. 9

고려대학교 의과대학

안 형 식

Slide2

What is bias?

Systematic error or deviation from the truth

may overestimate or underestimate

biased studies lead to misleading results

can’t measure the presence of bias, only the risk

rigorous methods

minimise

the risk of bias

question: “should I believe the results?”

Slide3

Biases in randomised controlled trialsBias - A process that tends to produce results that depart systematically from the true values existing in the study population

Selection bias

Avoid with randomisation and concealment

Performance bias

Avoid with standardisation of care and blinding

Attrition bias

Avoid using all subjects intention to treat analysis

Measurement bias

Avoid with blinding of outcome assessors and patients

Slide4

Study sample

Control

Intervention

Experimental

Intervention

Allocation of subjects

Outcome

Follow up

Selection bias

Randomisation

Concealment

Performance bias

Standardisation of care protocol

Blinding of care providers and patients

Measurement bias

Blinding of outcome assessors and patients

Follow up

Attrition

bias

Drop-outs?

Cross-over?

Is everyone accounted for?

Outcome

Slide5

Should we use quality scales? > 30 availablereliability and validity of many scales not established

different scales lead to conflicting conclusions

may include criteria not related to bias

no evidence for numerical weighting of different elements

how do readers interpret the score?

not recommended

for Cochrane reviews

Slide6

The Cochrane approachdescribe the following for each study in detail:

random sequence generation

allocation concealment

blinding

incomplete outcome data

selective outcome reporting

any other risks

empirical research shows that these components can have a significant effect on results, often leading to exaggerated effects

Slide7

For each study, in each domainis there enough information to understand what happened?if not, rate unclear

what is your judgement: are you satisfied that the study is at a low risk of bias?

yes

indicates a low risk

indicates a high risk

based on the context of your review, empirical evidence of bias effect, likely direction and magnitude of effect

Slide8

Random sequence generation

occurs at the start of a trial before allocation of

participants

determines the order of allocation into intervention and control groups

avoids systematic differences between groups

accounts for known and unknown confounders

minimises

selection bias

Slide9

Results of 32 comparative studies of anti-coagulant therapy for MI patients (Chalmers et al, 1977)

Study design

Apparent

risk reduction

Historical controls

(18 studies, 9000 subjects)

62%

Concurrent non

random

Controls (8 studies, 3000

subjects)

34%

Randomised

(6 studies, 4000 subjects)

22%

Slide10

Identifying a random sequenceAdequate

random number table

computer random number generator

coin toss

shuffling cards or envelopes

throwing dice

drawing lots

Not adequate

date of birth

day of visit

ID or record number

alternate allocation

choice of the clinician or participant

test results

intervention availability

Slide11

Allocation concealmentoccurs at the start of the trial during allocation of participantswhen a person is recruited to the study, no-one knows which group they will be allocated to

protects the random sequence: prevents

changing the order of recruitment, or deciding not to recruit

strongest empirical evidence showing this

is important to results

minimises

selection bias

Slide12

Selection bias in trial with fore-knowledge of treatment allocation: Amniotomy or oxytocin for induction of labour (Bakose & Backstrom re-analysed by M Keirse)

indicates an unfavourable cervix: Keirse hypothesised that such patients would be less likely to be entered in the trial if it were known that they would be allocated amniotomy.

This trial was described as the first “prospective randomised study “between amniotomy and oxytocin for induction of labour in a “totally unselected population” !!!

Bishop score at

entry to trial

Allocated oxytocin

(even date of

birth)

Allocated

amniotomy (odd

date of birth)

3 or less*

4 or 5

6 or more

Total

110

113

P value

16.1

<0.00025

Slide13

Identifying allocation concealment

Adequate

central allocation (phone, web, pharmacy)

sequentially numbered, identical drug containers

serially numbered, sealed, opaque envelopes

Not adequate

random sequence known to staff in advance

envelopes without all three safeguards

non-random, predictable sequence

Slide14

Blindingoccurs during the intervention and

measurement of outcomes

minimises

performance bias

different treatment of the two groups

participant expectations

minimises

detection

bias

different measurement of outcomes between the two groups

subjective outcomes particularly vulnerable

can blind the participant, care provider, outcome assessor, other personnel – more than “double blinding”

check for intention and success of blinding

Schulz KF & Grimes DA 2002 Lancet

Slide15

위약 효과 - 관절염 통증 완화

Slide16

Identifying blinding

Adequate

participants and key study personnel blinded

blinding probably not broken

outcomes not likely to be influenced

Not adequate

any of the above not met

Slide17

Allocation concealment vs blinding

Time

Randomisation

Concealment

of allocation

Blinding

Selection bias

Performance bias

Slide18

Incomplete outcome datawhen complete outcome data is not available for all participants

can indicate

attrition

bias

can have important impact when:

enough data is missing to affect the results: no. of participants missing (dichotomous) or effect size (continuous)

the no. of people missing is not balanced between groups

the reason for absence is related to the study outcomes (e.g. moved away vs adverse event)

two causes

loss of participants to follow up

exclusion of participants by trialists

Slide19

추적 탈락 (Losses-to-follow-up)어느 정도가 적당한가?

“

5 : 20

의 법칙

”

이하는 바이어스가 적을 것임

20%

이상은 타당도에 심각한 영향을 줌

→ 그러나 과도한 단순화→ 비교되는 추적관찰 손실 비율(Losses-to-follow-up)과 결과발생율

(outcome event rate)에 의존함 * 추적관찰의 손실율이 결과발생율을 초과하지는 않아야 함

Slide20

Intention-to-Treat 원칙 무작위화의 유지원칙 :

대상이 무작위 할당 된후에는 애초에 배정된 군에 따라 분석할 것

실험중단

치료를 받지 않거나

, crossover

의 경우에도 무작위 할당 분석은 유지되어야 한다

예외 : 환자가 무작위 할당 전에 작성된 기준에 따라 맹검 재평가에서 부적당하다고 판명된 경우

Slide21

Identifying incomplete outcome data

Adequate

no missing data

reasons for missing data not related to outcome

missing data balanced across groups with similar reasons

number of participants missing or plausible effect size not enough to change observed effect

Not adequate

any of the above criteria not met

‘as-treated’ analysis with substantial departure from allocated intervention

missing data imputed using inappropriate methods

Slide22

Selective outcome reporting

when outcomes are not reported as planned

outcomes missing

new outcomes added (can be justified, e.g. adverse events)

unexpected statistics, subscales or subgroups

reporting that cannot be used in a review

can indicate ‘

within-study publication bias

’ or ‘data mining’

difficult to determine

compare methods to results

refer to study protocol or trial register

look for commonly used outcomes

Slide23

Identifying selective outcome reporting

Adequate

protocol is available and all pre-specified outcomes reported in the pre-specified way

protocol not available but all expected outcomes are reported

most studies will be judged ‘unclear’ in this category

Not adequate

outcomes not reported as planned or expected

limited information provided for some outcomes (e.g. only direction of effect and significance)

Slide24

Assessing the Risk of Bias - PowerPoint Presentation

Assessing the Risk of Bias - PPT Presentation

Share:

Link:

Embed:

Related Contents