Lunchbijeenkomst 14 maart 2013 HBO Universiteit Post Grad EMBA Business Schools Wat moet een student in de praktijk straks kunnen zelf onderzoek doen onderzoek kunnen beoordelen toepassen ID: 224803
Download Presentation The PPT/PDF document "Evidence-Based Practice" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Evidence-Based Practice
Lunchbijeenkomst14 maart 2013 Slide2Slide3
HBO
Universiteit
Post Grad
(E)MBA
Business SchoolsSlide4
Wat moet een student in de praktijk straks kunnen:
- zelf onderzoek doen? - onderzoek kunnen beoordelen + toepassen?
Wat betekent dat voor het curriculum?
Wat betekent dat voor de accreditatie?
AanleidingSlide5
“In onze moderne samenleving is het cruciaal dat hbo-bachelors over een onderzoekend vermogen beschikken dat leidt tot reflectie, tot evidence-based practice
, en tot innovatie.”HBO standaard
(2009)Slide6
Evidence-Based Practice ?Slide7
Wat
is het?Waar komt
het
vandaan?
Hoe
ziet
het
er
uit
in
een
opleiding
?
Hoe zit
dat
bij
4e
jaars
studenten
?Slide8
Evidence
based practice:Wat is het?Slide9
Uitgangspunt bij evidence-based
practice is dat beslissingen gebaseerd dienen te zijn op een combinatie van wetenschappelijk denken en de best beschikbare 'evidence'.
Evidence-based practiceSlide10
Met het begrip 'evidence' wordt niet meer bedoeld dan 'informatie'.
Dit kan informatie zijn afkomstig uit wetenschappelijk onderzoek, maar ook interne bedrijfsinformatie en zelfs persoonlijke ervaring geldt als 'evidence’.
Evidence
based
practiceSlide11
In principe neemt iedere manager dus beslissingen op basis van 'evidence'.
De meeste managers besteden echter nauwelijks aandacht aan de kwaliteit van de 'evidence' waarop ze hun beslissingen baseren.
Evidence
based
practiceSlide12
Evidence-based practice:
k
ritisch en wetenschappelijk denken
van
verschillende informatiebronnen gebruik
maken
de beschikbare
evidence kritisch tegen het licht
houden
denken
in termen van
waarschijnlijkheid
in plaats van 'golden
bullets
'. Slide13
Professional
expertise
and
judgement
Best available organizational evidence
Stakeholders’ values and concerns
Best available scientific evidence
Evidence-based decision
Evidence
based
practiceSlide14
Evidence
based practice:Waar komt
het
vandaan
?Slide15
“there is a large research-user gap”
“practitioners do not read academic journals”“the findings of research into what is an effective intervention are not being translated into actual practice”
“academics not practitioners are driving the research agenda”
“the relevance, quality and applicability of research is questionable”
“practice is being driven more by fads and fashions than research”
“many practices are doing more harm than good”
What field is this?Slide16
McMaster University Medical School, Canada
Medicine: Founding
fathers
David Sackett
Gordon GuyattSlide17
How it all startedSlide18
if you’re hyperventilating
breathe into a bag
Problem
I:
persistent convictionsSlide19
elderly people
who have an irregular heartbeat are much more likely to die of coronary diseasegive them a drug that reduces the number of irregular beats
Problem
I:
persistent convictionsSlide20
H
ow 40,000 cardiologists can be wrongIn the early 1980s newly introduced anti-arrhythmic drugs were found to be
highly successful at suppressing arrhythmias.
Not
until a RCT was performed was it realized that, although
these drugs suppressed arrhythmias, they actually increased
mortality.
By the time the results of this trial were published, at least
100,000 such patients had been taking these drugs.Slide21
David Sackett
Half of what you learn in medical school will be shown to be either dead wrong or out-of-date within 5 years of your graduation; the trouble is that nobody can tell you which half.The most important thing to learn is how to learn on your own: search for the evidence!
(
Remember that your teachers are as full of bullshit as your parents)Slide22
More than 1 million articles in
40,000 medical journals per year (= 1995; now probably more than 2 million). For a specialist to keep up this means reading 25 articles every day (for a GP more than 100!)
Problem
II:
too much informationSlide23
Problem II:
too much informationHRM: 1,350 articles in 2010 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a ‘general’ manager more than 50!
)
BTW
: most of the research is
seriously flawed or irrelevant for practiceSlide24
The 5 steps EBP
Formulate a focused question (
Ask
)
Search for the best available evidence (
Acquire
)
Critically appraise the evidence (
Appraise
)
Integrate the evidence with your professional expertise and apply (
Apply
)
Monitor the outcome (
Assess
)
Zelf
onderzoek
doen
?Slide25
Evidence-Based Practice
1991 Medicine1998 Education
1999
Social
care, public policy
2000
Nursing
2000
Criminal justice
????
Management
?Slide26
Evidence
based practice:Hoe ziet
het
er
uit
in de
opleiding
?Slide27
Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken
Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical
appraisal)
Fase 4:
Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (
application
of
science
)
Evidence-based practiceSlide28
Professional
expertise
and
judgment
Best available organizational evidence
Stakeholders’ values and concerns
Best available scientific evidence
Evidence-based decision
Evidence
based
practiceSlide29
Waarom
(wetenschappelijk) onderzoek?Slide30
Trust me,
20 years of experienceSlide31
Bounded rationality Slide32
“The first principle
is that you must not fool
yourself
-
and
you
are the
easiest
person
to
fool”.
Richard FeynmanSlide33
Bounded rationality Slide34
Het
feilbare brein
System 1
Snel
,
actie
Intuitief
,
associatief
shortcuts &
biasses
System 2
Langzaam
(
lui
!)
Rationeel
NadenkenSlide35
Systeem 1Slide36
Seeing order in randomness
Mental corner cuttingMisinterpretation of incomplete dataHalo effect
False consensus effect
Group think
Self serving bias
Sunk cost fallacy
Cognitive dissonance reduction
Confirmation bias
Authority bias
Small numbers fallacy
In-group bias
Recall bias
Anchoring bias
Inaccurate covariation detection
Distortions due to plausibility
Systeem
1: het
feilbare
breinSlide37
Het
feilbare
brein
Meningen
Assumpties
(
aannames
)
Overtuigingen
Persoonlijke
ervaringen
Wetenschappelijk
onderzoek
Feiten
Evidence:
bewijs
/
aanwijzingen
Waarom
onderzoek
?
BIASSlide38
Denkfouten
InformatiebronnenMythbusting
Assumpties
Fase
1:
Kritisch
&
wetenschappelijk
denkenSlide39
Seeing order in randomness
Confirmation bias Small numbers fallacyOutcome bias
Halo effect
Authority bias
Group think
7
Denkfouten
(die u
beter
aan
anderen
kunt
overlaten
)Slide40
Een
Type I fout of
een
vals
positief
:
denken
dat
er
een
patroon
/
verband
is
terwijl
dat
er
in het
echt
niet
is.
Een
Type II
fout
of
een
vals
negatief
: denken dat er geen patroon /
verband
is terwijl dat
er in het echt wel isDr. Michael Shermer
(Director of the Skeptics Society)
Seeing order in randomnessSlide41
Een
Type I fout of een
vals
positief
:
denken
dat
het
geritsel
in de
bosjes
een
gevaarlijk
roofdier
is,
terwijl
het
gewoon
de wind is (
goedkoop
foutje
)
Het
feilbare
brein
:
patern
recognitionSlide42
Een
Type II fout
of
een
vals
negatief
:
denken
dat
het
geritsel
in de
bosjes
gewoon
de wind is,
terwijl
het
een
gevaarlijk
roofdier
is (
duur
foutje
)
Het
feilbare
brein
:
patern
recognitionSlide43
Het
probleem met patroon herkenning
:
Het
kritisch
beoordelen
of
er
sprake
is van
een
Type I of
een
Type II
fout
is best
moeilijk
, (
vooral
in ‘split second life and death’
situaties
),
dus
de default
positie
is
om
aan
te
nemen
dat
alle
patronen
echt zijn.
Het
feilbare brein
: patern recognitionSlide44
Een
Type I fout of een
vals
positief
:
denken
dat
het
geritsel
in de
bosjes
een
gevaarlijk
roofdier
is,
terwijl
het
gewoon
de wind is (
goedkoop
foutje
)
DEFAULT
Een
Type II
fout
of
een
vals
negatief
:
denken dat het geritsel in de bosjes gewoon de wind is,
terwijl
het een gevaarlijk
roofdier is (duur foutje)Het feilbare brein
:
patern recognitionSlide45
Ook
ervaren
mensen
en experts
zien
patronen
en
verbanden
waar
ze
niet
zijn
.
stress & lifestyle
peptic ulcer
Het
feilbare
brein
:
patern
recognitionSlide46
Peptic ulcer – an infectious disease! This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By using technologies generally available (
fibre
endoscopy, silver staining of histological sections and culture techniques for
microaerophilic
bacteria), they made an irrefutable case that the bacterium Helicobacter pylori is causing disease. By culturing the bacteria they made them amenable to scientific study.
In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is now
firmly established that Helicobacter pylori causes more then 90% of duodenal ulcers. The link between Helicobacter pylori infection and peptic ulcer disease has been established through studies of human volunteers, antibiotic treatment studies and epidemiological studies.
Oct 2005Slide47
Ook slimme mensen houden er verkeerde ideeën op na, niet omdat ze dom of eigenwijs zijn, maar omdat het de meest logische conclusie is op basis van hun eigen ervaringen.
Het
feilbare
brein
:
patern
recognition
(
systeem
1
doet
altijd
mee
!)Slide48
Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken
Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical
appraisal)
Fase 4:
Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (
application
of
science
)
Evidence-based practiceSlide49Slide50
5-step approach
EBMgt is a 5-step approach
Formulate an answerable question (PICOC)
Search for the best available evidence
Critically
appraise the quality of the found evidence
Integrate the evidence with managerial expertise and organizational concerns and apply
Monitor and evaluate the resultsSlide51
Formulate
a
focused
questionSlide52
Focused question?
Does team-building work?
What are the costs and benefits of self
-steering
teams?
What are the success factors for culture change?
Does management development improve the performance of managers?
Does employee participation prevent resistance to change?
How do employees feel about 360
degree
feedback?Slide53
What is a ‘team’?
What kind of teams?In what contexts/settings?What counts as ‘team-building’?What does ‘work’ mean?
What
outcomes are
relevant?
O
ver
what time periods
?
Foreground question?
Does team-building work?Slide54
P
=
Population
I
=
Intervention
or
success
factor
C
=
Comparison
O
=
Outcome
C
= Context
Answerable
question: PICOCSlide55
2. Finding the best available evidenceSlide56
Searching evidence
What do we search?Slide57
Current Information
Overview of a subject
General background
Academic Information
Statistical Information
Theories about a subject
What do we search?
Company informationSlide58
Peer reviewed journalsSlide59
Searching evidence
Where do we search?Slide60
Searching for evidenceSlide61
Databases
ABI/INFORM
Business Source Elite
PsycINFO
Web of Knowledge
ERIC
Google ScholarSlide62
Searching for evidenceSlide63
Searching evidence
How do we search?
Search StrategySlide64
Two types of search strategies
Search strategyBuilding blocks method
Snowball methodSlide65
Search strategySlide66
Search in ABI/Inform:
How many articles has Stephen Covey published in peer reviewed journals?How many of these articles are based on scientific research?
Are there articles (by other authors) that are critical of Covey’s 7 Habits?
How many of these critical articles are based on scientific research?
Exercise: Search for evidenceSlide67
Search in ABI/Inform or BSE:
Search for peer reviewed research articles to answer the following question: What is the long term effect of a hostile take-over on the financial performance of the acquired organization? Use the following search terms: “hostile takeovers”, “financial performance”, “long term”
How many studies did you find?
Exercise: Search for evidenceSlide68
Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken
Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical
appraisal)
Fase 4:
Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (
application
of
science
)
Evidence-based practiceSlide69
Which
design for which question?Research designsSlide70
Randomized controlled study?
Grounded theory approach?
Cohort / panel study?
Qualitative field research?
Longitudinal study?
Post-test only study?
Survey?
Action research?
Case study?
What is the best design?Slide71
What is the BEST car?Slide72
What is the best design?
quants vs quallies, positivists
vs
post structuralist
,
etcSlide73
on the research questionSlide74
Effect
vs Non-effectSlide75
Types of questions
Does it work?Does it work better than ....?
Does it have an effect on ....?
What is the success factor for ....?
What is required to make it work ...?
Will it do more good than harm?
EffectSlide76
Types of questions: non-effect
Needs: What do people want or need?
Attitude:
What do people think or feel?
Experience:
What are peoples’ experiences?
Prevalence:
How many / often do people / organizations ...?
Procedure:
How can we implement ...?
Process:
How does it work?
Explanation:
Why does it work?
Economics:
How much does it cost?Slide77
Internal validitySlide78
internal validity
= indicates to what extent the results of the research may be biased and is thus a comment on the degree to which alternative explanations for the outcome found are possible.
Internal validitySlide79Slide80
We are pattern seeking primates:
we are predisposed to see order
and causal relations in the
world
Causal relationsSlide81
Are the "cause" and the "effect”
related? Does the
"cause"
precede
the "effect" in
time?
Are there no
plausible alternative explanations for the observed
effect?
effect size
before and after measurement
randomization, blinding, control group, measurements
Considerations for research:
CausalitySlide82
Bias & Confounding
Research shows:Shoe size
> quality
of handwriting
Smoking youngsters > better lung functionSlide83
Levels of internal validitySlide84
Explanation
Which design for which question?Slide85
Different types of research questions require different types of research designs,
but ...Slide86
Best research design?
But 1: feasibilitySlide87
But 1: feasibilitySlide88
Step 3: Critical
appraisal of studiesSlide89
Intermezzo
How to read a research article?Slide90
Critical
appraisal: quick and dirty
Is
the study design appropriate to the stated
aims?
Are
the measurements likely to be valid
and reliable?
W
as there a relevant effect size?
Is the outcome (population, type of organization) generalizable to your situation?Slide91
Levels of internal validity
Were there enough subjects in the study?
Was
a control group used?
Were the subjects randomly assigned?
Was a pretest used?
Was the study started prior to the intervention or event?
Was the outcome measured in an objective and reliable way?
6x yes =
very high (A)
5x yes =
high (A)
4-3x yes =
limited (B)
2x yes =
low (C)
1-0x yes =
very low (D)Slide92
Appraisal
Critical appraisal questionnaires
www.cebma.org
/
ebp
-toolsSlide93
CAT: Critically Appraised TopicSlide94
CAT:
Critically Appraised TopicA critically appraised topic (or CAT) is a structured, short (3 pages max) summary of evidence on a topic of interest, usually focused around a practical problem or question. A CAT is like a “quick and dirty” version of a systematic review, summarizing the best available research evidence on a topic. Usually more than one study is included in a CAT.
Slide95
CAT: structure
Background / context
Question (PICOC)
Search strategy
Results / evidence summary
Findings
Limitations
RecommendationSlide96
CAT-walk