Dr Andrew Booth Illustrating a Realist Review approach For Which Schools Do School Performance League Tables Work How Do School Performance League Tables Work Do they work in a similar way to Hospital Performance League Tables ID: 380801
Download Presentation The PPT/PDF document "Realist Synthesis 101" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Realist Synthesis 101
Dr Andrew BoothSlide2
Illustrating a Realist Review approach
For Which Schools Do School Performance League Tables Work?
How
Do School Performance League Tables Work?
Do they work in a similar way to Hospital Performance League Tables?
What about Individual Surgeon Performance League Tables? Slide3
Developing Programme Theory
Programme Theory #1:
Proponents
of performance league tables believe their publication stimulates competition,
Programme Theory
#2:
and
that, as each provider adopts “best practice,” the quality of services will improve.
6
Supporting Data:
Some
published evidence seems to support this.
New
York probably has the best established system for providing the public with
information…
Soon after publication of performance league tables based on providers, the risk adjusted mortality for bypass surgery declined, leading some to conclude that this was a direct result.
7
Counter Argument:
One
alternative explanation being that, once providers know their data will be used for comparative purposes, they may resort to “creative reporting
.”
8
9
Programme Theory
#3:
Performance
league tables may also improve patients' choice, and proponents argue that this is necessary for an efficient market economy by encouraging consumers to seek out high ranking providers.
9
Slide4
Developing Propositional Statements (IF-THENs)
IF Performance Tables are published THEN:
Staff at
Well-performing schools
(
C
ontext
1
) will change their reasoning/actions because (
M
echanisms
1
) (e.g. Maintain competitive advantage)…..with
the result (
O
utcome
1
) that……
Staff at
Average-performing schools
(
C
ontext
2
) will
change their
reasoning/actions
because (
M
echanisms
2
) (e.g. Identification of good practice)…..
with the result (
O
utcome
1
) that…….
Staff at
Poor-performing schools
(
C
ontext
3
) will
change their
reasoning/actions
because (
M
echanisms
3
)
(e.g. Shame
)……with the result (
O
utcome
1
) that……..
CMO
configurations where Mechanisms = changes that occur in people's reasoning and actionsSlide5
Realist synthesis
Programme
intervention
Context
Altered context
Context triggers and Interacts with
Mechanisms
Mechanisms
Outcomes (intended and
unintended
)Slide6
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide7
Formulating the review questionSlide8
Formulating
the review question
Realist questions aim to identify diverse literature on the intervention of interest, in order to evaluate what works for whom in what circumstances
They are relatively open ended, aiming to capture a broad spectrum of relevant primary research in the first instance
They aim to identify the relationships between Intervention, context, mechanisms and outcomesSlide9
Our realist
questions
What approaches to community engagement are most effective in promoting peer support, to which people and in what circumstances?
(2) How does community-based peer support impact on understanding of existing health information and use of health information and health services to improve health and reduce health inequalities? Slide10
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide11
Preliminary
theory
for COPES
C
ommunity engagement aiming
t
o promote peer support
increases health literacy
improves health
reduces health inequalitiesSlide12
Assumptions
Communities initiate and foster peer support
The process of providing community-based peer support influences the ability to develop health
literacy
Health literacy creates a sense of empowerment, which is associated with a reduction in health inequalities
Community-based peer support raises awareness of issues with literacy and empowerment
Community
Peer support
Health literacySlide13
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide14
Scoping the literature
For the initial scoping, do a comprehensive search to ‘map the territory’
Abstract sift: Identify
and
categorise
abstracts by
programme
theory, type
of interventionSlide15
Scoping the
literature for key concepts
Community engagement
: Very small number of articles documenting ‘working collaboratively with and through’ community engagement for peer support
There were variations in
community-based peer support
in terms of
A
mount
of ‘emotional
, appraisal and informational
assistance’
Degree of ‘created social network member’Degree of ‘experiential knowledge of a specific behaviour or stressor’
Definitions of ‘Similar characteristics’The term health literacy was not used in peer support articlesso included all articles on health education/health promotion relating to ‘Understanding and use of health information and health services’Reduction of health inequalities – or any explicit discussion of health inequalities – was absentSlide16
Identifying sources of information in real life settings
Define concepts
Compare definitions
Identify what works
Network with participants
Scope the literature
Explore gaps
Test the emerging theory
Propose a model
Research knowledge
Community knowledge
Practitioners
Community organizations Peer supporters
Expert patients
Health librarian
Qualitative researchers
Systematic reviewers
CBPR researchersSlide17
Steps for conducting a realist synthesis
Formulate review question
Scope the primary
sources of information
Decide on the scope of the review
Map the
programme
theory(s)
Identify
programme
components
Assess effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide18
Defining key concepts:
community engagement and peer support
Community engagement
is ‘the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those
people…It
often involves partnerships and coalitions that help mobilize resources and influence systems, change relationships among partners, and serve as catalysts for changing policies, programs, and practices (CDC,
1997)
Peer
support
within a healthcare context is “...the provision of emotional, appraisal and informational assistance by a created social network member who possesses experiential knowledge of a specific behaviour or stressor and similar characteristics as the target
population.” (
Dennis, 2003). Emotional support includes expressions of caring, empathy, and encouragement, and is seen to enhance self-esteem. Appraisal support involves encouraging persistence and optimism for resolving problems, affirmation of a peer’s feelings, and reassurance that frustrations can be handled. Informational support is providing advice, suggestions, alternative actions, feedback and information relevant to the issue with which the peer is dealing.Slide19
Defining key concepts: health literacy
Health literacy
–
‘the
personal, cognitive and social skills which determine the ability of individuals to gain access to, understand, and use information to promote and maintain good health. These include such outcomes as improved knowledge and understanding of health determinants, and changed attitudes and motivations in relation to health behaviour, as well as improved self-efficacy in relation to defined tasks. Typically these are outcomes related to health education activities…’ (
Nutbeam
, 2001
)
From a
number of definitions for health literacy – we chose one that related to the notion of empowerment because it was relevant to health inequalitiesSlide20
(
Nutbeam
, 2009)Slide21
Health literacy model proposed by Sorensen et al (2012)
A year into our review, a conceptual analysis of the definitions was conducted, and a unifying model was produced (Sorensen et al, 2011
). This model focuses on the capabilities of accessing, understanding, appraising and applying health information. Slide22
World
Decide on the
scope of the review
Things to consider:
Amount of literature: Comprehensive versus sampling?
Diversity of literature: Consensus on
programme
theory? Variations on theory by what, who, when,
setting
Access to other sources of information
Interests of those who commissioned the review
What was the original problem that needs research evidence?
International, national, regional, local?
What will the findings be used for?
Who is the audience? (this is sometimes wider than just commissioners)
Time and money
Experience
Me
Team
WorldSlide23
Setting the
parametres
via sampling
Programme
theory was not used for sampling because
Theory was only explicitly cited in 28 instances
Studies combined elements from the Heath Belief Model, Theory of Planned
Behaviour
,
social cognitive and social learning theories, the social support literature, social comparison theory, social network approaches, and empowerment
models
Studies used both individual behaviour change and community-based theories, including community organization, diffusion of innovation, and social marketing
Domains for all the individual behaviour change theories are included in the Capabilities Opportunity Motivation Behaviour theory (COM-B, Michie et al 2011)
Pragmatic sampling on location: the most common United Kingdom health conditions in the peer support literature were reviewed firstSlide24
Mapping literature topics to UK settings
TOPIC
Advisory Network
Breastfeeding
Sheffield Well Being Consortium; Sheffield Breastfeeding study
Darnall
Wellbeing
AGE UK; Stroke Association
Irish Gypsy & Travellers Movement in Britain; Leicester Health Ambassadors
Centre for HIV & Sexual Health; Parent to Parent, Shout; Sheena Amos Trust; Expert Patients Programme
Sheffield Community Chronic Pain;
Manor & Castle Development TrustMECOPP Minority Ethnic Carers of People Project; Autism Plus
Sharrow ShipShape; Sheffield PCTMIND; MencapSheffield City Council SOAR (Southey Owlerton Area Regeneration)ZEST Community Development TrustDiabetes
Healthy Living Older PeopleHIV/AIDSHealth Trainers
NutritionSmokingSlide25
Summary of searching, sifting and samplingSlide26
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide27
Identify the
programme
components
Programmes
categorised
by
Stage
Recruitment of peersTraining and supporting peersImplementing peer support - what was delivered
Outcomes Within each Stage, components were identified in the literature and through workshops with community membersComponents
put together in CMO configurations by the topic leadsSlide28
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide29
Mapping
programme
theories
The theories and assumptions were documented using Context Mechanism Outcome (CMO) configurations for each health topicSlide30
Dealing with diverse theories
Some topics used a single theory, for example Diffusion of Innovation
Other topics did not explicitly state how theory informed design, but contained implicit assumptions about what would work
Concepts
from both
stated/implicit
theories
identified
via
CMOs
Dimensions
of
concepts for one topic used to question concepts in another topic areaFor example, ‘
recognising someone as a peer’ was an important mechanism. We asked ‘what does recognising someone as a peer mean across different health topics? The dimensions we discovered included age, ethnicity, experience with the problem or condition We also used Advisory Network as a source of propositional statementsSlide31
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)
IterativeSlide32
Turning CMOs into propositional statements: analysis of the literature
Overarching propositional
statement
Supporting data by topic
Smoking cessation young people
Healthy living in older people
Smoking in ethnic minorities
IF target
community recognise
PS
as their
peer THEN PS is effective
I 5/6:
Target community must recognise the peer supporters as their peers.(Propositional statement)
I12:When age is only common characteristic, and differences like ethnicity exist (meaning potentially different needs and value systems), the peers may have a negative experience. (Propositional statement)D88Smoking cessation service is offered by health professionals, who created a formal and impersonal atmosphere, but smokers feel a sense of distrust towards providers
(Mechanism)Slide33
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide34
Searching for related theories
CMOs
and propositional statements
(IF-THENs) considered
in terms of higher order concepts, for example
Overarching propositional statements
Concepts
IF the
target community recognise the PS as their
peer THEN peer support is effective
Social
learning
Relational learning
Participatory parityTrustIF there is a positive relationship between the PS and the peers (trust and a sense that the person is believable, credible) THEN peer support is effectiveSlide35
Steps for conducting a realist synthesis
Formulate review question
Scope the primary literature
Decide on the scope of the review
Identify the
programme
components
Map the
programme
theory(s)
Assess
effectiveness of components, considering context and mechanisms
Develop propositional statements
Identify patterns in propositional statements across cases (demi-regularities)Search sources for theories related to the propositionsProduce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)Slide36
Mid-range theory
‘Theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change.’ (Merton, 1967: 39
)Slide37
Mid-range
theory
Mid-range theories relate empirical data to higher order constructs, and are ‘sufficiently abstract to deal with different spheres of social behaviour and social structure, so that they transcend sheer description or empirical generalisation’ (Merton, 1967: 68
).
Middle range
theory involves abstraction, of course,
but…close
enough
to observed data to be incorporated
in propositions
that permit empirical testing.”Slide38Slide39
References
Dennis, C.-L. (2003). Peer support within a health care context: a concept analysis. International journal of nursing studies, 40(3), 321–32.
Kincheloe
, J. and McLaren. Rethinking theory and research. (2005) In NK
Denzin
& YS Lincoln (
eds
) Sage Handbook of Qualitative Research.
Merton
R. On Theoretical Sociology. Five Essays, Old and New. New York: The
Free Press
,
1967
Nutbeam
, D. (2008). The evolving concept of health literacy. Social science & medicine (1982), 67(12), 2072–8.
Pawson, R. (2013) the science of evaluation: A realist manifesto. London: Sage.Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., & Brand, H. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC public health, 12(1), 80. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: realist syntheses. BMC medicine, 11(1), 21. doi:10.1186/1741-7015-11-21Slide40
Acknowledgements
Funded
by the National Institute of Health
Research
Public Health programme PHR -
09/3008/04
:
Community-based peer support: A
realist
synthesis and model for promoting health literacy (COPES)