Most methods of primary qualitative data analysis can be applied to qualitative evidence synthesis New evidence synthesis approaches have emerged eg metaethnography but use recognisable qualitative analysis principles ID: 418499
Download Presentation The PPT/PDF document "Issues and Challenges around Synthesisin..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Issues and Challenges around Synthesising Qualitative Research Slide2
Most methods of primary qualitative data
analysis
can be applied to qualitative evidence synthesis‘New’ evidence synthesis approaches have emerged –eg meta-ethnography – but use recognisable qualitative analysis principles Many methods have similar sounding names – rush to publish ‘my method’ and call it something slightly different – meta-narrative, meta-ethnography, meta-summary, meta-synthesis, meta-aggregation – etc etc. Some reviewers mix or combine methods – eg thematic and realist into a single review
What have we learned so far? Slide3
Many authors appear to stick with familiar methods rather than select the most appropriate method to address the question and type of evidence
Software evolving and improving: EPPI reviewer / Atlas Ti
Searching methods and approaches have evolved for specific methods of qualitative synthesis (purposive versus exhaustive, tipping point) Increasing numbers of (very good and inevitably some very bad) QES published!Cochrane finally getting to grips with qualitative evidence synthesis! What have we learned so far cont? Slide4Slide5
Looking to the future: where are we now and where do we
want to go next?
Increasing number of reviews including qualitative evidencePrescription versus pragmatismProtocolsRevManAnd, revisiting primary qualitative research toolbox to help advance methodological challenges Slide6
Huge increases in qualitative syntheses published
There is a big gap between what people claim to use as a synthesis approach and what is actually done in practice
Do not need more methods – but better application and further evaluation of existing onesSlide7
New and updated guidance and new acceptance of the value of qualitative evidence
Slide8
Greater specification and further adaptation of methods for qualitative evidence synthesis
Thematic synthesis –various types with different starting points -
inductive and deductive 3 stage thematic synthesis - Line by line inductive coding- eg Thomas and HardenA priori - eg 5 stage Ritchie and Spencer Framework SynthesisConceptual frameworks and models to guide analysis and synthesisAdditional adaptations to
conceptual frameworks to ‘best-fit
’ specific
reviews –
eg
conceptual development by the back
door (pragmatism)Slide9
Gallacher,
et al.
(2013). Qualitative systematic reviews of treatment burden in stroke, heart failure and diabetes-Methodological challenges and solutions. BMC medical research methodology, 13(1), 10.Framework analysis… was appropriate due to our a priori knowledge in this area. …always a risk with framework analysis that data has been ‘shoe horned’ into the framework, with the possibility that some data may be missed. .. although this work was deductive to some extent, we were careful to augment the framework during analysis, being careful to ensure that our findings were derived directly from the data, and importantly, made a careful note of any data that fell outside of our framework. We failed to find any such data [that fell outside of framework], which suggests that the use of NPT as the underpinning theory for our analysis proved to be appropriate in this case.Slide10
‘Best Fit Framework Synthesis’
Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). “Best fit” framework synthesis: refining the method.
BMC medical research methodology
,
13
(1), 1-16.Slide11
BMC
Med Res
Methodol. 2014 Jun 21;14(1):80. Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. Toye F et al. Builds on Noblit & Hare to explore challenges of including a large number of qualitative studies into a QES. Challenges hinge upon epistemological and practical issues alongside expectations of what determines high quality research. Central to method - collaborative interpretation of concepts and decision to exclude original material where team could not decipher a concept. Uses excerpts from research team's reflexive statements to illustrate development of
methods.Slide12
Lee
,
et al. Qualitative synthesis in practice: some pragmatics of meta-ethnographyQualitative Research, February 24, 2014 doi:10.1177/1468794114524221Reviews existing meta-ethnographies with focus on methods reported by authors. Considers implications of these methods and reason for presence (and absence) of practices in reporting on meta-ethnographies. Draws upon team experiences of conducting meta-ethnographies with focus on two key practices: ‘reading’ and ‘conceptual innovation’. Concludes by discussing how meta-ethnographic process requires active reading, recognition of multiplicity, realistic
approach to conceptual innovation and, importantly, collaborative work
.
Slide13
T
herapist
dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both); Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels);
P
rofessional
or patient education interventions
(e.g., introduction of clinical guidelines).
Complex
interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or
dysynergistically
or be interdependent.
Greater
emphasis on
QES for complex
interventionsSlide14
MICCI -
Methodological Investigation of Cochrane Reviews of Complex Interventions Slide15
Applying qualitative methods to explore complexitySlide16
Development of little used methods
Qualitative Comparative Analysis
(QCA); (actually a mixed method approach) is a promising method for providing evidence in situations where interventions interact with contexts, enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes.Byrne D. Evaluating complex social interventions in a complex world 2013Blackman et al (2013) Using Qualitative Comparative Analysis to understand complex policy problemsCandy, B., King, M., Jones, L., & Oliver, S. (2013). Using qualitative evidence on patients' views to help understand variation in effectiveness of complex interventions: a qualitative comparative analysis. Slide17
QCA for synthesisSlide18
Development of little used methods
Use of
content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews Finfgeld-Connett 2013Systematically reviewing and synthesizing evidence from conversation analytic and related discursive research to inform healthcare communication practice and policy: an illustrated guide Parry and Land 2013 Systematic text condensation: A strategy for qualitative analysis Malterud 2012The procedure consists of the following steps: 1) total impression – from chaos to themes; 2) identifying and sorting meaning units – from themes to codes; 3) condensation – from code to meaning; 4) synthesizing – from condensation to descriptions and concepts. Slide19
Greater use of Conceptual Frameworks, Theoretical and Logic models
Programme logic
is the way in which a ‘programme’ fits together, usually in a simple sequence of inputs, activities, outputs, and outcomes. Programme theory goes a step further and attempts to build an explanatory account of how the intervention/programme/service works, with whom, and under what circumstances.Slide20
Baxter,
S
K., et al. "Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions.“ BMC Medical Research Methodology 14.1 (2014): 62.Use of logic models
Turley et al 2013 Slum upgrading review: methodological challenges that arise
in systematic reviews of complex interventionsSlide21
Health belief model: public information to prevent skin cancerSlide22
Looking to future
: where are we now and where
do we want to go next?Increasing the number of reviews including qualitative evidencePrescription versus pragmatismProtocolsRevManAnd, revisiting primary qualitative research toolbox to help advance methodological challenges Slide23
Increasing number
of reviews including qualitative
evidenceSlide24
Multiple methods within a single HTA review Slide25
Prescription versus pragmatismSlide26
Purposive Sampling and Excluding Studies
More
purposeful way of sampling papers - as opposed to the comprehensive search suggested by the Collaboration for reviews that are focused on the effects of interventions. Purposive sampling not comprehensive… Interest of authors not in seeking a single ‘correct’ answer, but rather in examining the complexity of different conceptualizations. Reviewers synthesizing qualitative research may see little value in limiting the critical appraisal of studies to detecting potential methodological flaws in them, because methodologically flawless studies are no guarantee for an in-depth or rich contribution to a QES Slide27
Protocols
Most
reviewers have “pragmatic orientation toward standards and protocols. They make them work to the extent that they serve their particular goals, but once these goals diverge too much from the interests advanced by those stakeholders, then little space remains for interaction. In such cases, the standard may need to be renegotiated”. Need to be “accompanied by a movement in the direction of considering flexible, iterative protocols as an acceptable standard”Slide28
RevMan
Currently, the template used for the review process only supports a linear approach to synthesis. Over recent years,
RevMan developers have been increasingly more receptive….to negotiate with members of the CQIMG.Substantial efforts from both sides to try to adapt the software to support the inclusion of QES findings. “Complete consonance between what qualitative researchers would like to see and what The Cochrane Collaboration is able to deliver is unlikely to happen in the next few years, due to limited manpower and resources”. Slide29
Revisiting the primary qualitative research toolboxSlide30
Harnessing “dual heritage”
Faced with
dual heritage, [conventional SR methods + primary qualitative research] exponents of QES may select judiciously from competing techniques, adapt from richness of both traditions or maintain open dialogue around viable alternatives. “Placing [quantitative/qualitative] approaches in opposition does a great disservice by detracting from the contribution to be made by each, including what each can contribute to the other” (Wolcott, 2001). Rapprochement of two heritages heralded by recent case study that recognises unique contribution from each source::
“such
reviews are, to
some extent
, methodologically
sui generis
and cannot be governed solely
by concepts
imported either from SRs of quantitative evidence (e.g
. comprehensiveness
) or from primary qualitative research (e.g. saturation)” (Lorenc
et al, 2012).Slide31
Finfgeld-Connett D.
Use
of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qualitative Research June 2014 14: 341-352Content analysis is a flexible data analysis method; but application for qualitative systematic reviews has not been fully explicated. Qualitative systematic reviewers are urged to adapt content analysis methods to accommodate data that are, by nature, highly organized and contextualized. Reviewers encouraged to use reflective memoing and diagramming to ensure valid integration, interpretation, and synthesis of findings across studies. Reviewers advised to clearly and fully explain their data analysis methods in research reports.Slide32
New publication standards
Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ BMC Medical Research Methodology, Vol. 12, No. 1. (27 November 2012), 181,
doi:10.1186/1471-2288-12-181
by
Allison Tong
,
Kate Flemming
,
Elizabeth McInnes
,
Sandy Oliver
,
Jonathan CraigSlide33
Continuing challenges and issues
Many methods, small
number consistently usedNeed
more high quality published examples and methodological testing
Evidence
synthesis
cheaper than primary research – funders increasingly fund systematic reviews – need more teams with
quan
/
qual
skills
Need
greater specification of methods for implementation
reviews/complex interventions