/
Issues and Challenges around Synthesising Qualitative Resea Issues and Challenges around Synthesising Qualitative Resea

Issues and Challenges around Synthesising Qualitative Resea - PowerPoint Presentation

celsa-spraggs
celsa-spraggs . @celsa-spraggs
Follow
423 views
Uploaded On 2016-07-24

Issues and Challenges around Synthesising Qualitative Resea - PPT Presentation

Most methods of primary qualitative data analysis can be applied to qualitative evidence synthesis New evidence synthesis approaches have emerged eg metaethnography but use recognisable qualitative analysis principles ID: 418499

methods qualitative research synthesis qualitative methods synthesis research analysis reviews interventions evidence meta complex data systematic challenges framework 2013

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Issues and Challenges around Synthesisin..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Issues and Challenges around Synthesising Qualitative Research Slide2

Most methods of primary qualitative data

analysis

can be applied to qualitative evidence synthesis‘New’ evidence synthesis approaches have emerged –eg meta-ethnography – but use recognisable qualitative analysis principles Many methods have similar sounding names – rush to publish ‘my method’ and call it something slightly different – meta-narrative, meta-ethnography, meta-summary, meta-synthesis, meta-aggregation – etc etc. Some reviewers mix or combine methods – eg thematic and realist into a single review

What have we learned so far? Slide3

Many authors appear to stick with familiar methods rather than select the most appropriate method to address the question and type of evidence

Software evolving and improving: EPPI reviewer / Atlas Ti

Searching methods and approaches have evolved for specific methods of qualitative synthesis (purposive versus exhaustive, tipping point) Increasing numbers of (very good and inevitably some very bad) QES published!Cochrane finally getting to grips with qualitative evidence synthesis! What have we learned so far cont? Slide4
Slide5

Looking to the future: where are we now and where do we

want to go next?

Increasing number of reviews including qualitative evidencePrescription versus pragmatismProtocolsRevManAnd, revisiting primary qualitative research toolbox to help advance methodological challenges Slide6

Huge increases in qualitative syntheses published

There is a big gap between what people claim to use as a synthesis approach and what is actually done in practice

Do not need more methods – but better application and further evaluation of existing onesSlide7

New and updated guidance and new acceptance of the value of qualitative evidence

Slide8

Greater specification and further adaptation of methods for qualitative evidence synthesis

Thematic synthesis –various types with different starting points -

inductive and deductive 3 stage thematic synthesis - Line by line inductive coding- eg Thomas and HardenA priori - eg 5 stage Ritchie and Spencer Framework SynthesisConceptual frameworks and models to guide analysis and synthesisAdditional adaptations to

conceptual frameworks to ‘best-fit

’ specific

reviews –

eg

conceptual development by the back

door (pragmatism)Slide9

Gallacher,

et al.

(2013). Qualitative systematic reviews of treatment burden in stroke, heart failure and diabetes-Methodological challenges and solutions. BMC medical research methodology, 13(1), 10.Framework analysis… was appropriate due to our a priori knowledge in this area. …always a risk with framework analysis that data has been ‘shoe horned’ into the framework, with the possibility that some data may be missed. .. although this work was deductive to some extent, we were careful to augment the framework during analysis, being careful to ensure that our findings were derived directly from the data, and importantly, made a careful note of any data that fell outside of our framework. We failed to find any such data [that fell outside of framework], which suggests that the use of NPT as the underpinning theory for our analysis proved to be appropriate in this case.Slide10

‘Best Fit Framework Synthesis’

Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). “Best fit” framework synthesis: refining the method. 

BMC medical research methodology

13

(1), 1-16.Slide11

BMC

Med Res

Methodol. 2014 Jun 21;14(1):80. Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. Toye F et al. Builds on Noblit & Hare to explore challenges of including a large number of qualitative studies into a QES. Challenges hinge upon epistemological and practical issues alongside expectations of what determines high quality research. Central to method - collaborative interpretation of concepts and decision to exclude original material where team could not decipher a concept. Uses excerpts from research team's reflexive statements to illustrate development of

methods.Slide12

Lee

,

et al. Qualitative synthesis in practice: some pragmatics of meta-ethnographyQualitative Research, February 24, 2014 doi:10.1177/1468794114524221Reviews existing meta-ethnographies with focus on methods reported by authors. Considers implications of these methods and reason for presence (and absence) of practices in reporting on meta-ethnographies. Draws upon team experiences of conducting meta-ethnographies with focus on two key practices: ‘reading’ and ‘conceptual innovation’. Concludes by discussing how meta-ethnographic process requires active reading, recognition of multiplicity, realistic

approach to conceptual innovation and, importantly, collaborative work

.

 Slide13

T

herapist

dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both); Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels);

P

rofessional

or patient education interventions

(e.g., introduction of clinical guidelines).

Complex

interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or

dysynergistically

or be interdependent.

Greater

emphasis on

QES for complex

interventionsSlide14

MICCI -

Methodological Investigation of Cochrane Reviews of Complex Interventions Slide15

Applying qualitative methods to explore complexitySlide16

Development of little used methods

Qualitative Comparative Analysis

(QCA); (actually a mixed method approach) is a promising method for providing evidence in situations where interventions interact with contexts, enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes.Byrne D. Evaluating complex social interventions in a complex world 2013Blackman et al (2013) Using Qualitative Comparative Analysis to understand complex policy problemsCandy, B., King, M., Jones, L., & Oliver, S. (2013). Using qualitative evidence on patients' views to help understand variation in effectiveness of complex interventions: a qualitative comparative analysis. Slide17

QCA for synthesisSlide18

Development of little used methods

Use of

content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews Finfgeld-Connett 2013Systematically reviewing and synthesizing evidence from conversation analytic and related discursive research to inform healthcare communication practice and policy: an illustrated guide Parry and Land 2013 Systematic text condensation: A strategy for qualitative analysis Malterud 2012The procedure consists of the following steps: 1) total impression – from chaos to themes; 2) identifying and sorting meaning units – from themes to codes; 3) condensation – from code to meaning; 4) synthesizing – from condensation to descriptions and concepts. Slide19

Greater use of Conceptual Frameworks, Theoretical and Logic models

Programme logic

is the way in which a ‘programme’ fits together, usually in a simple sequence of inputs, activities, outputs, and outcomes. Programme theory goes a step further and attempts to build an explanatory account of how the intervention/programme/service works, with whom, and under what circumstances.Slide20

Baxter,

S

K., et al. "Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions.“ BMC Medical Research Methodology 14.1 (2014): 62.Use of logic models

Turley et al 2013 Slum upgrading review: methodological challenges that arise

in systematic reviews of complex interventionsSlide21

Health belief model: public information to prevent skin cancerSlide22

Looking to future

: where are we now and where

do we want to go next?Increasing the number of reviews including qualitative evidencePrescription versus pragmatismProtocolsRevManAnd, revisiting primary qualitative research toolbox to help advance methodological challenges Slide23

Increasing number

of reviews including qualitative

evidenceSlide24

Multiple methods within a single HTA review Slide25

Prescription versus pragmatismSlide26

Purposive Sampling and Excluding Studies

More

purposeful way of sampling papers - as opposed to the comprehensive search suggested by the Collaboration for reviews that are focused on the effects of interventions. Purposive sampling not comprehensive… Interest of authors not in seeking a single ‘correct’ answer, but rather in examining the complexity of different conceptualizations. Reviewers synthesizing qualitative research may see little value in limiting the critical appraisal of studies to detecting potential methodological flaws in them, because methodologically flawless studies are no guarantee for an in-depth or rich contribution to a QES  Slide27

Protocols

Most

reviewers have “pragmatic orientation toward standards and protocols. They make them work to the extent that they serve their particular goals, but once these goals diverge too much from the interests advanced by those stakeholders, then little space remains for interaction. In such cases, the standard may need to be renegotiated”. Need to be “accompanied by a movement in the direction of considering flexible, iterative protocols as an acceptable standard”Slide28

RevMan

Currently, the template used for the review process only supports a linear approach to synthesis. Over recent years,

RevMan developers have been increasingly more receptive….to negotiate with members of the CQIMG.Substantial efforts from both sides to try to adapt the software to support the inclusion of QES findings. “Complete consonance between what qualitative researchers would like to see and what The Cochrane Collaboration is able to deliver is unlikely to happen in the next few years, due to limited manpower and resources”. Slide29

Revisiting the primary qualitative research toolboxSlide30

Harnessing “dual heritage”

Faced with

dual heritage, [conventional SR methods + primary qualitative research] exponents of QES may select judiciously from competing techniques, adapt from richness of both traditions or maintain open dialogue around viable alternatives. “Placing [quantitative/qualitative] approaches in opposition does a great disservice by detracting from the contribution to be made by each, including what each can contribute to the other” (Wolcott, 2001). Rapprochement of two heritages heralded by recent case study that recognises unique contribution from each source::

“such

reviews are, to

some extent

, methodologically

sui generis

and cannot be governed solely

by concepts

imported either from SRs of quantitative evidence (e.g

. comprehensiveness

) or from primary qualitative research (e.g. saturation)” (Lorenc

et al, 2012).Slide31

Finfgeld-Connett D.

Use

of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qualitative Research June 2014 14: 341-352Content analysis is a flexible data analysis method; but application for qualitative systematic reviews has not been fully explicated. Qualitative systematic reviewers are urged to adapt content analysis methods to accommodate data that are, by nature, highly organized and contextualized. Reviewers encouraged to use reflective memoing and diagramming to ensure valid integration, interpretation, and synthesis of findings across studies. Reviewers advised to clearly and fully explain their data analysis methods in research reports.Slide32

New publication standards

Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ BMC Medical Research Methodology, Vol. 12, No. 1. (27 November 2012), 181,

doi:10.1186/1471-2288-12-181

by 

Allison Tong

Kate Flemming

Elizabeth McInnes

,

Sandy Oliver

Jonathan CraigSlide33

Continuing challenges and issues

Many methods, small

number consistently usedNeed

more high quality published examples and methodological testing

Evidence

synthesis

cheaper than primary research – funders increasingly fund systematic reviews – need more teams with

quan

/

qual

skills

Need

greater specification of methods for implementation

reviews/complex interventions