Choosing a Method of Qualitative Synthesis Dr
Description: Choosing a Method of Qualitative Synthesis Dr Andrew Booth Caveat Reviewer: Pandoras Box! Confusing Terminology, Variety of Choices Qualitative Systematic Review Qualitative Meta-Synthesis Qualitative Research Synthesis Qualitative
Related Topics
Download Presentation
"Choosing a Method of Qualitative Synthesis Dr" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Choosing a Method of Qualitative Synthesis Dr Andrew Booth<br>
slide2. Caveat Reviewer: Pandora’s Box!<br>
slide3. Confusing Terminology, Variety of Choices Qualitative Systematic Review
Qualitative Meta-Synthesis
Qualitative Research Synthesis
Qualitative Evidence Synthesis
Qualitative Interpretive Meta-Synthesis Best Fit Synthesis
Critical Interpretive Synthesis
Framework Synthesis
Meta-Aggregation
Meta-Ethnography
Meta-Interpretation
Meta-Narrative
Meta-Study
Meta-Summary
Narrative Synthesis
Qualitative Meta-Synthesis
Realist Synthesis
19. Rapid Realist Synthesis
Thematic Synthesis<br>
slide4. Decisions, Decisions! Novice – Bewildering variety of methods of synthesis – compounded by choice of checklists, conflicting guidance etcetera
Experienced - Many authors stick to familiar methods rather than select most appropriate method to address question and type of evidence
Much description of methods, little evaluation and critique
Much hiding behind “labels”, misuse of methods<br>
slide5. Caveat Reviewer: TREAD Carefully
Time/Timeframe
Resources
Expertise
Audience & Purpose
Data – Quality, Quantity, Thickness, Richness<br>
slide6. Time/Timeframe Most Time Consuming
Meta-ethnography – interpretive approach
Realist Synthesis – Identification of theory and development of programme theories Least Time Consuming
Meta-Aggregation –aggregating existing themes
Thematic Synthesis – inductively coding themes
Framework Synthesis
Best Fit Framework Synthesis “Framework-based synthesis is an important advance in conducting reviews of qualitative synthesis. The 'best fit' strategy is a variant of this approach that may be very helpful when policymakers, practitioners or other decision makers need answers quickly, and are able to tolerate some ambiguity about whether the answer is the very best that could be given”. Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011 Apr 14;9:39.<br>
slide7. Seven steps for meta-ethnography<br>
slide8. Process of Realist Synthesis from: McCormack, B., Wright, J., Dewar, B., Harvey, G., & Ballantine, K. (2007). A realist synthesis of evidence relating to practice development: findings from the literature analysis. Practice Development in Health Care, 6(1), 25-55.<br>
slide9. Resources Most Resource Intensive Least Resource Intensive<br>
slide10. Expertise Expertise in Qualitative Research Methods (e.g. Grounded Theory; Framework Analysis, Thematic Analysis)
Expertise in Synthesis Methods (incl. Searching, Data Extraction, Quality Assessment, Interpretation)
Knowledge of Topic Area<br>
slide11. Audience & Purpose “output of some methods… (Thematic Synthesis, textual Narrative Synthesis, Framework Synthesis, and ecological triangulation) is more directly relevant to policymakers and designers of interventions than…methods with a more constructivist orientation (Meta-Study, Meta-Narrative, Meta-Ethnography, Grounded Theory, CIS)…generally more complex and conceptual” (Barnett-Page & Thomas, 2009) Thematic Synthesis (including Meta-Aggregation) and Framework Synthesis produce findings that directly inform practitioners (Thomas & Harden, 2009)
Interpretive approaches (e.g. CIS, Meta-Ethnography) produce a model that requires practitioners to interpret relevance and applicability to their own context
Narrative Synthesis or EPPI-Centre (matrix) methods may help to integrate and present quantitative/qualitative work<br>
slide12. Data – Quality, Quantity, Thickness, Richness Rich, Thick Data
Meta-ethnography characterised by data of high methodological quality, rich data and systematic presentation.
Rich/“Thick” reports sustain Meta-Ethnography/Grounded Theory – may allow selective sampling/ theoretical saturation Poor, Thin Data
Qualitative data from “thin” studies (or textual responses to surveys) will not sustain interpretive approaches
Limited to Meta-Aggregation, Thematic Synthesis, Framework Synthesis, Meta-Summary Narrative Synthesis –type approaches NB. Is “Unit of Analysis” Individual Study (Meta-Aggregation, Thematic Synthesis) or “Body of Evidence” (e.g. Meta-Narrative or Critical Interpretive Synthesis approaches) or even Theory (Framework Synthesis/Best Fit Synthesis)?<br>
slide13. Overview of Approaches Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012 Jun 9;1:28. doi: 10.1186/2046-4053-1-28.<br>
slide14. Choice of Synthesis (Adapted from Noyes & Lewin, 2011) To aggregate/ summarise/ integrate qualitative data to address questions in relation to a specific intervention review Primarily to integrate and interpret qualitative and quantitative evidence within a single approach or integrated model. Can be used to develop explanatory theory. Thematic analysis without theory generation
Meta-aggregation
Meta-summary Product: Aggregated findings from source papers Meta-ethnography
Grounded theory
Thematic analysis with theory generation Framework synthesis Realist Review
EPPI Approach
Matrix Method
Narrative Synthesis
Bayesian Synthesis
Critical Interpretive synthesis Product: Explanatory theory, analytical or conceptual framework or interpretative framework/mechanism Best fit synthesis<br>
slide15. Some Examples<br>
slide16. The Scenario HTA Report – Systematic Review, Modelling & Qualitative Synthesis on Group Therapy for Postnatal Depression (UK)
Stevenson M, Scope A, Sutcliffe P, Booth A, et al. Group cognitive behavioural therapy for postnatal depression: a systematic review of clinical effectiveness, cost effectiveness and value of information analyses. Health Technol Assess 2010;14(44).
Scope, A., Booth, A. and Sutcliffe, P. (2012), Women’s perceptions and experiences of group cognitive behaviour therapy and other group interventions for postnatal depression: a qualitative synthesis. Journal of Advanced Nursing. doi: 10.1111/j.1365-2648.2012.05954<br>
slide17. The Choice Type of Question – Acceptability of Intervention
Extent of Description versus Interpretation – Factors Making Group Therapy More or Less Acceptable (Descriptive)
Role of Theory – No Theory – Trying to separate Group Effect from Therapy Effect
Type of Data – Very Thin Data, Small Number of Studies (n = 6) – Descriptive Case Study Accounts in Nursing Journals
Intended Output – Alongside Effectiveness Review & Cost Effectiveness Study – for Implementation
Other Considerations:
Methodological Expertise in Team – Novice Reviewer
Available Resources – Limited Time in Comparison to “Main” Review<br>
slide18. Your Choice? ST<br>
slide19. The Contribution While Group Therapy is Acceptable on Average there are some for whom it is unfavourable because:
Group Comparison – Some People are Getting Better Quicker than I am
Group Comparison – Ms. X has been Coming Here longer than I am and is still not any better
Identified Key Issue: Identification of those Most Likely to Benefit.<br>
slide20. The Scenario Aim: To analyse meaning and motivation of the Wish To Hasten Death in patients with chronic illness or advanced disease
Monforte-Royo C, Villavicencio-Chávez C, Tomás-Sábado J, Mahtani-Chugani V, Balaguer A (2012) What Lies behind the Wish to Hasten Death? A Systematic Review…. from the Perspective of Patients. PLoS ONE 7(5): e37117. doi:10.1371/journal.pone.0037117<br>
slide21. The Choice Type of Question – Explanatory
Extent of Description versus Interpretation – Factors that Help to Explain a Phenomenon (Interpretative)
Role of Theory – Theory Generation
Type of Data – Contextually Rich – Qualitative Research Reports – Thick Data [NB. 7 studies]
Intended Output – Stand Alone Research Report for Enlightenment
Other Considerations:
Methodological Expertise in Team – Large Experienced Team with Qualitative Researcher
Available Resources – Extant Funded Project<br>
slide22. Your Choice? E/M<br>
slide23. The Contribution Wish to hasten death (WTHD) is multifactorial construct with multiple meanings that do not necessarily imply genuine desire to hasten one's death or actually taking steps towards this.
Wish to hasten death (WTHD) is a phenomenon that appears, among patients in advanced stages of illness, as response to extreme suffering that affects all aspects of their human existence.<br>
slide24. The Scenario Aim: To evaluate benefits and constraints of collaborative health-related research by researchers and those affected by the issues under study and/or those who would apply research results and to examine how variations in programme context and mechanisms influence the process and outcomes of collaborative health intervention research?
Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L., Henderson, J., ... & Greenhalgh, T. (2012). Uncovering the benefits of participatory research… Milbank Quarterly, 90(2), 311-346.<br>
slide25. The Choice Type of Question – Explanatory
Extent of Description versus Interpretation – Factors that Help to Explain Variation in Outcomes
Role of Theory – Theory Exploration
Type of Data – Contextually Rich – Quantitative and Qualitative Research Reports – Thick Data [NB. Twenty-three PR partnerships described in 276 publications.]
Intended Output – Explanatory Report for Multiple Funders
Other Considerations:
Methodological Expertise in Team – Large Experienced International Team of Experienced Researchers
Available Resources – Extant Funded Project<br>
slide26. Your Choice? S/R<br>
slide27. The Contribution Used middle-range theory of partnership synergy to demonstrate how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase quality of outputs and outcomes over time, (6) increase sustainability of project goals, and (7) create system changes and new unanticipated projects and activities.
Generated new insights into benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and generation of systemic change.<br>
slide28. References - 1 Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009 Aug 11;9:59.
Booth, A, Papaioannou, D and Sutton, A J (2011). Systematic Approaches to a Successful Literature Review. SAGE publications
Candy B, King M, Jones L, Oliver S. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol. 2011 Aug 26;11:124. Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. (2004) Integrative approaches to qualitative and quantitative evidence. London: Health Development Agency
Gough, D, Oliver, S, Thomas J (2012) An Introduction to Systematic Reviews. London: Sage Publications.
Lorenc, T., Pearson, M., Jamal, F., Cooper, C. and Garside, R. (2012), The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res. Synth. Method, 3: 1–10.<br>
slide29. References - 2 Popay J, Roberts H, Sowden A, Pettticrew M, Arai L, Rodgers M, Britten N: Guidance on the conduct of narrative synthesis in systematic reviews. http://www.lancs.ac.uk/fass/projects/nssr/2007 .
Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007. Ring N., Ritchie K, Mandava L, Jepson R. (2011) A guide to synthesising qualitative research for researchers undertaking health technology assessment and systematic reviews. NHS Quality Improvement Scotland and University of Stirling, Edinburgh.
Thomas J, Harden A (2009) Methods for the thematic synthesis of qualitative research in systematic reviews, BMC Medical Research Methodology 8:45<br>
slide2. Caveat Reviewer: Pandora’s Box!<br>
slide3. Confusing Terminology, Variety of Choices Qualitative Systematic Review
Qualitative Meta-Synthesis
Qualitative Research Synthesis
Qualitative Evidence Synthesis
Qualitative Interpretive Meta-Synthesis Best Fit Synthesis
Critical Interpretive Synthesis
Framework Synthesis
Meta-Aggregation
Meta-Ethnography
Meta-Interpretation
Meta-Narrative
Meta-Study
Meta-Summary
Narrative Synthesis
Qualitative Meta-Synthesis
Realist Synthesis
19. Rapid Realist Synthesis
Thematic Synthesis<br>
slide4. Decisions, Decisions! Novice – Bewildering variety of methods of synthesis – compounded by choice of checklists, conflicting guidance etcetera
Experienced - Many authors stick to familiar methods rather than select most appropriate method to address question and type of evidence
Much description of methods, little evaluation and critique
Much hiding behind “labels”, misuse of methods<br>
slide5. Caveat Reviewer: TREAD Carefully
Time/Timeframe
Resources
Expertise
Audience & Purpose
Data – Quality, Quantity, Thickness, Richness<br>
slide6. Time/Timeframe Most Time Consuming
Meta-ethnography – interpretive approach
Realist Synthesis – Identification of theory and development of programme theories Least Time Consuming
Meta-Aggregation –aggregating existing themes
Thematic Synthesis – inductively coding themes
Framework Synthesis
Best Fit Framework Synthesis “Framework-based synthesis is an important advance in conducting reviews of qualitative synthesis. The 'best fit' strategy is a variant of this approach that may be very helpful when policymakers, practitioners or other decision makers need answers quickly, and are able to tolerate some ambiguity about whether the answer is the very best that could be given”. Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011 Apr 14;9:39.<br>
slide7. Seven steps for meta-ethnography<br>
slide8. Process of Realist Synthesis from: McCormack, B., Wright, J., Dewar, B., Harvey, G., & Ballantine, K. (2007). A realist synthesis of evidence relating to practice development: findings from the literature analysis. Practice Development in Health Care, 6(1), 25-55.<br>
slide9. Resources Most Resource Intensive Least Resource Intensive<br>
slide10. Expertise Expertise in Qualitative Research Methods (e.g. Grounded Theory; Framework Analysis, Thematic Analysis)
Expertise in Synthesis Methods (incl. Searching, Data Extraction, Quality Assessment, Interpretation)
Knowledge of Topic Area<br>
slide11. Audience & Purpose “output of some methods… (Thematic Synthesis, textual Narrative Synthesis, Framework Synthesis, and ecological triangulation) is more directly relevant to policymakers and designers of interventions than…methods with a more constructivist orientation (Meta-Study, Meta-Narrative, Meta-Ethnography, Grounded Theory, CIS)…generally more complex and conceptual” (Barnett-Page & Thomas, 2009) Thematic Synthesis (including Meta-Aggregation) and Framework Synthesis produce findings that directly inform practitioners (Thomas & Harden, 2009)
Interpretive approaches (e.g. CIS, Meta-Ethnography) produce a model that requires practitioners to interpret relevance and applicability to their own context
Narrative Synthesis or EPPI-Centre (matrix) methods may help to integrate and present quantitative/qualitative work<br>
slide12. Data – Quality, Quantity, Thickness, Richness Rich, Thick Data
Meta-ethnography characterised by data of high methodological quality, rich data and systematic presentation.
Rich/“Thick” reports sustain Meta-Ethnography/Grounded Theory – may allow selective sampling/ theoretical saturation Poor, Thin Data
Qualitative data from “thin” studies (or textual responses to surveys) will not sustain interpretive approaches
Limited to Meta-Aggregation, Thematic Synthesis, Framework Synthesis, Meta-Summary Narrative Synthesis –type approaches NB. Is “Unit of Analysis” Individual Study (Meta-Aggregation, Thematic Synthesis) or “Body of Evidence” (e.g. Meta-Narrative or Critical Interpretive Synthesis approaches) or even Theory (Framework Synthesis/Best Fit Synthesis)?<br>
slide13. Overview of Approaches Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012 Jun 9;1:28. doi: 10.1186/2046-4053-1-28.<br>
slide14. Choice of Synthesis (Adapted from Noyes & Lewin, 2011) To aggregate/ summarise/ integrate qualitative data to address questions in relation to a specific intervention review Primarily to integrate and interpret qualitative and quantitative evidence within a single approach or integrated model. Can be used to develop explanatory theory. Thematic analysis without theory generation
Meta-aggregation
Meta-summary Product: Aggregated findings from source papers Meta-ethnography
Grounded theory
Thematic analysis with theory generation Framework synthesis Realist Review
EPPI Approach
Matrix Method
Narrative Synthesis
Bayesian Synthesis
Critical Interpretive synthesis Product: Explanatory theory, analytical or conceptual framework or interpretative framework/mechanism Best fit synthesis<br>
slide15. Some Examples<br>
slide16. The Scenario HTA Report – Systematic Review, Modelling & Qualitative Synthesis on Group Therapy for Postnatal Depression (UK)
Stevenson M, Scope A, Sutcliffe P, Booth A, et al. Group cognitive behavioural therapy for postnatal depression: a systematic review of clinical effectiveness, cost effectiveness and value of information analyses. Health Technol Assess 2010;14(44).
Scope, A., Booth, A. and Sutcliffe, P. (2012), Women’s perceptions and experiences of group cognitive behaviour therapy and other group interventions for postnatal depression: a qualitative synthesis. Journal of Advanced Nursing. doi: 10.1111/j.1365-2648.2012.05954<br>
slide17. The Choice Type of Question – Acceptability of Intervention
Extent of Description versus Interpretation – Factors Making Group Therapy More or Less Acceptable (Descriptive)
Role of Theory – No Theory – Trying to separate Group Effect from Therapy Effect
Type of Data – Very Thin Data, Small Number of Studies (n = 6) – Descriptive Case Study Accounts in Nursing Journals
Intended Output – Alongside Effectiveness Review & Cost Effectiveness Study – for Implementation
Other Considerations:
Methodological Expertise in Team – Novice Reviewer
Available Resources – Limited Time in Comparison to “Main” Review<br>
slide18. Your Choice? ST<br>
slide19. The Contribution While Group Therapy is Acceptable on Average there are some for whom it is unfavourable because:
Group Comparison – Some People are Getting Better Quicker than I am
Group Comparison – Ms. X has been Coming Here longer than I am and is still not any better
Identified Key Issue: Identification of those Most Likely to Benefit.<br>
slide20. The Scenario Aim: To analyse meaning and motivation of the Wish To Hasten Death in patients with chronic illness or advanced disease
Monforte-Royo C, Villavicencio-Chávez C, Tomás-Sábado J, Mahtani-Chugani V, Balaguer A (2012) What Lies behind the Wish to Hasten Death? A Systematic Review…. from the Perspective of Patients. PLoS ONE 7(5): e37117. doi:10.1371/journal.pone.0037117<br>
slide21. The Choice Type of Question – Explanatory
Extent of Description versus Interpretation – Factors that Help to Explain a Phenomenon (Interpretative)
Role of Theory – Theory Generation
Type of Data – Contextually Rich – Qualitative Research Reports – Thick Data [NB. 7 studies]
Intended Output – Stand Alone Research Report for Enlightenment
Other Considerations:
Methodological Expertise in Team – Large Experienced Team with Qualitative Researcher
Available Resources – Extant Funded Project<br>
slide22. Your Choice? E/M<br>
slide23. The Contribution Wish to hasten death (WTHD) is multifactorial construct with multiple meanings that do not necessarily imply genuine desire to hasten one's death or actually taking steps towards this.
Wish to hasten death (WTHD) is a phenomenon that appears, among patients in advanced stages of illness, as response to extreme suffering that affects all aspects of their human existence.<br>
slide24. The Scenario Aim: To evaluate benefits and constraints of collaborative health-related research by researchers and those affected by the issues under study and/or those who would apply research results and to examine how variations in programme context and mechanisms influence the process and outcomes of collaborative health intervention research?
Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L., Henderson, J., ... & Greenhalgh, T. (2012). Uncovering the benefits of participatory research… Milbank Quarterly, 90(2), 311-346.<br>
slide25. The Choice Type of Question – Explanatory
Extent of Description versus Interpretation – Factors that Help to Explain Variation in Outcomes
Role of Theory – Theory Exploration
Type of Data – Contextually Rich – Quantitative and Qualitative Research Reports – Thick Data [NB. Twenty-three PR partnerships described in 276 publications.]
Intended Output – Explanatory Report for Multiple Funders
Other Considerations:
Methodological Expertise in Team – Large Experienced International Team of Experienced Researchers
Available Resources – Extant Funded Project<br>
slide26. Your Choice? S/R<br>
slide27. The Contribution Used middle-range theory of partnership synergy to demonstrate how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase quality of outputs and outcomes over time, (6) increase sustainability of project goals, and (7) create system changes and new unanticipated projects and activities.
Generated new insights into benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and generation of systemic change.<br>
slide28. References - 1 Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009 Aug 11;9:59.
Booth, A, Papaioannou, D and Sutton, A J (2011). Systematic Approaches to a Successful Literature Review. SAGE publications
Candy B, King M, Jones L, Oliver S. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol. 2011 Aug 26;11:124. Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. (2004) Integrative approaches to qualitative and quantitative evidence. London: Health Development Agency
Gough, D, Oliver, S, Thomas J (2012) An Introduction to Systematic Reviews. London: Sage Publications.
Lorenc, T., Pearson, M., Jamal, F., Cooper, C. and Garside, R. (2012), The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res. Synth. Method, 3: 1–10.<br>
slide29. References - 2 Popay J, Roberts H, Sowden A, Pettticrew M, Arai L, Rodgers M, Britten N: Guidance on the conduct of narrative synthesis in systematic reviews. http://www.lancs.ac.uk/fass/projects/nssr/2007 .
Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007. Ring N., Ritchie K, Mandava L, Jepson R. (2011) A guide to synthesising qualitative research for researchers undertaking health technology assessment and systematic reviews. NHS Quality Improvement Scotland and University of Stirling, Edinburgh.
Thomas J, Harden A (2009) Methods for the thematic synthesis of qualitative research in systematic reviews, BMC Medical Research Methodology 8:45<br>