A worked example of META-AGGREGATION Its history
LD
Published · 33 slides · 0 views
1 / 1
Description
A worked example of META-AGGREGATION Its history Its philosophy A worked example Meta-aggregation A structured, and process driven approach to systematic review drawing on the classical understandings and methods associated with systematic
Related Topics
Share
Embed code
Download this presentation From Below
"A worked example of META-AGGREGATION Its history" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
01
A worked example of META-AGGREGATION Its history
Its philosophy
A worked example<br>
Its philosophy
A worked example<br>
02
Meta-aggregation A structured, and process driven approach to systematic review drawing on the classical understandings and methods associated with systematic review of quantitative literature as practiced by the Cochrane and Campbell Collaboration
Based on an a-priori protocol
Established, answerable question
Explicit criteria for inclusion
Documented review methods for searching, appraisal, extraction and synthesis of data<br>
Based on an a-priori protocol
Established, answerable question
Explicit criteria for inclusion
Documented review methods for searching, appraisal, extraction and synthesis of data<br>
03
Meta-aggregation: History Participative consensus project involving Australian academics
The outcome = an aggregative approach to the synthesis of qualitative evidence that:
emphasised the complexity of interpretive and critical understandings of phenomena; recognised the need to ensure that the process is practical and usable;
balanced utility of the outcomes with the complexity of the material;
was grounded in pragmatism.<br>
The outcome = an aggregative approach to the synthesis of qualitative evidence that:
emphasised the complexity of interpretive and critical understandings of phenomena; recognised the need to ensure that the process is practical and usable;
balanced utility of the outcomes with the complexity of the material;
was grounded in pragmatism.<br>
04
Meta-aggregation: A pragmatic approach The philosophy of pragmatism - Charles Peirce, William James, John Dewey
the value of any thought lies in its practical use and consequences. It is the focus on practical consequences that characterizes meta-aggregation as a synthesis approach
Central to pragmatism were communities of inquiry or groups of people with a shared interest, problem or issue trying to resolve doubt through critical reasoning.
Actions are assessed in the light of their practical consequences. Aggregation concentrates on the original researchers’ findings (processed data) and summarizes common and competing findings to produce cross-study generalizations that lead to recommendations for action The utility of an outcome is important. Research articles<br>
the value of any thought lies in its practical use and consequences. It is the focus on practical consequences that characterizes meta-aggregation as a synthesis approach
Central to pragmatism were communities of inquiry or groups of people with a shared interest, problem or issue trying to resolve doubt through critical reasoning.
Actions are assessed in the light of their practical consequences. Aggregation concentrates on the original researchers’ findings (processed data) and summarizes common and competing findings to produce cross-study generalizations that lead to recommendations for action The utility of an outcome is important. Research articles<br>
05
Meta-aggregation: a worked example Review Topic: Obstacles to the implementation of Evidence-Based Practice in Belgium: a context-specific study
Question formulation:
Setting: Belgium (Belgian health care system)
Population: Health Care Practitioners
(Phenomena of) Interest: Obstacles toward the implementation of Evidence-Based Practice
Comparison: (additional literature search on strategies developed by countries with other systems)
Evaluation: Experiences and perceptions<br>
Question formulation:
Setting: Belgium (Belgian health care system)
Population: Health Care Practitioners
(Phenomena of) Interest: Obstacles toward the implementation of Evidence-Based Practice
Comparison: (additional literature search on strategies developed by countries with other systems)
Evaluation: Experiences and perceptions<br>
06
Meta-aggregation: a worked example Search strategy:
Major databases: Medline, CINAHL, Psychinfo, Embase, Social Sciences abstracts and ERIC (1990-May 2008).
Where possible we used a methodological filter for qualitative research.
Other information sources:
‘Federal research actions’-database from the Belgian governmental department of science
Consultation of Belgian experts in qualitative research methods and/or Evidence-Based Practice to check on any other published material that could be of use for the synthesis.
Screening of references
Keywords:
‘Evidence-Based Practice (Medical Subject Heading (MeSH) term)’, including Evidence-Based Dentistry, Nursing and Medicine.
Additional keywords: evidence-based combined with the geographical notion ‘Belgian’, ‘Flemish/Flanders’ or ‘Walloon’.<br>
Major databases: Medline, CINAHL, Psychinfo, Embase, Social Sciences abstracts and ERIC (1990-May 2008).
Where possible we used a methodological filter for qualitative research.
Other information sources:
‘Federal research actions’-database from the Belgian governmental department of science
Consultation of Belgian experts in qualitative research methods and/or Evidence-Based Practice to check on any other published material that could be of use for the synthesis.
Screening of references
Keywords:
‘Evidence-Based Practice (Medical Subject Heading (MeSH) term)’, including Evidence-Based Dentistry, Nursing and Medicine.
Additional keywords: evidence-based combined with the geographical notion ‘Belgian’, ‘Flemish/Flanders’ or ‘Walloon’.<br>
07
Meta-aggregation: a worked example Inclusion criteria:
Study type: Qualitative, empirical research papers. Opinion pieces and descriptive articles were excluded.
Study participants: All types of health care practitioners e.g. physicians, dentists, nurses, physiotherapists, psychologists
Topic of interest: Obstacles to the implementation of EBP.
Context: the Belgian health care system.
Outcome of interest: Experiences and/or perceptions from participants.
8 studies met the inclusion criteria<br>
Study type: Qualitative, empirical research papers. Opinion pieces and descriptive articles were excluded.
Study participants: All types of health care practitioners e.g. physicians, dentists, nurses, physiotherapists, psychologists
Topic of interest: Obstacles to the implementation of EBP.
Context: the Belgian health care system.
Outcome of interest: Experiences and/or perceptions from participants.
8 studies met the inclusion criteria<br>
08
Meta-aggregation: a worked example Critical appraisal of selected studies:
No criteria based quality appraisal, although the software QARI provides a standard critical appraisal checklist.
limited amount of studies found
the majority of the selected studies were written by the lead reviewer, who would also be one of the appraisers.
An overall judgment approach was used instead. This approach has been proven to deliver the same outcome (Dixon-Woods, 2007). However, it tends to be less explicit about potential reasons for exclusion.
An evaluation based on the JBI-critical appraisal instrument is recommended for other review teams opting for a meta-aggregative approach to synthesis.<br>
No criteria based quality appraisal, although the software QARI provides a standard critical appraisal checklist.
limited amount of studies found
the majority of the selected studies were written by the lead reviewer, who would also be one of the appraisers.
An overall judgment approach was used instead. This approach has been proven to deliver the same outcome (Dixon-Woods, 2007). However, it tends to be less explicit about potential reasons for exclusion.
An evaluation based on the JBI-critical appraisal instrument is recommended for other review teams opting for a meta-aggregative approach to synthesis.<br>
09
85 different findings were found
The findings were classified in 9 major categories:
(1) evidence is hard to implement
(2) decision making processes are influenced by practitioner variables
(3) decision-making processes are influenced by practitioner variables
(4) commercial/financial interests affect EBP
(5) governmental regulations influence the process of implementation,
(6) EBP is more feasible for practitioners working in a scientific setting compared to those working in other settings
(7) physicians’ display of power hampers other professionals to put evidence-based practice into effect
(8) a lack of knowledge and skills hinders the implementation of evidence-based practice
(9) attitudes of health care practitioners hinder the implementation of EBP. Meta-aggregation: a worked example<br>
The findings were classified in 9 major categories:
(1) evidence is hard to implement
(2) decision making processes are influenced by practitioner variables
(3) decision-making processes are influenced by practitioner variables
(4) commercial/financial interests affect EBP
(5) governmental regulations influence the process of implementation,
(6) EBP is more feasible for practitioners working in a scientific setting compared to those working in other settings
(7) physicians’ display of power hampers other professionals to put evidence-based practice into effect
(8) a lack of knowledge and skills hinders the implementation of evidence-based practice
(9) attitudes of health care practitioners hinder the implementation of EBP. Meta-aggregation: a worked example<br>
10
These categories were further analysed to produce 4 synthesized statements
Synthesis 1: Evidence might have a limited role in decision-making processes in daily practice, if the importance of the scientific component is not stressed (categories 1, 2 and 3).
Synthesis 2: Aspects other than quality of care will steer the EBP agenda, if governmental regulations and economic interests are contra-productive for delivering the best possible care (categories 4 and 5).
Synthesis 3: Although EBP is intended to serve all practitioners, some health care providers will benefit less from EBP than others, if inequity issues between practitioners are not solved and support for field workers is not established (categories 6 and 7).
Synthesis 4: A lack of competences will hinder the implementation of EBP, if gaps in knowledge and skills are not being filled and efforts to change contra-productive attitudes are not undertaken (categories 8 and 9). Meta-aggregation: a worked example<br>
Synthesis 1: Evidence might have a limited role in decision-making processes in daily practice, if the importance of the scientific component is not stressed (categories 1, 2 and 3).
Synthesis 2: Aspects other than quality of care will steer the EBP agenda, if governmental regulations and economic interests are contra-productive for delivering the best possible care (categories 4 and 5).
Synthesis 3: Although EBP is intended to serve all practitioners, some health care providers will benefit less from EBP than others, if inequity issues between practitioners are not solved and support for field workers is not established (categories 6 and 7).
Synthesis 4: A lack of competences will hinder the implementation of EBP, if gaps in knowledge and skills are not being filled and efforts to change contra-productive attitudes are not undertaken (categories 8 and 9). Meta-aggregation: a worked example<br>
11
Meta-aggregation: a worked example<br>
12
Implications for practice and policy
Consider educating patients and exploring potential information channels that influence patients’ opinions to create space for well-informed decisions (synthesis 1).
Provide easy and free access to well-structured, compact and relevant information targeted to a particular discipline and consider helpdesks. Screen information, control its quality and translate it to the field (synthesis 1).
Consider updating the Belgian nomenclature and reimbursement system to bring it in line with the latest evidence (synthesis 2).
Consider incentives for those practitioners who are engaging themselves for the implementation of EBP in daily practice, to keep them motivated and to prevent from creating a negative spiral in which practitioners tend to believe that efforts to improve practice automatically lead to a loss of income (synthesis 2). Meta-aggregation: a worked example<br>
Consider educating patients and exploring potential information channels that influence patients’ opinions to create space for well-informed decisions (synthesis 1).
Provide easy and free access to well-structured, compact and relevant information targeted to a particular discipline and consider helpdesks. Screen information, control its quality and translate it to the field (synthesis 1).
Consider updating the Belgian nomenclature and reimbursement system to bring it in line with the latest evidence (synthesis 2).
Consider incentives for those practitioners who are engaging themselves for the implementation of EBP in daily practice, to keep them motivated and to prevent from creating a negative spiral in which practitioners tend to believe that efforts to improve practice automatically lead to a loss of income (synthesis 2). Meta-aggregation: a worked example<br>
13
Implications for practice and policy
Enhance communication and cooperation between physicians and other partners in health care, via their professional groups and journals (synthesis 3)
Consider programs tailored to the needs of specific disciplines and stimulate multidisciplinary education should be stimulated to create mutual understanding across disciplines (synthesis 3, 4).
Consider direct access to allied health services to increase autonomy (synthesis 3)
Integrate EBP in the basic curricula (synthesis 4). Meta-aggregation: a worked example<br>
Enhance communication and cooperation between physicians and other partners in health care, via their professional groups and journals (synthesis 3)
Consider programs tailored to the needs of specific disciplines and stimulate multidisciplinary education should be stimulated to create mutual understanding across disciplines (synthesis 3, 4).
Consider direct access to allied health services to increase autonomy (synthesis 3)
Integrate EBP in the basic curricula (synthesis 4). Meta-aggregation: a worked example<br>
14
The Qualitative Assessment and Review Instrument (QARI) Designed to provide a systematic process mirroring that taken for systematic reviews of quantitative research, whilst being sensitive to the nature of qualitative data.<br>
15
Data Extraction Reduce
The findings of many studies into a single document
Summarise
Methods
Phenomena
Findings<br>
The findings of many studies into a single document
Summarise
Methods
Phenomena
Findings<br>
16
The units of extraction in this process are specific findings and illustrations from the text that demonstrate the origins of the findings.
In QARI a finding is defined as: A conclusion reached by the researcher(s) and often presented as themes or metaphors.<br>
In QARI a finding is defined as: A conclusion reached by the researcher(s) and often presented as themes or metaphors.<br>
17
Levels of credibility
Unequivocal - relates to evidence beyond reasonable doubt which may include findings that are matter of fact, directly reported/observed and not open to challenge
Credible - those that are, albeit interpretations, plausible in light of data and theoretical framework. They can be logically inferred from the data. Because the findings are interpretive they can be challenged.
Not Supported - when 1 nor 2 apply and when most notably findings are not supported by the data<br>
Unequivocal - relates to evidence beyond reasonable doubt which may include findings that are matter of fact, directly reported/observed and not open to challenge
Credible - those that are, albeit interpretations, plausible in light of data and theoretical framework. They can be logically inferred from the data. Because the findings are interpretive they can be challenged.
Not Supported - when 1 nor 2 apply and when most notably findings are not supported by the data<br>
18
Meta-Synthesis The aim of meta-synthesis is to assemble findings; categorise these findings into groups on the basis of similarity in meaning; and to aggregate these to generate a set of statements that adequately represent that aggregation. These statements are referred to as synthesised findings - and they can be used as a basis for evidence based practice.<br>
19
In QARI, a synthesised finding is defined as an overarching description of a group of categorised findings that allow for the generation of recommendations for practice.<br>
20
Can be stated propositionally as “if-then” statements - for example: “If students are advised reconsider their choice of study , their relatives will sometimes feel as if they are not involved”. (a somewhat awkward and eccentric form)
the declamatory form that emphasizes the probability of the claim: “Relatives of students that are advised to reconsider their choice of study may feel as if they are not involved if strategies to include them are not pursued”.<br>
the declamatory form that emphasizes the probability of the claim: “Relatives of students that are advised to reconsider their choice of study may feel as if they are not involved if strategies to include them are not pursued”.<br>
21
Should look familiar!<br>