interventions Challenges and opportunities Simon French PhD MPH BAppSc Chiro Associate Professor Canadian Chiropractic Research Foundation Professorship in Rehabilitation Therapy Faculty of Health Sciences ID: 1043102
Download Presentation The PPT/PDF document "Developing and evaluating knowledge tran..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Developing and evaluating knowledge translation interventions:Challenges and opportunitiesSimon French PhD, MPH, BAppSc(Chiro)Associate ProfessorCanadian Chiropractic Research Foundation Professorship inRehabilitation Therapy, Faculty of Health SciencesQueen’s Universitysimon.french@queensu.ca
2. Declarations of interestNo conflicts of interestAcknowledgements:Canadian Chiropractic Research FoundationAustralian National Health and Research CouncilMany colleagues and collaborators
3. OutlineMethods for developing KT interventionsEvaluating KT trialsProcess evaluationOutcomemeasurement
4. Early days…Research must be actively pursued and developed and as fast as new knowledge is acquired it must be appliedCommonwealth Minister for Health,William (Billy) Hughes, 1936
5. All breakthrough, no follow throughFor every dollar allocated to develop breakthrough treatments, one cent is allocated to ensure that patients actually receive them1Much of the US$240 billion/year worldwide investment in biomedical and healthcare research is wasted due to implementation failures21. Woolf Washington Post 20062. Macleod et al Lancet 2014
6. Evidence-to-practice gaps for the management of MSK conditionsOsteoporosisOsteoarthritisLow back painBuchbinder et al. Nat Rev Rheumatol 2015;11:597-605
7. Overcoming the problem? => Knowledge TranslationEnsuring stakeholders* are aware of, and use, research evidence to inform their health and healthcare decision-makingEnsuring practice is informed by current available evidence and the experiences and information needs of stakeholders*stakeholders include:healthcare professionals consumers of health care (i.e. patients, family members, carers)policy makerseducatorsresearch fundersresearchersGrimshaw et al, Implement Sci 2012;7:50
8. Does this KT intervention work?Seemingly simple questionswhether a KT intervention can work, did work, or is working?Leads to complex questions…
9. So…More careful planning needed…
10. UK Medical Research Council:Developing and evaluating complex interventionsCraig et al: Developing and evaluating complex interventions:the new Medical Research Council guidance. BMJ 2008;337:a1655
11. Practical considerationsTheoryEvidenceUnderstand behaviourChange behaviourDefine behaviourDescriptive research:interviews, surveyEvaluative research:Design and test targeted, theory-informed intervention in cRCTProcess evaluationKT intervention development & evaluationFrench et al Implement Sci 2012;7:38
12. Trial analysis & interpretationRecruitment & RandomisationInterventionOutcome measurementDefinitive trial phaseEconomic evaluationIntervention development phaseDescribing and understanding practiceComparatorProcess evaluationOutcome measurementProcess evaluationTypical KT intervention trial design
13. Process evaluation in KT trialsStudies run in parallel to, or follow, intervention trials1Understanding of: Why? How? Under what conditions?=> ‘‘black box’’ of evaluating effectiveness of KT interventionsCan be used to assess2,3:KT intervention fidelityExplore causal mechanisms Identify contextual factors associated with variation in outcomesGrant et al, Trials 2013;14:15Craig et al, BMJ 2008;337:a1655Moore et al, BMJ 2015;350:h1258
14. UK MRC guidance for process evaluationKT interventionand its causalassumptionsMoore et al: Process evaluation of complex interventions: Medical Research Council guidance BMJ 2015;350:h1258OutcomesCONTEXTImplementationMechanisms of impact
15. Process evaluation:KT intervention fidelityWhat if KT intervention fidelity not assessed?Fidelity particularly important for KT interventionsNo accepted framework/method for assessing KT intervention fidelityMichie et al Implement Sci 2009;4:40Slaughter et al Implement Sci 2015;10:129
16. IMPLEMENT interventionDelivered by trained facilitators in a 2-session workshopAcross all workshop sessions, there were 189 instances when BCTs were planned to be deliveredFrench et al, PLoS One 2013;8(6):e65471
17. French et al. BMJ Open 2015;5(7)Study designComparison between planned and actual, and observed versus self-assessed, delivery of behaviour change techniques (BCTs)ProcedureEach workshop session was audio-recorded and transcribed verbatim- Observed adherence- Facilitator self-reported adherence
18. IMPLEMENT fidelity: Results75% of GPs assigned to the intervention group attended both sessions of the workshopOverall observed adherence to BCTs overall was 79%No significant difference in adherence to BCTs between different facilitators χ2 test (df=5) = 9.7, p=0.08=> confident that trial results were due to an intervention delivered as planned
19. Outcome measurement in KT trials
20. KT intervention outcomes are complexDifferent stakeholders and knowledge users prioritise different outcomesThe “levels” of those outcomes vary:Individual patient level outcomesHealth measures; health behavioursPractitioner level outcomesOrganisational level outcomes
21. PARTNER logic modelFamily doctor intervention componentsEducationAudit & feedbackElectronic decision supportCare support teamTailored behaviour change techsOther health professional directed intervention componentsPatient behaviourslose weightregular exercisemanage painImproved patient outcomesOutcomes:System level, practitioner level, or patient level?
22. Hypothesised determinants of behaviour changeBeliefs about capabilitiesBeliefs about consequencesSocial influencesProfessional roleKnowledgeIntention (vignettes)X-ray referralAdherence to CPG rec’sX-ray referralLBP-specific disabilityPain severityClinicianbehaviourPatientoutcomesMcKenzie et al, Implement Sci 2010;5:86
23. Beliefs about capabilities0.40 (0.20, 0.59) p<0.001Beliefs about consequences0.60 (0.31, 0.90) p<0.001Social influences0.50 (0.26, 0.74) p<0.001Professional role0.42 (0.17, 0.68) p=0.001Knowledge1.73 (0.68, 4.44) p=0.25Intention (vignettes)X-ray referral0.27 (0.17, 0.44)p<0.001Adherence to CPG rec’sX-ray referral1.40 (0.51, 3.87)p=0.514LBP-specific disability0.37 (-0.48, 1.21)p=0.393Pain severity-0.10 (-0.48, 0.28)p=0.606Hypothesised determinants of behaviour changeClinicianbehaviourPatientoutcomes
24. Challenges for KT intervention outcome measurementWhich level?Does intention predict behaviour?Lack of reliable and valid measurese.g.: x-ray ratesPatient reported:Baseline = 25%Follow up = 15%Practitioner reported: 3%File audit: 10%
25. Future research direction:Outcome measurement in KT trialsChoose outcome/process measures that stakeholders/KUs wantUse valid and reliable measures (where possible…)Very few are psychometrically validated1More KT science methods development neededKT science an emerging field, lots of current activity2,3Rabin et al Implement Sci 2016;11:42Lewis et al Implement Sci 2015;10:2Martinez et al Implement Sci 2014;9:118
26. I know times are changingIt’s time we all reach out for something newThat means you too!Prince, Purple Rain: 1984
27. Implications for evaluation of KT interventions:…going beyond PICOPopulation: Patients, health practitioners, decision makersIntervention: What is the intervention? Delivered at what level?Comparison: Usual care or another KT intervention?Outcomes: Whose outcomes? At which level - health, provider or organisational? Are they valid and reliable?Context: How do we assess the mediating effect of context?Process evaluation: What do we wish to know and how will we determine this?Theory: allows predictors, moderators, and mediators to be determined
28. ConclusionKnowledge translation needs to be informed by implementation researchMore rigorous approach required for the development and evaluation of KT interventionsProcess evaluation: will contribute substantially to the knowledge base in this fieldOutcome measures: more methods development needed to produce reliable and valid measures
29. Merci!Thank you for listeningKnowing, Sharing, Doing: 1st National Knowledge Translation Conference in Rehabilitation, May 2016