Interactions between clinician and organizational characteristics as predictors of evidence-based - PowerPoint Presentation

1. Interactions between clinician and organizational characteristics as predictors of evidence-based and non-evidence based practice useEmily M. Becker-Haimes, Nathanial J. Williams, Kelsie H. Okamura,, & Rinad S. Beidas

2. OverviewBrief(!) overview of the complex literature surrounding predictors of EBP use and need for current studyMethod and analytic approach ResultsConclusions, Challenges, & Implications

3. Why care about predictors?Variability in EBP use in youth mental health servicesCurrent implementation strategies (e.g., trainings) are insufficientIdentifying predictors  designing effective implementation strategies

4. Complex Predictor LiteratureRaghavan et al., 2008

5. Examining interactions across contextual levels is an important next stepDespite theoretical acknowledgement, little work has considered interactions between predictors of EBP useImportant for answering what strategies work for whom  tailoring implementation strategies Guide future causal work to understand mechanisms and mediators of implementation processes

6. Current StudyExamining predictors of EBP use and non-EBP useEBP = Cognitive Behavioral TherapyNon-EBP = Psychodynamic Therapy Examine interactions between: Individual clinician characteristicsOrganization characteristicsHyp 1: Interactions will emerge to explain associations with practice use Hyp 2: In the absence of interactions, organizational characteristics will predict EBP use more strongly than clinician characteristics

7. Predictors ExaminedOrganizational VariablesClinician Variables Proficiency Culture (OSC Proficiency Subscale)Attitudes (EBPAS subscales)Functional Climate(OSC Functionality Subscale)Knowledge (KEBSQ)Implementation Climate (ICS Total Score)Years of experience

8. Method247 clinicians across 28 agencies M age = 38.7 (SD = 11.9)78% FemaleHeterogeneous ethnic backgroundM years of experience = 10.1 (SD = 8.6)

9. Analytic ApproachOutcomes of interest: Two-level mixed-effects regression models predicting clinician use of CBT and psychodynamic techniquesEach model contained all clinician-level predictors, a single organizational characteristic, and their interactionsInterpret main effects in absence of interactions (Hypothesis 2)

10. Results: CBT Use

11. CBT use: Clinician Attitudes (Appeal) by Organizational Proficiency Culture

12. CBT use: Clinician Attitudes (Appeal) by Organizational Functional Climate

13. CBT use: Clinician Knowledge by Organizational Proficiency Culture

14. Main effects Higher CBT use associated with:Clinician CharacteristicsGreater openness to EBPs (EBPAS Openness)Lower EBP knowledgeMore years of experience

15. Results: Psychodynamic Use

16. Psychodynamic Use: Clinician Knowledge by Implementation Climate

17. Main EffectsHigher Psychodynamic use associated with:Clinician CharacteristicsGreater openness to EBPs (EBPAS Openness)Organizational CharacteristicsNone significant

18. Putting the Puzzle TogetherWhile preliminary, findings point to the importance of specifying cross-level interactions There may be boundary conditions of the impact of organizational culture – importance of attending to multiple levels simultaneously during implementation?

19. LimitationsDependent variable issues!!Pearson r between psychodynamic and CBT use = .55!More CBT = good? EBP ≠ Kitchen sink therapy Need to better assess clinical applicability of practice use

20. Translating Research Into PracticePractice Or Policy Implications For My Setting/Stakeholders Practice Or Policy Implications Beyond My Setting/StakeholdersAttending to organizational culture and climate is important for CBT uptakeANDThere may be limits to the impact of organizational factors as a function of clinician attitudes Assessing use of a single EBP to determine implementation outcomes may overlook the complex, eclectic practices used in community settingsNeed for more attention to causal models that encompass cross-level effects to inform design of prospective trials and understanding of implementation mechanismsFidelity measurement specificity: More attention needed to understand how to best assess a clinician’s application and sequencing of interventions to understand the implementation process

21. Using Theory to Predict ImplementationAdapted from Azjen (1986, 1991) and Williams and Glisson (2013)IntentionsSelf-EfficacyNormsAttitudesSkillBeliefsBehaviorKnowledgePolicy/OrganizationalEnvironmentUsed with permission from Dr. David Mandell

22. Questions?

23. Unintended Consequences of Evidence-Based Treatment Policy ReformAlayna L. Park, M.A.1, Katherine H. Tsai, Ph.D., M.P.H.2,Karen Guan, M.A.1, Richard Border, M.A.3, Bruce F. Chorpita, Ph.D.11University of California, Los Angeles2Five Acres – The Boys’ and Girls’ Aid Society of Los Angeles3University of Colorado, Boulder

24. Recent proliferation of evidence-based treatments (EBTs)Numerous policy initiatives support the routine use of EBTs in the communityBackgroundChorpita, Bernstein, & Daleiden (2011); Cooper & Aratani (2009)

25. Background20042009PresentMental Health Services Act (MHSA)Prevention and Early Intervention (PEI) Plan50,000+ youth and families servedCalifornia Department of Mental Health (2016); Los Angeles County Department of Mental Health (2016)

26. BackgroundFiscal incentives for 32 EBTsFree trainings in 6 EBTsUse of EBTs?

27. Determine the applicability of county-supported EBTs to youth accessing community mental health servicesExplore community providers’ patterns of EBT implementation with youth and families in needStudy Aims

28. 21 community mental health providers95% female52% Latino/Hispanic, 29% Caucasian, 10% Asian, 10% mixed ethnicity90% Master’s levelAverage clinical experience: 3.86 years (SD = 4.18)38% cognitive-behavioral, 19% eclectic, 19% psychodynamic, 14% humanistic, 10% family systemsAverage # of EBTs trained: 2.55 (SD = 1.61)60 youth with anxiety, depressive, trauma, or disruptive behavior concerns5-14 years (Mean = 8.30, SD = 2.70)55% boys73% Latino/Hispanic92% annual family income below $40,000ParticipantsChorpita et al. (2017)

29. Consultation Record (Ward et al., 2013)Evidence-Based Practice Training SurveyMeasuresWhat was the session date?Was there a specific intervention that you used?___________ TF-CBT PCIT FFT CBITS Incredible Years PPP Seeking Safety CPP Other: ______________________10/4/2011Communication SkillsHave you completed formal training in an evidence-based practice? Trauma Focused Cognitive Behavioral Therapy (TF-CBT)Date: ___________ Parent Child Interaction Therapy (PCIT)Date: ___________2/21/2011

30. Determined age range and problem area that each EBT was designed to coverEBT CoverageEBTAge RangeProblem AreaParent-Child Interaction Therapy (PCIT)2 - 7Disruptive BehaviorTrauma-Focused Cognitive Behavioral Therapy (TF-CBT)3 - 18TraumaInterpersonal Psychotherapy for Depression (IPT)12 +DepressionPromoting Alternative Thinking Strategies (PATHS)5 - 12Disruptive BehaviorTriple P Positive Parenting Program (Triple P)0 - 18Conduct

31. Traditional delivery of EBTEBT that matched youth’s age and presenting problem were delivered in >50% sessionsOff-label use of EBTEBT that did not match youth’s age and presenting problem were delivered in >50% sessionsModified delivery of EBTEBT practices (e.g., problem solving, communication skills) were delivered in >50% sessionsMinimal delivery of EBTEBT practices were delivered in <50% sessionsPatterns of EBT Implementation

32. How applicable are county-supported EBTs to service-seeking youth?

33. How applicable are county-supported EBTs to service-seeking youth?

34. How applicable are county-supported EBTs to service-seeking youth?

35. Are youth receiving evidence-informed mental health care?

36. Are youth receiving evidence-informed mental health care?

37. Are youth receiving evidence-informed mental health care?

38. Are youth receiving evidence-informed mental health care?

39. Are youth receiving evidence-informed mental health care?

40. Youth matched with providers trained in relevant EBTsBut... it’s not a perfect matchMajority of sample received evidence-informed mental health servicesBut... many youth received off-label or modified use of EBTsLimitationsAssumptions were made about quality and integrity of treatment deliveryClient outcomes were not examinedDissemination of EBTs ≠ use of EBTsSummary: Key Points

41. Translating Research into PracticeSpecific practice and policy implicationsLos Angeles County PEI Plan providers and policy makers should consider:Efficiency of county-supported EBTsEfficiency of EBT training plansHow to promote evidence-informed therapy with all children and familiesGeneralizable practice and policy implicationsMental health services practice and policy initiatives should consider:Characteristics of service-seeking populationTraining of service providersTreatment options that can fill gaps in EBT coverage

42. Thank you!

43. Coordinated Knowledge Systems: Enhancing the Use of Evidence in Clinical Decision Making Kimberly D. Becker University of South Carolina Alayna L. ParkBruce F. ChorpitaUniversity of California, Los Angeles

44. Our Mission

45. No Shortage of Evidence

46. No Shortage of EvidenceThere are more than 900 RCTs in children’s mental health demonstrating the beneficial effects of over 650 interventions (PracticeWise, 2017).

47. Adapted from Graham et al. (2006)Knowledge InquiryKnowledge SynthesisKnowledge Tools and ProductsRaw research studiesTreatment manualsResearch reviewsHow Is Evidence Created and Packaged?

48. Knowledge As It Is Typically Packaged Is Not Sufficient…Thank you for the knowledge!

49. …To Answer All QuestionsIs there a problem?What type of problem?How big is the problem?Is there evidence that fits this problem? What if the evidence doesn’t fit the family?What if the family has other preferences?What if my supervisor has a different idea?What do I do if there are constraints to applying the evidence in my setting?How do I know if what I am doing is working?What do I need to do to feel competent implementing the evidence?What do I do if it is not working?

50. Adapted from Graham et al. (2006)Monitor Knowledge UseSelect KnowledgeIdentify ProblemAdapt Knowledge to Local ContextSustain Knowledge UseEvaluate OutcomesAssess Barriers to Knowledge UseSelect InterventionsACTION CYCLEKNOWLEDGE CREATIONKnowledge InquiryKnowledge SynthesisKnowledge Tools and Products

51. Adapted from Graham et al. (2006)Identify ProblemGenerate IdeasImplement IdeasEvaluateKNOWLEDGE CREATIONACTION CYCLEKnowledge InquiryKnowledge SynthesisKnowledge Tools and Products

52. Method Los Angeles Unified School District 4 supervisors, 4 clinicians each Random assignment:Engagement Knowledge SystemTraditional Resource 1 day training Screened 2 families each for low engagement Recorded supervision session, coded discussion

53. Traditional Resource

54. Knowledge InquiryKnowledge SynthesisKnowledge Tools and ProductsScreenerIdentify the problemREACHing FamiliesGenerate ideasImplement ideasEvaluate the outcomeCoordinated Knowledge System

55. Workflow Process

56. Supervision: Traditional ResourceClinician: “So, I think it will be interesting to see...if Mom is more willing to talk. And maybe she will be willing to share about what wasn’t good. Maybe she just needs to process what happened this past week and then she will be able to talk about it out loud next week.”

57. Supervision: Engagement Knowledge System Supervisor: “You have a great relationship with the caregiver. Client has been a challenge. We have looked into revisiting the relationship building with the client. So tell me, out of the practices that are suggested [through the EKS] to help with these issues of rapport building, what have you been doing differently?” The supervisor prompts discussion about ways that rapport-building practices can be adapted for use with this family. “How do you know that [the client] is responding to you in a positive way or that she likes what you are doing? How can you tell she is engaged?”

58. Supervision Focus on Engagement

59. Discussion Across Domains

60. Practice Planning Across Domains

61. Perspectives on Engagement Knowledge SystemSupervisor: “We found it so helpful in supervision. Doing the [REACHing Families Worksheet], sometimes the therapist automatically checked yes or no, but in supervision, we really walked through all of the items, and through discussion, sometimes the therapist changed her answer. So, it was really helpful to talk through.”

62. SummarySharing knowledge (traditional resource) is not sufficient to increase the use of evidence in clinical practiceA feasible, acceptable, and effective strategy is to embed evidence within a set of tools that fits within a workflow processProblem detection and specificationGeneration of ideasImplementationEvaluation

63. Translating Research Into Practice

64.

65. Translating Research Into Practice

66. Translating Research Into Practice

67. Translating Research Into Practice

68. Final CommentWe believe that we best serve the mission of helping children and families when we consider strategies for translating evidence into practice that are independent of any single EBP.

69. Thank You

70. DiscussantBruce F. Chorpita, Ph.D.