Greg Ogrinc Louise Davies Paul Batalden Frank Davidoff David Stevens D isclosures No financial conflicts The SQUIRE evaluation and revision process were supported by generous contributions from the Health Foundation and the Robert Wood Johnson Foundation ID: 561186
Download Presentation The PPT/PDF document "Daisy Goodman," is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Daisy Goodman,
Greg
Ogrinc, Louise Davies, Paul Batalden, Frank Davidoff, David StevensSlide2
Disclosures
No financial conflictsThe SQUIRE evaluation and revision process were supported by generous contributions from the Health Foundation and the Robert Wood Johnson Foundation.Slide3
Objectives
Describe the importance of sharing your improvement work through the published literatureDescribe the process through which the SQUIRE guidelines were developed and revised
Identify key components of the Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) that distinguish academic, scholarly QI writing Walk through the process of preparing a manuscript for publicationSlide4
Why publication guidelines?
What’s different about SQUIRE?Slide5
Evolution of
SQUIRE: an improvement journey
1st iteration of SQUIRE was produced by experts Both the evaluation and revision process included end-users in evaluating and pilot testing through an iterative processSlide6
SQUIRE 2.0 Development
Evaluation of the initial SQUIRE guidelines (SQUIRE 1.0, 2008)
Assess usability and claritySemi-structured interviews / focus groups with 29 end usersInput from 18 experts (editors, researchers, improvers)Early revisions of versions 1.2 and 1.4Two consensus conferences (Nov 2013 & Nov 2014)Pilot testing of version 1.6 with late revisions44 authors used interim draft to write sections of a manuscriptProvided feedback on utility and understandability of the draft guidelinesSemi-structured interviews with 11 journal editorsVersion 1.8 sent to over 450 individuals around the worldDavies, Louise, et al.
"The SQUIRE Guidelines: an evaluation from the field, 5 years post release." BMJ quality & safety (2015): bmjqs-2015.
Davies, Louise, et al.
"Findings from a novel approach to publication guideline revision: user road testing of a draft version of SQUIRE 2.0."
BMJ quality & safety (2015): bmjqs-2015.Slide7
SQUIRE 2.0 PublicationsSlide8
Standards for
QUality Improvement Reporting Excellence v 2.0
Provides a framework for writing and thinking about improvement workFamiliar IMRAD format Describes the essential elements to include in a reportFrequently used in planning improvement work- although not the intended purpose of a publication guidelineThe companion E&E explores the meaning of each guideline item in more depthwww.squire-statement.orgSlide9
Components of SQUIRE 2.0:
Title and Abstract
Title and AbstractTitle
Indicate that the manuscript concerns an initiative to improve healthcare (broadly defined to include the quality, safety, effectiveness, patient-centeredness, timeliness, cost, efficiency, and equity of healthcare)
Abstract
Provide adequate information to aid in searching and indexing
Summarize all key information from various sections of the text using the abstract format of the intended publication or a structured summary such as: background, local problem, methods, interventions, results, conclusionsSlide10
Introduction
Introduction
Why did you start?3. Problem Description
Nature and significance of the local problem
4. Available knowledge
Summary of what is currently known about the problem, including relevant previous studies
5. Rationale
Informal or formal frameworks, models, concepts, and/or theories used to explain the problem, any reasons or assumptions that were used to develop the intervention(s), and reasons why the intervention(s) was expected to work
6. Specific
aims
Purpose of the project and of this report Slide11
Methods….
Methods
What did you do?7. Context
Contextual elements considered important at the outset of introducing the intervention(s)
8. Intervention(s
)
Description of the intervention(s) in sufficient detail that others could reproduce it
Specifics of the team involved in the work
9. Study
of the Intervention(s)
Approach chosen for assessing the impact of the intervention(s)
Approach used to establish whether the observed outcomes were due to the intervention(s)Slide12
…More about Methods
10. Measures
Measures chosen for studying processes and outcomes of the intervention(s), including rationale for choosing them, their operational definitions, and their validity and reliabilityDescription of the approach to the ongoing assessment of contextual elements that contributed to the success, failure, efficiency, and cost Methods employed for assessing completeness and accuracy of data
11. Analysis
Qualitative and quantitative methods used to draw inferences from the data
Methods for understanding variation within the data, including the effects of time as a variable
12. Ethical Considerations
Ethical aspects of implementing and studying the intervention(s) and how they were addressed, including, but not limited to, formal ethics review and potential conflict(s) of interest
Methods
What did you do?Slide13
Results
Results
What did you find?13. Results
Initial steps of the intervention(s) and their evolution over time (
e.g., time-line diagram, flow chart, or table), including modifications made to the intervention during the project
Details of the process measures and outcome
Contextual elements that interacted with the intervention(s)
Observed associations between outcomes, interventions, and relevant contextual elements
Unintended consequences such as unexpected benefits, problems, failures, or costs associated with the intervention(s).
Details about missing data Slide14
Discussion
Discussion
What does it mean?14. Summary
Key findings, including relevance to the rationale and specific aims
Particular strengths of the project
15. Interpretation
Nature of the association between the intervention(s) and the outcomes
Comparison of results with findings from other publications
Impact of the project on people and systems
Reasons for any differences between observed and anticipated outcomes, including the influence of context
Costs and strategic trade-offs, including opportunity costsSlide15
Limitations and Conclusions
16. Limitations
Limits to the generalizability of the workFactors that might have limited internal validity such as confounding, bias, or imprecision in the design, methods, measurement, or analysisEfforts made to minimize and adjust for limitations
17. Conclusions
Usefulness of the work
Sustainability
Potential for spread to other contexts
Implications for practice and for further study in the field
Suggested next steps
Other information
18. Funding
Sources of funding that supported this work. Role, if any, of the funding organization in the design, implementation, interpretation, and reportingSlide16
Key concepts introduced in
SQUIRE 2.0
RationaleInformal or formal frameworks, models, concepts, and/or theories used to explain the problem reasons or assumptions that were used to develop the intervention(s) reasons why the intervention(s) was expected to workNot the method used for the work (e.g., lean or six sigma)Context“All the things that are not your intervention” Everything that affects your workStudy of the intervention(s)Reflecting upon and evaluating the work done Approach chosen for assessing the impact of the intervention(s)Approach used to establish whether the observed outcomes were due to the intervention(s): Did
things get better for the reasons you thought?Were there unintended consequences?Slide17
The Evolution of “Context”
“Nothing
exists, and therefore can be understood, in isolation from its context” -Paul Bate Although previously described as the “setting” for an interventionWe now see context more ecologicallyAs intervention and context interact, both change and adaptSlide18
Examples of Contextual Factors: Systems Level
External environment
External motivatorsSponsorshipOrganizationImprovement leadershipSenior leader as sponsorSupportive cultureMaturity of organizational improvement Physician payment structureIs the Problem /Project seen as strategically important to the organization?Improvement support and capacityData infrastructureResource availabilityWorkforce focused on improvementMicrosystemImprovement leadershipCulture supportive of improvementStaff training in improvementMotivation for changeKaplan, et al, 2012Slide19
Improvement team
Team diversityPhysician involvementSubject matter expert
Team tenurePrior improvement workTeam leadershipTeam decision making processTeam improvement skillsWhat about patient level factors? Kaplan, et al, 2012Examples of Contextual Factors: Intervention LevelSlide20
Methods
What did you do?
7. ContextContextual elements considered important at the outset of introducing the intervention(s)
10. Measures
Description
of the approach to the ongoing assessment of contextual elements that contributed to the success, failure, efficiency, and cost
Results
What did you find?
13. Results
Contextual
elements that interacted with the intervention(s
)
17. Conclusions
Usefulness of the work
Sustainability
Potential for spread to other contexts
Implications for practice and for further study in the field
Suggested next steps
Context is present throughout SQUIRE 2.0Slide21
Why is it important to describe context in writing about improvement?
S
ometimes improvement efforts work and sometimes they don’tContext is likely to be part of the reason your work succeeded or failedIf you can describe it, you can learn from it- and help others be successful What we discover from studying and reporting about context may be broadly relevant, even if a specific intervention cannot be transplanted from one context to anotherSlide22
Writing about Context
Ask
yourself- Why did this happen the way it did? Why didn’t it happen the way I thought it would? Direct ObservationAttitudes and behaviors- what are they telling us?Day to day workflowMinutes of meetingsField notesSafety culture survey, Staff engagement surveySemi-structured interviews/focus groupsReflect on and document the process of your improvement work Victories and failuresAdjustments made along the way (and why)Memorable events and conversationsConcurrent organizational eventsSlide23
Field notes: improvement process
Courtesy of Grey Ogrinc, MDSlide24
Writing about Context
When describing
your results:Describe the contextual factors that interacted with your intervention(s) in an important wayWhen interpreting your results:Consider reasons for the difference between the outcomes that occurred, and what you initially thought would happenHow did contextual factors influence this? It’s too much to report everything… report the unexpected, and what’s crucial to telling the storySlide25
Take Home Points about Context
Context can make or break an improvement program
Consider contextual factors carefully from the beginningOver time, the impact of contextual factors became clear Interventions are often re-designed as a result of interacting with contextShare what you learnProvide specific information about important features of your context Describe how contextual factors interacted with your intervention(s) Describe how intervention(s) and context changed as a resultRead with an eye for how other writers communicate about context effectively Slide26
Table work: The influence of context
Review the methods and results section of this paper.
Discuss with your neighbor:What do the authors tell you about context of their interventions? How did contextual factors impact the results of their interventions? How did contextual factors change the interventions?J. Pediatrics 2015; 135(1)Slide27
The setting for the intervention…
Teaching hospital
Suburban settingDedicated pediatric EDSpecifics about the physician team that provided ER coverage What could be added to this description?More about context!METHODSSetting“UNC Hospitals is a large, suburban,academic medical center …Tertiary pediatric care, including pediatric and neonatal level IV intensive care, level I trauma care, and emergency care, are provided. Pediatric emergency medicinephysicians provide care in thepediatric ED between 8:00 AM and2:00 AM. During the remaining hours, board-certified emergency medicine physicians in the general ED care for pediatric cases.”Slide28
C
hanges made as a result of contextual factors
RESULTSIn phase II, strategies focused on improving performance by providing data and other information for learning…A key intervention implemented during phase II as the result of this engagement was to develop placards to address access and language barriers…CONTEXTUAL FACTORSFactors that interacted with the intervention(s)Multi-lingual communityLack of staff knowledge about the problemStaff engagement was also addressed through incentives Slide29
Phase 1 interventions were not effective
RESULTS
“During phase I, the existing processes for identifying and managing febrile patients with central lines were mapped and analyzed. Key interventions [were] tested and implemented… Phase I interventions alone had little effect on timeliness.” Why was there a difference between observed and anticipated outcomes?Slide30
The influence of context on the intervention(s)
RESULTS
“In phase III…. A new patient identification and initial management process was designed based on the steps, weaknesses, and challenges identified ...This process benefited from feedback from frontline ED staff and the results of multiple PDSA cycles during phases I and II”The intervention changed as a result of interaction with contextual factorsFeedback from frontline staff was essential to understand PDSA cycles were used to adapt Slide31
Rationale
“Informal
or formal frameworks, models, concepts, and/or theories used to explain the problem, any reasons or assumptions that were used to develop the intervention(s), and reasons why the intervention(s) was expected to work”Always present, whether acknowledged or notImportant to make this explicit, to facilitate understanding about the mechanism of action of the intervention, and how it might vary across contextsMay be a formal model….
Michie, et al. , 2014Slide32
Rationale: Example
…or an informal theory or assumption
Brady, 2015Slide33
Table work: Rationale
Review the
background and methods section of the paper in front of you:What were the informal or formal frameworks, models, concepts, and/or theories used to explain the problem? What were the reasons or assumptions used to develop the intervention?Why did the authors think the intervention would work?Slide34
Study of the Intervention
Approach
chosen for assessing the impact of the intervention(s)Including its unintended consequencesMay be qualitative or quantitativeApproach used to establish whether the observed outcomes were due to the intervention(s)Informed by the rationale for the interventionShould include assessment of the internal and external validity of the interventionSlide35
Table work: Study of the Intervention
How did the authors assess the impact of
their interventions?How did the authors show that the observed outcomes were due to the interventions?Slide36
Using Statistical Process Control to Assess the impact of the intervention(s)
Jobson, et al 2014Slide37
SQUIRE Summary
P
rovide a framework for reporting new knowledge about how to improve healthcareIntended for reports that describe system level work to improve the quality, safety, and value of healthcare, and used methods to establish that observed outcomes were due to the intervention(s)May be adapted for reporting a range of approaches for improving healthcareAuthors should consider every SQUIRE item, but it may be inappropriate to include every one in a particular manuscript. The Explanation and Elaboration document provides specific examples of well-written SQUIRE items with an explanation.Please cite SQUIRE when it is used to write a manuscript. Slide38
Getting published…..Slide39
Why is it important to write
about improvement work?
Personal IncentivesPublicity administrative support for your projectBuilds credibility for “QI”Getting and renewing fundingBuilding morale for your teamNetworking/career building“Publish or perish”Benefits to the fieldReport new approachesIncrease transparencyShare lessons learnedWhat workedWhat didn’t workInteractions with contextContribute to general knowledge about improving health care
A few reasons (among many)…Slide40
Why is writing about improvement work so hard?
Improvement work
is more complex to write about than biomedical research Boundaries may be ill definedIntervention protocols may have multiple iterations Context changes interventionsReporting about methods and context may be more important than the results achievedThere is much to learn from an “unsuccessful” projectFrontline clinicians rarely have protected time Especially true for nursesWriting about improvement work requires changing perspectives, from “doing” to observing and reflectingEditors may not understand quality improvement or want to publish itSlide41
Table work:
5 minutes of reflection about your writing process
(borrowed from David Stevens)Do you find writing fun?Which aspects of writing do you find to be fun?What keeps you from writing?Why do you want to write?How frequently do you actually sit down and write for scholarly purposes?How often do you wish you sat down to write?What prevents you from writing as often as you would like?What is one thing you think might help you increase the amount of time you spend writing?http://www.ihi.org/education/IHIOpenSchool/resources/Pages/Publications/GuidedReflectionOnYourWriting.aspxSlide42
Debrief
How did it feel to talk to someone about this?
What are the best things about writing? The worst?What is the biggest barrier?Lack of time for writing is generally the most difficult barrier to overcomeWriting is a process which requires focused attention- a systematic approach is necessary!SQUIRE can be used as a blueprint for both planning and writingSlide43
Process map for journal submission
I
deaIs it worth writing?Which audience?Literature review Which Journal?Which format?
Can I use SQUIRE?Outline & feedback
1st draft
More Feedback
2nd draft
More Feedback
Recheck style guide
Final draft
Cover letter, disclosure etc.
Submit
PaperSlide44
Getting started
Questions to
answer before starting writingWhat do I have to say?Is this paper worth writing? Has it been written already? (The “so-what?” test)Who do I want to write it with?What is the right format? What is the right audience for the message? (The “who-cares?” test)What journal should I target? Huth, E. Writing and Publishing in Medicine (3rd ed). 1999Slide45
The “so-what?” test
Description of the problem
(Introduction: problem description/available knowledge)What you did about it (Methods: context/intervention)Description of what happened (Results: interaction between context and intervention) Why it is important (Discussion)SQUIRE elements can help structure your thinking at this stageSlide46
Co-authorship
Co-authors must meet
4 criteria (ICMJE)“Make substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; ANDDrafting the work or revising it critically for important intellectual content; ANDFinal approval of the version to be published; ANDAgreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.” http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.htmlSlide47
The “who cares?” test
Who needs to know about your work? To whom is it relevant?
Choose a journal that speaks to that audience…AND publishes quality improvement workJournal selectionDoes the journal publish improvement work?What has this journal published on your topic in the past?What type of papers does this journal accept? Review the “Instructions for Authors”Do they offer pre-submission review?Which is the right format for your message?Should I use SQUIRE? Slide48
Choosing wisely
Journals publishing SQUIRE 2.0
Am. J. Critical Care Permanente JournalJ. Surgical ResearchJ. Continuing Education in NursingAm. J. Medical QualityCanadian J. DiabetesJ. Nursing Care QualityJoint Commission Journal J. American College of SurgeonsBMJ Quality and SafetyJournals publishing QI reportsHealth AffairsHealth care: J. of Delivery ScienceImplementation ScienceInternational J. for Quality in Health CareJ. for Healthcare QualityJ. of Clinical Outcomes MgmtJ. of Healthcare Risk MgmtPLoS OneQuality Mgmt in Health CareJ. Nursing Care MgmtNEJMJAMASlide49
Process map for journal submission
idea
Is it worth writing?Which audience?Literature reviewWhich Journal?Which format?
Can I use SQUIRE?
Outline & feedback
1st draft
More Feedback2nd draft
More feedback
Recheck style guide
Final draft
Cover letter, disclosures (etc.)
SubmitSlide50
Process map for journal submission
idea
Is it worth writing?Which audience?Literature reviewWhich Journal?
Which format?
Can I use SQUIRE?
Outline & feedback1st draft
More Feedback
2
nd
draft
More feedback
Recheck style guide
Final draft
Cover letter, disclosures (etc.)
SubmitSlide51
Help from others
T
he purpose of writing is to communicate about your work- so don’t wait until after the paper is writtenTake advantage of opportunities for posters or oral presentationsGet feedback early and often!Seek out peers and mentors to review sections of your paperPeer reviewKeeps your writing movingImproves quality of communicationLearn by providing the same assistance to othersMentorshipInvolve mentors from the planning stage onAllows you to pilot test your message Expert adviceA “friendly enemy”Slide52
Final Steps
idea
Is it worth writing?Which audience?Literature reviewWhich Journal?
What format?Can I use SQUIRE?
Outline
1st draft
More feedback2
nd
draft
More feedback
Recheck style guide
Final draft
Cover letter, disclosures (etc.)
SubmitSlide53
What happens next?
Accepted with revisionsRevise and resubmitNot acceptedSlide54
Clarity and brevity require self-discipline
“Je
n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte. ““I have made this longer than usual because I have not had time to make it shorter.”Blaise Pascal (1657)Slide55
Summary
Publishing quality improvement work is essential
For transparency To share new approachesTo build general knowledge Writing about quality improvement is different than writing about researchFrontline clinicians are most likely to write during nights and weekendsSQUIRE is designed to guide and facilitate the writing processWriting is about communicating your work to others: don’t do it alone! Slide56
Breaking news!
SQUIRE website:
www.squire-statement.orgUpcoming writing conference in US and UK: fall, 2016Slide57
November 15 - 16, 2016
Hanover Inn at Dartmouth Hanover, NH.LEARN HOW TO PUBLISH YOUR QUALITY IMPROVEMENT WORKSECOND ANNUAL SQUIRE WRITING CONFERENCEwww.squire-statement.orgSlide58
Bamberger Re. Perspectives on context. The health foundation,2014
Brady PW, Zix J, Brilli
R, et al. Developing and evaluating thesuccess of a family activated medical emergency team: a qualityimprovement report. BMJ Qual Saf 2015;24:203–11Kaplan, et al, 2012 The Model for Understanding Success in Quality (MUSIQ) BMJ Qual Safety 2012 21:13-20Michie S, Atkins L, West R. The behaviour change wheel—aguide to designing interventions. London: SilverbackPublishing, 2014