Research in Surgical Education Hilary Sanfey MB.
Description: Research in Surgical Education Hilary Sanfey MB. BCh. MHPE. FACS Professor of Surgery Vice Chair for Surgical Education Southern Illinois University, Springfield Disclosures I have no relevant financial relationships to disclose pertinent
Related Topics
Download Presentation
"Research in Surgical Education Hilary Sanfey MB." is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Research in Surgical Education Hilary Sanfey MB. BCh. MHPE. FACS
Professor of Surgery & Vice Chair for Surgical Education
Southern Illinois University, Springfield<br>
slide2. Disclosures I have no relevant financial relationships to disclose pertinent to the content of my presentation.<br>
slide3. Overview What is surgical education research?
Identifying & avoiding the pitfalls
Academic success as surgeons performing research in surgical education<br>
slide4. Research in Education - Criticisms Opportunistic
Reproduce what has already been done
Do not add new knowledge.
Lack a literature review
Inappropriate methodologies
Albert M et al. Adv in Health Sciences Educ. 2007
40% published abstracts lack essential elements
Cook D et al. Med Educ 2007<br>
slide5. Research in Surgical Education A scholarly approach is “one that demonstrates evidence of drawing from and building on the work of others, and contributing work through public display, peer review and dissemination; both involve engagement with the community of educators”.
Consensus Conference on Educational Scholarship convened by the Association of American Medical Colleges (AAMC) Group on Education Affairs (GEA). Simpson D et al. Med Educ 2007<br>
slide6. Research in Surgical Education “Systematic process of defining, testing & analyzing a problem”
Identify problem & research question
Define independent & dependent variables
Review literature
Research design (methods & collaborators)
IRB
Funding
Dissemination<br>
slide7. Identify a Problem - FINER Significance, rationale
Feasible, Interesting, Novel, Ethical, Relevant
Breadth & complexity,
Resources, time, measurement
Narrow the focus<br>
slide8. Example<br>
slide9. Literature Review Is this an original question?
Has it been answered in other disciplines?
Has it been answered in other learners?
Harvard Business Review
New York Times
Pubmed /Ovid / Medline
ERIC/ EBSCO<br>
slide10. Evaluating the Education Literature What groups are being compared?
Confounding factors?
Is comparison based on risk factor or outcome?
Is there any bias?
Sample size / p value?
Literature inclusion / exclusion criteria ?
Generalizable?
Conclusions justified?<br>
slide11. Research Design Involve a statistician early in the process to ensure appropriate sample size, effect size, reliability , validity<br>
slide12. Research Design : Purpose Exploratory
Seek to uncover important questions & variables prior to establishing formal hypotheses.
Descriptive
Seek to characterize & measure sets of variables about people or phenomena.
Analytic
Seek to determine if there are relationships between specific groups.
Bordage G et al Med Educ 2003<br>
slide13. Research Design : Time Frame Retrospective
Begins with outcome & searches for prior existing variables to explain the outcome
Cross – sectional
Measures two or more variables to establish a relationship, therefore one must precede the other
Prospective
Follows subjects forwards in time to measure an outcome
Bordage G et al Med Educ 2003<br>
slide14. Research Design: Role of Observer Henry R in Guide for Researchers in Surgical Education 2010<br>
slide15. Research Design: Collaborators Establish authorship & responsibilities early!
Challenges to research “ownership”
Role confusion & power differential
Long term projects
“Abstract first” approach
Research hand off
Culture of “gifting” authorship
Hauge L in Guide for Researchers in Surgical Education 2010<br>
slide16. Research Design: Collaborators For authorship, most journals require contribution in each of the following:
Conception & design, or acquisition of data, or analysis and interpretation of the data
Drafting the manuscript or revising it critically for important intellectual content
Approval of the version to be published
International Committee of Medical Journal Editors http://www.icmje.org/urm_main.html<br>
slide17. IRB Do I need IRB approval?
Research means “a systematic investigation including research development, testing & evaluation designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for another purpose”
45CFR46 http://ohsr.od.nih.gov/guidelines/index.html<br>
slide18. IRB The IRB can be your friend
IRBs differ
Even if you know you are exempt you must submit an application for an exemption
Being exempt does not mean that you don’t need to worry about consent, or provide access to study results
Gaining consent from subjects is worth the time investment<br>
slide19. Funding & Grant Writing 30% of studies that received funding, received about half actual cost
Reed D JAMA 2005
Identify projects previously funded by agency
Read requirements & provide exactly what is requested
Abstract / summary
Background/ significance / preliminary work/ hypothesis / aims/ design/ methods /
Budget
Resources / CV<br>
slide20. Dissemination - Guide<br>
slide21. Tips - Survey Design Need to know versus nice to know!
Engage the respondent
Short and focused
Format
Introductory statement + consent
Basic demographic questions
Essential questions
Comment box
Concluding statement<br>
slide22. Tips - Survey Design Language
Keep language consistent
Instructions
Brief and clear for each question
Sequencing
Cluster by theme and format
Pilot Test
Need to know denominator to calculate a response rate!<br>
slide23. Tips: Kirkpatrick Evaluation Framework Kirkpatrick, D. L. In Training and Development Handbook New York: McGraw Hill. 1967<br>
slide24. Academic Success - Resources Statistician with expertise in educational research
PhD in education
Qualitative research methods
Research assistants
Graduate students
Nurse educators/coordinators
Residents<br>
slide25. Academic Success Job description should reflect educational effor
Double dip
Each educational innovation has potential for research & publication
Find a mentor
Improve your skills
Masters degree/ Workshops/Literature in education
Be visible
Offer to write critiques / speak / moderate
Collaborate outside surgery<br>
slide26. Additional Resources The following slides contain lists of resources specific to health professions educators<br>
slide27. Professional Development ACS Surgeons as Educators Course
http://www.facs.org/education/sre/surgedbrochure.pdf
Association for Surgical Education Workshops
http://www.surgicaleducation.com/mc/page.do?sitePageId=31641&orgId=ase
ASE Surgical Education Research Fellowship
http://www.surgicaleducation.com/mc/page.do?sitePageId=28279<br>
slide28. Professional Development Association of Program Directors in Surgery
http://www.apds.org/index.htm
Association of American Medical Colleges http://www.aamc.org/
Regional AAMC Groups e.g. CGEA
http://cgea.net/
ACGME http://www.acgme.org/acWebsite/home/home.asp<br>
slide29. Masters Degree Programs University of Illinois Chicago:
http://www.uic.edu/com/mcme/mhpeweb/MHPE_ProgramOverview.htm
University of Illinois Urbana Champaign & Southern Illinois University
http://www.siumed.edu/dme/online_masters1.html
University of Southern California, Rossier School of Education
http://rsoeweb2.usc.edu/helpdesk/master/edd.html<br>
slide30. Masters Degree Programs Master of Medical Education, University of Toronto http://www.oise.utoronto.ca/oise/ Prospective_Students/Graduate_Studies.html
Masters of Education in Teaching with an Emphasis in Health Sciences. University of Houston http://www.coe.uh.edu/academic-programs/health-education/index.php<br>
slide31. Funding Sources CESERT http://www.surgicaleducation.com/mc/page.do?sitePageId=28551
SAGES
http://www.sages.org/leadership/committees/research/grants.php
AHRQ
http://www.ahrq.gov/fund/<br>
slide32. Funding Sources AERA
http://www.aera.net/grantsprogram/
Macy Foundation
http://www.josiahmacyfoundation.org/apply/grant-programs
Department of Defense
http://www.federalgrantswire.com/department-of-defense-federal-grants.html<br>
slide33. Journals Academic Medicine.
Advances in Health Science Education Theory & Practice
Joint Commission Journal on Quality & Patient Safety.
Journal of Surgical Education.
Medical Education.
Medical Teacher.
Teaching, & Learning in Medicine.<br>
slide34. Journals - Clinical Archives of Surgery
American Journal of Surgery
Journal of the American College of Surgeons
Annals of Surgery
Surgery
Surgical Endoscopy
Best Evidence in Medical Education
http://www2.warwick.ac.uk/fac/med/beme/<br>
slide35. Jeannette Capella, MD, MEd, Steven J. Kasten, MD, Susan Steinemann, MD, Laura Torbeck, PhD, 2010. $35.00
Cine-Med at www.cine-med.com Guide for Researchers in Surgical Education<br>
Professor of Surgery & Vice Chair for Surgical Education
Southern Illinois University, Springfield<br>
slide2. Disclosures I have no relevant financial relationships to disclose pertinent to the content of my presentation.<br>
slide3. Overview What is surgical education research?
Identifying & avoiding the pitfalls
Academic success as surgeons performing research in surgical education<br>
slide4. Research in Education - Criticisms Opportunistic
Reproduce what has already been done
Do not add new knowledge.
Lack a literature review
Inappropriate methodologies
Albert M et al. Adv in Health Sciences Educ. 2007
40% published abstracts lack essential elements
Cook D et al. Med Educ 2007<br>
slide5. Research in Surgical Education A scholarly approach is “one that demonstrates evidence of drawing from and building on the work of others, and contributing work through public display, peer review and dissemination; both involve engagement with the community of educators”.
Consensus Conference on Educational Scholarship convened by the Association of American Medical Colleges (AAMC) Group on Education Affairs (GEA). Simpson D et al. Med Educ 2007<br>
slide6. Research in Surgical Education “Systematic process of defining, testing & analyzing a problem”
Identify problem & research question
Define independent & dependent variables
Review literature
Research design (methods & collaborators)
IRB
Funding
Dissemination<br>
slide7. Identify a Problem - FINER Significance, rationale
Feasible, Interesting, Novel, Ethical, Relevant
Breadth & complexity,
Resources, time, measurement
Narrow the focus<br>
slide8. Example<br>
slide9. Literature Review Is this an original question?
Has it been answered in other disciplines?
Has it been answered in other learners?
Harvard Business Review
New York Times
Pubmed /Ovid / Medline
ERIC/ EBSCO<br>
slide10. Evaluating the Education Literature What groups are being compared?
Confounding factors?
Is comparison based on risk factor or outcome?
Is there any bias?
Sample size / p value?
Literature inclusion / exclusion criteria ?
Generalizable?
Conclusions justified?<br>
slide11. Research Design Involve a statistician early in the process to ensure appropriate sample size, effect size, reliability , validity<br>
slide12. Research Design : Purpose Exploratory
Seek to uncover important questions & variables prior to establishing formal hypotheses.
Descriptive
Seek to characterize & measure sets of variables about people or phenomena.
Analytic
Seek to determine if there are relationships between specific groups.
Bordage G et al Med Educ 2003<br>
slide13. Research Design : Time Frame Retrospective
Begins with outcome & searches for prior existing variables to explain the outcome
Cross – sectional
Measures two or more variables to establish a relationship, therefore one must precede the other
Prospective
Follows subjects forwards in time to measure an outcome
Bordage G et al Med Educ 2003<br>
slide14. Research Design: Role of Observer Henry R in Guide for Researchers in Surgical Education 2010<br>
slide15. Research Design: Collaborators Establish authorship & responsibilities early!
Challenges to research “ownership”
Role confusion & power differential
Long term projects
“Abstract first” approach
Research hand off
Culture of “gifting” authorship
Hauge L in Guide for Researchers in Surgical Education 2010<br>
slide16. Research Design: Collaborators For authorship, most journals require contribution in each of the following:
Conception & design, or acquisition of data, or analysis and interpretation of the data
Drafting the manuscript or revising it critically for important intellectual content
Approval of the version to be published
International Committee of Medical Journal Editors http://www.icmje.org/urm_main.html<br>
slide17. IRB Do I need IRB approval?
Research means “a systematic investigation including research development, testing & evaluation designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for another purpose”
45CFR46 http://ohsr.od.nih.gov/guidelines/index.html<br>
slide18. IRB The IRB can be your friend
IRBs differ
Even if you know you are exempt you must submit an application for an exemption
Being exempt does not mean that you don’t need to worry about consent, or provide access to study results
Gaining consent from subjects is worth the time investment<br>
slide19. Funding & Grant Writing 30% of studies that received funding, received about half actual cost
Reed D JAMA 2005
Identify projects previously funded by agency
Read requirements & provide exactly what is requested
Abstract / summary
Background/ significance / preliminary work/ hypothesis / aims/ design/ methods /
Budget
Resources / CV<br>
slide20. Dissemination - Guide<br>
slide21. Tips - Survey Design Need to know versus nice to know!
Engage the respondent
Short and focused
Format
Introductory statement + consent
Basic demographic questions
Essential questions
Comment box
Concluding statement<br>
slide22. Tips - Survey Design Language
Keep language consistent
Instructions
Brief and clear for each question
Sequencing
Cluster by theme and format
Pilot Test
Need to know denominator to calculate a response rate!<br>
slide23. Tips: Kirkpatrick Evaluation Framework Kirkpatrick, D. L. In Training and Development Handbook New York: McGraw Hill. 1967<br>
slide24. Academic Success - Resources Statistician with expertise in educational research
PhD in education
Qualitative research methods
Research assistants
Graduate students
Nurse educators/coordinators
Residents<br>
slide25. Academic Success Job description should reflect educational effor
Double dip
Each educational innovation has potential for research & publication
Find a mentor
Improve your skills
Masters degree/ Workshops/Literature in education
Be visible
Offer to write critiques / speak / moderate
Collaborate outside surgery<br>
slide26. Additional Resources The following slides contain lists of resources specific to health professions educators<br>
slide27. Professional Development ACS Surgeons as Educators Course
http://www.facs.org/education/sre/surgedbrochure.pdf
Association for Surgical Education Workshops
http://www.surgicaleducation.com/mc/page.do?sitePageId=31641&orgId=ase
ASE Surgical Education Research Fellowship
http://www.surgicaleducation.com/mc/page.do?sitePageId=28279<br>
slide28. Professional Development Association of Program Directors in Surgery
http://www.apds.org/index.htm
Association of American Medical Colleges http://www.aamc.org/
Regional AAMC Groups e.g. CGEA
http://cgea.net/
ACGME http://www.acgme.org/acWebsite/home/home.asp<br>
slide29. Masters Degree Programs University of Illinois Chicago:
http://www.uic.edu/com/mcme/mhpeweb/MHPE_ProgramOverview.htm
University of Illinois Urbana Champaign & Southern Illinois University
http://www.siumed.edu/dme/online_masters1.html
University of Southern California, Rossier School of Education
http://rsoeweb2.usc.edu/helpdesk/master/edd.html<br>
slide30. Masters Degree Programs Master of Medical Education, University of Toronto http://www.oise.utoronto.ca/oise/ Prospective_Students/Graduate_Studies.html
Masters of Education in Teaching with an Emphasis in Health Sciences. University of Houston http://www.coe.uh.edu/academic-programs/health-education/index.php<br>
slide31. Funding Sources CESERT http://www.surgicaleducation.com/mc/page.do?sitePageId=28551
SAGES
http://www.sages.org/leadership/committees/research/grants.php
AHRQ
http://www.ahrq.gov/fund/<br>
slide32. Funding Sources AERA
http://www.aera.net/grantsprogram/
Macy Foundation
http://www.josiahmacyfoundation.org/apply/grant-programs
Department of Defense
http://www.federalgrantswire.com/department-of-defense-federal-grants.html<br>
slide33. Journals Academic Medicine.
Advances in Health Science Education Theory & Practice
Joint Commission Journal on Quality & Patient Safety.
Journal of Surgical Education.
Medical Education.
Medical Teacher.
Teaching, & Learning in Medicine.<br>
slide34. Journals - Clinical Archives of Surgery
American Journal of Surgery
Journal of the American College of Surgeons
Annals of Surgery
Surgery
Surgical Endoscopy
Best Evidence in Medical Education
http://www2.warwick.ac.uk/fac/med/beme/<br>
slide35. Jeannette Capella, MD, MEd, Steven J. Kasten, MD, Susan Steinemann, MD, Laura Torbeck, PhD, 2010. $35.00
Cine-Med at www.cine-med.com Guide for Researchers in Surgical Education<br>