Understanding the peer review process in academic
Description: Understanding the peer review process in academic publishing Sarah K. Bruch Agenda peer review: the basics What is Peer Review? Review process for assessing the validity, significance and originality of research Peers are experts in a given
Related Topics
Download Presentation
"Understanding the peer review process in academic" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Understanding the peer review process in academic publishing Sarah K. Bruch<br>
slide2. Agenda<br>
slide3. peer review: the basics What is Peer Review?
Review process for assessing the validity, significance and originality of research
Peers are experts in a given field of study
When is Peer Review Used?
publications (articles and books)
funding (fellowships and grants)
conferences (panel presentations and posters)
promotion and tenure<br>
slide4. peer review: the basics Why is Peer Review Important?
facilitates academic knowledge generation
serves to legitimate knowledge (epistemic and social legitimacy)
signals quality (self-regulation of the knowledge community)
Provides perspective on the work
Debates/Controversies
bias and subjectivity in editorial and reviewer assessments
limited accountability and justification for decision-making
replicability and transparency
gatekeeping “mistakes”<br>
slide5. peer review: the basics Source: Standing up for Science: Peer review, the nuts and bolts: A guide for early career researchers.<br>
slide6. Step 1: which journal? Audiences and Purposes
generalist vs specialist
discipline-specific vs interdisciplinary field
sub-field/sub-disciplinary specialty journals that have specific topic/area focus
methodological approach
format or type of research (ex. review, original research, theoretical ideas)
access How to:
look at your own reference list
look at journal homepage to identify aim and scope
look at TOC to identify the kinds of things they publish
look at introductions of articles to identify how research is framed (i.e. sociologists have long studied… a key question animating sociology of education is…)<br>
slide7. Step 1: which journal? Rankings, Prestige, and Quality
journals vary in quality and prestige
consider the quality of the contribution you think the piece is making How to:
ask scholars in the field
look at impact factors, citation reports, altmetrics, etc.<br>
slide8. Step 1: which journal? Resources:
ASPA Listing of Journals (link)
APSA Listing of Journals (link)
ASA Publishing Options Searchable Database (ASA member log in required)
U of Michigan Libraries. Guide on judging the reputation of a journal (link)
Suiter, Amy M. and Cathy C. Sarli 2019 “Selecting a Journal for Publication: Criteria to Consider”<br>
slide9. Step 2: prepare submission Submission Guides or Instructions for Authors
All journals have specific guidelines for every aspect of the paper submission
Word count, page count, table/figure limits
Page and reference formatting
Sometimes these can also be a partial selection criteria for your work Resources: Tips on Submitting Papers for Review by outgoing editors and professional associations: MacDonald et al. 2006 (link); AJE (link) EEPA (link); ASLO (link)<br>
slide10. What to expect - Peer review Typical format
double-blind or single-blind (impartiality)
editorial board with 1-2 editors and set of secondary editors
send to 2-4 reviewers (looking for level of agreement among reviewers)
editors and associate editors make final decision and send decision letter (sometimes with substantive assessment of what the editor thinks are the main issues that need to be addressed but sometimes not just the reviewer letters)<br>
slide11. What to expect - Peer review Rejection…
Most top journals in fields have acceptance rates that are less than 10%!
A lot of submissions are desk rejected by the editor (editor decision based largely on fit with journal aim or scope)
Majority of articles sent out for review are rejected Sources: Sanscartier, Matthew and Matthew Johnston. 2018 “How to thrive on academic criticism” (link)
Garand and Harman 2021. “Journal Desk-Rejection Practices in Political Science: Bringing Data to Bear on What Journals Do” PS: Political Science & Politics 54(4): 676-681.<br>
slide12. What to expect - Peer review Journal Responses:
Rejection
Invitation to Revise & Resubmit (R&R) – different types
Conditional acceptance (or Accept with minor revisions)
Acceptance
Time to Initial Decision & Entire Review Process
Initial decision typically 2-4 months or more (varies by field and journal)
Time from initial submission to publication takes several months or more Resources: American Sociological Association Journal Annual Reports (which includes acceptance rates) (link); American Psychological Association Journal Summary Statistics (link)<br>
slide13. What to expect - Peer review Invitations to Revise and Resubmit (R&R)
Read over the entire letter from the Editor and all the reviews
Use reviewers’ comments to help you identify and solve problems with your research and writing
Try your best to separate substantive content from tone
Always take the Editor’s guidance most seriously
Also be sure to notice any positive comments from either the Editor or the reviewers Resources: Sovacool et al. 2022 “Towards codes of practice for navigating the academic peer review process”<br>
slide14. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide15. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide16. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide17. Writing a peer review When accepting the invitation to review you are:
agreeing to provide a fair, robust and timely assessment that is useful for the authors in improving their manuscript
asserting that you do not have a conflict of interest (i.e. you would be advantaged in some way; work at same institution; are a close colleague; been a mentor or advisor in the past several years; etc.)<br>
slide18. Writing a peer review Source: Standing up for Science: Peer review, the nuts and bolts: A guide for early career researchers.<br>
slide19. Criteria of useful peer reviews The most useful reviews are:
Critical: assessment of both the strengths and weaknesses of the research
Constructive: describe particular concerns that you identify, explain why they matter, and offer constructive suggestions for strengthening the research Resources: JPAM Guide for Reviewers (link); PLOS How to Write a Review (link); Cambridge Guide to Peer Reviewing Journal Articles (link); Wiley Peer Review Guidelines (link); Sovacool et al. 2022 “Towards codes of practice for navigating the academic peer review process”<br>
slide20. Questions?<br>
slide21. resources Slide decks for previous sessions
CV and Academic Bio Workshop
Presenting and Talking about Your Work
Preparing for Research Careers
Funding, Research, & Training Opportunities
Preparing for the Academic Job Market
Preparing for Academic Presentations
Available on Biden School website at: https://www.bidenschool.udel.edu/student-resources Related Readings
Van Cott, D. L. 2005. “A Graduate Student’s Guide to Publishing Scholarly Journal Articles.” PS: Political Science and Politics 38 (4): 741–743.
Pirog, Maureen. 2014. “The Art and Science of Scholarly Publishing.” JPAM 33(3): 843-853.
Blair et al. 2020. “Publishing in political science journals” European Political Science 19:641–652.<br>
slide2. Agenda<br>
slide3. peer review: the basics What is Peer Review?
Review process for assessing the validity, significance and originality of research
Peers are experts in a given field of study
When is Peer Review Used?
publications (articles and books)
funding (fellowships and grants)
conferences (panel presentations and posters)
promotion and tenure<br>
slide4. peer review: the basics Why is Peer Review Important?
facilitates academic knowledge generation
serves to legitimate knowledge (epistemic and social legitimacy)
signals quality (self-regulation of the knowledge community)
Provides perspective on the work
Debates/Controversies
bias and subjectivity in editorial and reviewer assessments
limited accountability and justification for decision-making
replicability and transparency
gatekeeping “mistakes”<br>
slide5. peer review: the basics Source: Standing up for Science: Peer review, the nuts and bolts: A guide for early career researchers.<br>
slide6. Step 1: which journal? Audiences and Purposes
generalist vs specialist
discipline-specific vs interdisciplinary field
sub-field/sub-disciplinary specialty journals that have specific topic/area focus
methodological approach
format or type of research (ex. review, original research, theoretical ideas)
access How to:
look at your own reference list
look at journal homepage to identify aim and scope
look at TOC to identify the kinds of things they publish
look at introductions of articles to identify how research is framed (i.e. sociologists have long studied… a key question animating sociology of education is…)<br>
slide7. Step 1: which journal? Rankings, Prestige, and Quality
journals vary in quality and prestige
consider the quality of the contribution you think the piece is making How to:
ask scholars in the field
look at impact factors, citation reports, altmetrics, etc.<br>
slide8. Step 1: which journal? Resources:
ASPA Listing of Journals (link)
APSA Listing of Journals (link)
ASA Publishing Options Searchable Database (ASA member log in required)
U of Michigan Libraries. Guide on judging the reputation of a journal (link)
Suiter, Amy M. and Cathy C. Sarli 2019 “Selecting a Journal for Publication: Criteria to Consider”<br>
slide9. Step 2: prepare submission Submission Guides or Instructions for Authors
All journals have specific guidelines for every aspect of the paper submission
Word count, page count, table/figure limits
Page and reference formatting
Sometimes these can also be a partial selection criteria for your work Resources: Tips on Submitting Papers for Review by outgoing editors and professional associations: MacDonald et al. 2006 (link); AJE (link) EEPA (link); ASLO (link)<br>
slide10. What to expect - Peer review Typical format
double-blind or single-blind (impartiality)
editorial board with 1-2 editors and set of secondary editors
send to 2-4 reviewers (looking for level of agreement among reviewers)
editors and associate editors make final decision and send decision letter (sometimes with substantive assessment of what the editor thinks are the main issues that need to be addressed but sometimes not just the reviewer letters)<br>
slide11. What to expect - Peer review Rejection…
Most top journals in fields have acceptance rates that are less than 10%!
A lot of submissions are desk rejected by the editor (editor decision based largely on fit with journal aim or scope)
Majority of articles sent out for review are rejected Sources: Sanscartier, Matthew and Matthew Johnston. 2018 “How to thrive on academic criticism” (link)
Garand and Harman 2021. “Journal Desk-Rejection Practices in Political Science: Bringing Data to Bear on What Journals Do” PS: Political Science & Politics 54(4): 676-681.<br>
slide12. What to expect - Peer review Journal Responses:
Rejection
Invitation to Revise & Resubmit (R&R) – different types
Conditional acceptance (or Accept with minor revisions)
Acceptance
Time to Initial Decision & Entire Review Process
Initial decision typically 2-4 months or more (varies by field and journal)
Time from initial submission to publication takes several months or more Resources: American Sociological Association Journal Annual Reports (which includes acceptance rates) (link); American Psychological Association Journal Summary Statistics (link)<br>
slide13. What to expect - Peer review Invitations to Revise and Resubmit (R&R)
Read over the entire letter from the Editor and all the reviews
Use reviewers’ comments to help you identify and solve problems with your research and writing
Try your best to separate substantive content from tone
Always take the Editor’s guidance most seriously
Also be sure to notice any positive comments from either the Editor or the reviewers Resources: Sovacool et al. 2022 “Towards codes of practice for navigating the academic peer review process”<br>
slide14. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide15. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide16. What to expect - Peer review Revision Memo
Address all reviewer comments and concerns.
Provide rationale for why you made each particular research choice relative to the alternative(s)<br>
slide17. Writing a peer review When accepting the invitation to review you are:
agreeing to provide a fair, robust and timely assessment that is useful for the authors in improving their manuscript
asserting that you do not have a conflict of interest (i.e. you would be advantaged in some way; work at same institution; are a close colleague; been a mentor or advisor in the past several years; etc.)<br>
slide18. Writing a peer review Source: Standing up for Science: Peer review, the nuts and bolts: A guide for early career researchers.<br>
slide19. Criteria of useful peer reviews The most useful reviews are:
Critical: assessment of both the strengths and weaknesses of the research
Constructive: describe particular concerns that you identify, explain why they matter, and offer constructive suggestions for strengthening the research Resources: JPAM Guide for Reviewers (link); PLOS How to Write a Review (link); Cambridge Guide to Peer Reviewing Journal Articles (link); Wiley Peer Review Guidelines (link); Sovacool et al. 2022 “Towards codes of practice for navigating the academic peer review process”<br>
slide20. Questions?<br>
slide21. resources Slide decks for previous sessions
CV and Academic Bio Workshop
Presenting and Talking about Your Work
Preparing for Research Careers
Funding, Research, & Training Opportunities
Preparing for the Academic Job Market
Preparing for Academic Presentations
Available on Biden School website at: https://www.bidenschool.udel.edu/student-resources Related Readings
Van Cott, D. L. 2005. “A Graduate Student’s Guide to Publishing Scholarly Journal Articles.” PS: Political Science and Politics 38 (4): 741–743.
Pirog, Maureen. 2014. “The Art and Science of Scholarly Publishing.” JPAM 33(3): 843-853.
Blair et al. 2020. “Publishing in political science journals” European Political Science 19:641–652.<br>