Peer review – a view from the social sciences - PowerPoint Presentation

Slide1

Peer review – a view from the social sciences

Dr Karim MurjiSlide2

Aims of this session

To provide an overview of reviewing process for academic journals in the social sciences

To look at editorial role and processes

To identify good peer reviewing practice

To highlight benefits of peer reviewing [double blinded]Slide3

A gold standard in publishing… or a ‘sacred cow’?

Different kinds of ‘peer review’

-pre-publication [improves quality of papers; up to 80% rejected]

post-publication

Have we always had peer review?Slide4
Slide5
Slide6

Why is it important?

Peer-review in academia as a mechanism for assessment – for more than journals only

- Journal articles – and metrics

- Grants (awards and outcomes)

- Books (proposals and full manuscripts)

- REF/impactSlide7

How peer review is organised

The example of Sociology

The Editors - Are the ‘ultimate’ reviewers/decision makers

Editorial boards

- Ensure editorial accountability

- Provide continuity across editorships

- Maintain quality control

Adjudicate in differences of opinion

Individual [non board] reviewers

- Spread the workload

- Provide specialist expertise

- Have awareness of other national contextsSlide8

Becoming a reviewer

Submit a paper or book review

Contact the editors direct

Get yourself onto an editorial or advisory board

Keep a visible and searchable web presence

Add yourself to the database

Conferences and eventsSlide9

Why do it?

Professional responsibility/part of scientific community

You

become part of the process by which the academy self regulates (and by which academic schools and universities are measured and assessed)

Increases

recognition that you have expertise

Personal

career development

But also

Can

be satisfying and thought-provoking

It

improves level of intellectual engagement with all academic writingSlide10

Seeing an article through the review process

All papers are processed electronically. On submission

1) Administrator checks for length and anonymity

2) An editor then reviews the paper to

ascertain whether it is worth sending out for review

If no, immediate ‘desk reject’

If yes, at least two – sometimes three – referees assigned.

Reviewers allocated 33 days to review; average turnaround to decision is 60 daysSlide11

Getting reviewers

Board members can not decline, but non–board people can – and some/a few do not respond

Reviewer guidelines

Delays in responding to authors almost always due to late reviews; or, rarely, need for adjudication

‘League table’ of workload/average turnaround for board membersSlide12

The requirements of peer review

In Sociology, 4 components:

-A grid for ranking the paper along a number of criteria

- Confidential comments to the editors

- Comments to the author(s)

- Recommendation/decisionSlide13
Slide14
Slide15
Slide16

Reviewer questions

Read the manuscript thoroughly and ask yourself:

- Is it right for the journal

? Does it fit better somewhere else?

Does

it make clear what its main contribution is to

sociology? Is

it justified in claiming this contribution?

Is this contribution of significance and/or interest?

Is any empirical component sufficiently robust and explicit in method (including ethical considerations)?

Does it support the conclusions?

Is it well-written

? Slide17

Decision categories

[Immediate] Accept. This is exceptional/rare

Minor

revisions – are they really minor?

Major

revisions – the most common response

Are

they

really

major

? Or is

it really a reject? (almost any paper can be made better)

Are

you

criticising

it for not being the paper

you

would have written?

RejectSlide18

Areas to comment on

Overall – contribution,

argument, persuasiveness

Structure

Context/literature review

Methodology and methods

Data and evidence

Discussion and conclusion

Keep things in proportion

Are the

issues/problems fixable? Slide19

A good review…

Length – not too long or too short

Constructive comments

Appreciates the author’s

perspective

Being

critical – in a helpful way

Rejecting – and not using major revisionSlide20

Poor/bad reviews

Too

accepting/uncritical

Too

short

Focus on

reviewer’s interests [‘hobby horse’] rather

than the paper’s intentions

Dismissive and

destructive comments

Give a decision

in the review Slide21

Writing a review

Comments to the editors

-

In confidence so can be candid

here

- Examples of comments received: notification that this has just been reviewed elsewhere; very strong criticisms that might be bruising; warnings of reputational risk to the journal; personal knowledge of author; limits of your expertise; declaration of conflict of position.

Comments to the author(s)

-

Must

support the

recommendation but not

declare

it

-

As much detail as needed, but

not obliged

to write very

long, detailed

and justificatory comments (quality of judgement is more important than quantity)

- Do not feel you need to point out typos etc (although do comment on quality of presentation)Slide22

Editor decision making

Comments and

recommendations can

be very different in content and outcome

The editors decide

on the outcome

Where reviews differ, editors decide how to reconcile

- Make their own judgment, informed by divergent reviews or based on quality of reviewer comments

- Send it out to a third reviewer for extra review or adjudication

Usually the default outcome is to take the more

expert/informed/critical

review.Slide23

Re-reviewing a paper

Read the author’s accompanying note, but don’t leave it at that….

Check to see if they have addressed your concerns adequately

Do not feel obliged to accept the paper simply because they have revised it or because the other reviewer likes it.

Do not raise NEW issues that were already present in the original version.

It is quite alright to ask for further major revisionsSlide24
Slide25

Impact Factor:

1.617 | 

Ranking:

Sociology 21 out of 142Slide26

Is the future open review?

The

single-blind model (reviewers unknown to authors) is adopted by most biomedical journals.

Blinding

also reviewers (double-blind model) is tricky and does not improve reports' quality.

Open

peer review ensures transparency about study reports that may influence clinical practice.

Also

the editorial process should be made

public,

avoiding an excessive focus on reviewers' role

.

Vercellini et al, European Jnl of Internal Medicine, April 2016