Refereeing and Discussant Guidelines - PowerPoint Presentation

Slide1

Refereeing and Discussant Guidelines

Susan GodlontonAGRODEP AIEN III WorkshopDakar, Senegal4th June, 2014Slide2

Components of a referee report:

Cover letterSummary Key concerns

Decision

ReportOverall viewMain concernsSmaller concernsNo “one-size-fits-all approach”

Referee ReportSlide3

What is a good paper?

Clear Coherent (understandable)Correct

Additional contribution

Value added of answer to questionCredibility of identification strategyValidity and robustness of approachReferee Report: Good paper?Slide4

Don’t repeat the abstract, the editor can read the abstract

How would you summarize the paperWhat do you think is the key contributionDo you think the authors address it correctly

Referee Report: Overall viewSlide5

Key issues that the author needs to address

Ensure that it is clear which of your concerns are fundamental Don’t ask the author to write a different paper

Be constructive! Try to provide solutions where possible accounting for data limitations

Referee Report: Main ConcernsSlide6

Related literature

Inappropriate to the actual material in the paperInaccurately describedIncomplete (very common)Logical argument Incorrect application of economic conceptsInaccurate mathematical derivations

Loose

ties between the economic model and the empirical analysisReferee Report: Main concernsSlide7

Econometric tools are being used inappropriatelyIncorrectly used

Assumptions do not holdAssumptions not defended with evidence Estimated key coefficients of interestNot properly identified Not robust

Additional

robustness checks neededInterpretation of empirical results is inappropriate Overstates the contributionOverstates the claim (e.g. causation vs. correlation)Conclusions are incorrectly made or expressedReferee Report: Main concernsSlide8

Econometric tools are being used inappropriately

Incorrectly usedAssumptions do not holdAssumptions not defended with evidence Estimated key coefficients of

interest

Not properly identified Not robustAdditional robustness checks neededInterpretation of empirical results is inappropriate Overstates the contributionOverstates the claim (e.g. causation vs. correlation)Conclusions are incorrectly made or expressed

Referee Report: Main concernsSlide9

Do not bring these up when discussing a paperSpecific places in text where discussion hard

to follow or confusingNot using standard notationSpelling and grammatical errorsMissing data sources and poorly constructed Tables or Figures

References

to the literature that are missing or incorrectReferee Report: Minor concernsSlide10

Refereeing

http://blogs.worldbank.org/impactevaluations/how-much-to-referee-and-how-to-do-ithttp://marcfbellemare.com/wordpress/2012/01/contributing-to-public-goods-my-20-rules-for-refereeing/

http://

marginalrevolution.com/marginalrevolution/2006/10/how_to_be_a_goo.htmlDiscussanthttp://chrisblattman.com/2010/02/22/the-discussants-art/Other resources will be uploaded to the AIEN workshop

Additional ResourcesSlide11

Each been assigned a paper/set of slides from one of your peersFocus

Clearly define the research question Define and discuss the identification strategy Propose alternative credible identification strategyFor today… Slide12

Focus on the identification strategy

What is the identifying variation?What assumptions are made by using the chosen identification strategy (implicitly/explicitly)? Do they make sense?To what extent are these true?How else can the author test the assumptions?What robustness checks or placebo tests could the author run?What would be a more preferable strategy?

For today…Slide13

Thinking about Identification

Homework AssignmentSlide14

Group into:Health

EducationAgricultureDiscuss proposed ideas, provide feedback to one another Thinking about Identification Exercise