MethodS matter: on the importance of relevant

Transcript:MethodS matter: on the importance of relevant:
MethodS matter: on the importance of relevant evidence for policy solutions Prof. Jarosław Górniak Centre for Evaluation and Analysis of Public Policies Jagiellonian University in Krakow “Evidence-based policy in Erasmus+. Seminar on research and methodology” November 28-30, Warsaw Credit for the title: The importance of theory for Policy making “There is Nothing More Practical Than A Good Theory” - Kurt Lewin (Ludwig Boltzmann, James Maxwell?) "Theory is when you know everything but nothing works. Practice is everything works but no one knows why. In our laboratory, theory and practice are combined: nothing works and no one knows why!” – Albert Einstein “(…) practitioners and policymakers—at all levels—wanted to know the answers to questions about cause and effect. They wanted to know if A caused B, and wanted IES to commission research that would provide them with answers.” – Murnane & Willett A theory which matters for policy has to make evidence-based causal claims Evidence-based policy – policy analysis – evaluation Evidence-based policy – public policy based on scientifically sound evidence Policy analysis as the analysis for policymaking – providing policy makers with (evidence-based) advice on problems, causal mechanisms, instruments and potential consequences of the available options Evaluation as a source of knowledge about what works, for whom and in what circumstances Policy analysis Ex ante evaluation Ongoing evaluation Policy analysis without evaluation is blind evaluation without policy analysis is powerless A Nobel Prize winner, James Heckman ON evaluation of public policies facing three main problems: “Evaluating the impact of historical interventions on outcomes including their impact in terms of the well-being of the treated and society at large.” “Forecasting the impacts (constructing counterfactual states) of interventions implemented in one environment in other environments, including their impacts in terms of well-being.” “Forecasting the impacts of interventions (constructing counterfactual states associated with interventions) never historically experienced to various environments, including their impacts in terms of well-being.” (Heckman, 2008, p. 8; see also: Heckman, 2005). Good Evidence Policy-relevant – justifying the choice of policy conduct and instruments Trustworthy Sound theory – causal claims Proper scientific methodology – research design and measurement Dependable and up-to-date data Conclusive – “clearly speaks for or against the policy” “Will it work here? That is, will the policy that you are considering make a positive difference in the desired outcome if you implement it, bearing in mind how, where, and when you would do so? In the language