Public Expenditure Review Science, Technology, and

Public expenditure reviews for STI Motivation Increasing evidence that National Systems of Innovation are fragmented, uncoordinated and with little evidence of impact
Existing analysis and evaluations tend to be either too aggregated and institutional or too narrow focusing one program or instrument
Can we respond to basic questions with greater certainty?
How much is spent?
For what?
What is the quality and coherence of the policy mix?
How efficient is this investment?
With what results? Objectives To assess whether government’s allocation of resources has an impact on STI outcomes – through the quality and coherence of the policy mix; the quality of design, implementation and governance; its efficiency and its impact.

To help countries improve their capacity and quality of their STI policies. This includes building governments’ capacity to design, implement and monitor STI policies and to rationalize their policy mix – measure, learn and adapt. Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Methodology Logical framework of an STI intervention (Correa, 2014) Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Amount transferred to beneficiaries for every US$ 1 spent on the administration of the program. Impact of STI policies on Innovation and technology adoption Four stages of the PER analysis Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Differences with traditional PER Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

How does a PER add value?
Other reports do not provide a systematic framework to track public expenditures in R&I and measure its impact.
The most comprehensive review of innovation policies are conducted by OECD but they do not provide detailed disaggregated data on public spending and its effect on the innovation spending
The PER offers a systematic framework to organize public spending and track its impact on different aspects of the innovation system Public Expenditure Reviews (PER) would provide an analytical framework that opens the black box and analyzes public expenditure on R&I in detail to establish whether the results of these investments are effective (reached desirable outcomes) and efficient (at minimum costs). And in doing so creates information that addresses coordination issues inherent to the policymaking process. What is the value added of this expenditure review?<br>

Applications and Examples Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Public Expenditure Review of Science, Technology, and Innovation (STI) policies Unintended Consequences CASE OF TURKEY An effort to foster collaboration between science and industry
Tax breaks policies established the location at science parks as a condition -- fostering the demand for space in ‘science parks’ (infrastructure space in universities).
Subsidies supported the expansion of ‘science parks’.
Result : by 2009 Turkey had about 12.3 parks/ 1 billion of R&D (vs 3.2 in US and 2.9 in Canada) - 7.3 parks/10.000 researchers (vs 1.2 in US and 1.9 in Canada) – virtually no collaboration (only 4 firms out of all firms located in the parks) An effort to increase public R&D expenditures to improve economic performance
Reaching the 3 percent target would increase exports by 13,5 percent and GDP by 11,7 percent by 2025 (results from World Scan Model).
Between 2004-08, Romania’s Government duplicated their R&D effort (0.5-1.1) as a share of GDP.
In the same period, share of basic research increased from 22 to 40% of total investments.
The US was investing 17% and Israel and Japan less than 15%. CASE OF ROMANIA<br>

Mapping policy goals to outcomes Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Governments often decide on inputs with the implicit idea that they will generate a certain output – without considering the necessary and sufficient conditions or the chain of events that will lead to the desired output
Process of backward induction
Define desired outcome
Identify intermediate objectives
Identify the chain of events from public investments to intermediate objectives (outputs) Chain of Events
Necessary and sufficient conditions Logical Framework: Breaking down The innovation policy black box<br>

Mapping policy goals to outcomes Mismatch of budget allocation with the national development objectives Budget by objective 2010-2014 Productive improvement of existing businesses Diversification through new business dev. Investment inputs for innovation Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

The STI Policy Mix High budget concentration in a few instruments High concentration of direct support using matching grants Public Expenditure Review of Science, Technology, and Innovation (STI) policies Concentration:
12 instruments – 75% budget<br>

Functional Analysis The objective of this part is to evaluate the quality of design, implementation and governance of STI programs across three dimensions - 31 categories Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

IV. Functional Analysis 15 The Good Practice Model<br>

Functional Analysis Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Functional Analysis Clustering of instruments and identification of weaknesses Weaknesses of the system Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Key questions of the efficiency phase Cost structure
What is the composition of administrative costs?
How much is spent in administrative costs per project? What is the ratio of the administrative costs to the value of benefits transferred?
How do the administrative costs compare across programs?
Is it possible to reduce administrative costs without affecting the quality of program management? Is it possible to reduce transaction (monetary and nonmonetary) costs for applicants?

Input-Output
What is the value of program benefits per project?
How are the benefits distributed across beneficiaries?
What is the “value” of the results generated by the program (per program, per beneficiary, per each dollar spent)?
Are the outputs consistent with resources allocated to the program? Are the program benefits commensurate with the application / administration costs?
Could funding amounts be reduced without comprising the current performance? Could higher output levels be achieved with the existing funding level? Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

methodology – Logical framework<br>

methodology – Production functions<br>

Inputs Composition: admin costs vs implementation costs R&D tax credits Technology extension Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Beneficiary transfers per project Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Distribution of project benefits R&D tax credits Technology extension All projects awards
Total 44 projects Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Beneficiary feedback Public Expenditure Review of Science, Technology, and Innovation (STI) policies Beneficiaries’ perception towards the program outcomes

(Percentage of affirmative responses) Beneficiaries’ satisfaction with the program’s performance on each stage
(Percentage of affirmative responses)<br>

Monitoring vs Evaluating Monitoring
Collecting data to:
Improve day-to-day management and decisions
Track program performance against expected results
Measure costs/efficiency (unitary cost per product) Does the intervention reach the objectives in an efficient manner? Does it so accordingly with the original plans? Evaluating
Ex post evaluation of the results to:
Confirm or not the hypothesis of the results chain
Establish causality
Database to cost effectiveness (costs/benefits balance for a given program) Is the intervention effective? What would have happened in the absence of the intervention? Public Expenditure Review of Science, Technology, and Innovation (STI) policies<br>

Thanks for your attention Denis MEDVEDEV
LEAD ECONOMIST
dmedvedev@worldbank.org Regional Innovation Forum

Bratislava, Slovakia Leonardo iacovone
LEAD ECONOMIST
liacovone@worldbank.org Firm capabilities and innovation global unit<br>