current methodology and expected development Prof Zdenek Nemecek Charles University Prague Brief history and legal frame The distribution of the government research support is governed by the law 1302002 and its consecutive amendments ID: 336660
Download Presentation The PPT/PDF document "Research performance evaluation in the C..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Research performance evaluation in the Czech Republic:current methodology and expected development
Prof.
Zdenek
Nemecek
Charles University, PragueSlide2
Brief history and legal frame
The distribution of the government research support is governed by the law 130/2002 and its consecutive amendments
Evaluation of the research entities started in 2004 without any connection with financing
The amendment from 2005 states that the distribution of the support for the long-term conceptual development of research entities (SCD) among the support providers (ministries, Academy) should follow the evaluation of the results
Methodics
2008, 2010, 2013
The providers are obliged to follow this distribution for their entities unless they have their own evaluation systemSlide3
R&D budget
Government sources only, EU funds not included
Total
SCD
Grant agencies, international cooperation, etc.Slide4
Evaluation Methodics
Each research entity provides the list of their results to RIV
The results are divided into categories – journal paper, proceedings, patent, etc.
The results are further divided according topics – physics, social sciences, mathematics, chemistry, etc.
Methodics
assigns a predefined number of points to each result
The money that a particular entity receives would be directly proportional to the sum of the points gained by its results
Keep in mind that:
1 -
Methodics
is dedicated for division of money among the research entities
2 – it is not supposed to be used on the level of research teams or individuals
3 - it is not intended for identification of excellence Slide5
Topical groups
The number of points gained by a particular topical group is normalized to represent a given part of the points available (3 000 000) Slide6
Categories of results
8
2.
5
%
17.5%
The numbers of points assigned to publication and application results are normalized to represent a given part of the points availableSlide7
Evolution of the Methodics
The first
Methodics
2008 provided a reasonable evaluation of research entities
However, the production of “results for points” started immediately
The only category of results not affected by this trend were papers in good journals (
WoS
)
The consecutive
Methodis
(2010) introduced several limitations:
- percentage of applied results
- division of results into topics (mathematics, social sciences, etc.) with predefined number of points
These limitations brought a partial improvement but the main problem – equal number of points for all results in a given category - remainsSlide8
Evaluation Methodics
2013
Three parts -
Mainstays
Each mainstay has its own allocation of points
Mainstay I
– old system for papers in journals indexed in
WoS
or
Scopus, books and papers in the Czech journals come through peer review
Mainstay II -
peer review evaluation of a limited number of resultsMainstay III
– new system of evaluation of applications, the results are not weighted and the points are assigned to research entities.Slide9
Mainstay weight
Mainstay (
Pil
íř
) I
–
publication results
Mainstay (
Pil
íř
) II
–
NEW
– excellent results
Mainstay (
Pil
íř) III
– applicationsSlide10
Mainstay I
WoS
SCOPUS ERIH
Domestic
Books
Proceedings
SHVa,b
Jimp J
sc Jneimp J
rec BC DSHVc Jimp
Jsc
Jneimp
BC DTechnika a informatika Jimp
Jsc BC DZemědělství
Jimp Jsc
BC DVědy o zemi Jimp
Jsc BC Matematika J
imp Jsc BC D
Fyzika JimpChemie Jimp
Biologie
JimpLékařství
Jimp
Points
(10 – 305) (10 –
30) 4 (4
–
120) (8 – 60)
This mainstay is adopted and adapted from the previous
Methodics
2008 and 2010
82.5% of points is distributed according to this mainstay
Social
Sci
and Humanities
Social
Sci
Technical
Sci
and IT
Agriculture
Sci
Earth
Sci
Mathematics
Physics
Chemistry
Biology
MedicineSlide11
Mainstay II.
Each research unit submits
one excellent result per CZK 10 mil.
for evaluation
The submission is accompanied with reasoning why the result is excellent
The result is evaluated by the
Expert panel
of a given topical group and 20% of the best is chosen (category A)
10%
(60 000)points is allocated
to this mainstay
These points will be distributed to the research entities in accord with the number of results in category A
Additional
2000
points will be allocated to the entity that received an ERC grant in course of past 5 years
Note that this mainstay will be applied from 2015Slide12
Estimated number of results submitted to Mainstay II
Total: 999 resultsSlide13
Mainstay III.
105 000 points will be allocated according to this Mainstay
In the first step, a given number of points will be assigned to patents, novel races or cultivars
In the next step, the rest of points will be distributed in accord with the volume of money gained by a particular entity either as grants of applied research or by research contracts with private enterprises
∑
Total
= α∑Grants
+ β∑Contracted research (
α = 0.9;
β = 0.1) Slide14
Present status of M 2013Data for Mainstays I and III are collected
The software tools for the data processing are almost ready
The members of Expert panels are nominated
Peer review of books and Czech journals will start in course of June
The excellent results will be collected in near future
Since the process according to M 2013 will be applied only on the data from the last year, it will influence only about 20% of the budget
Our modeling has shown that no dramatic changes in the money distribution can be expected