Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Trai
9K - views

Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Trai

Similar presentations


Download Presentation

Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Trai




Download Presentation - The PPT/PDF document "Research Grade Evaluation (RGE) and Equi..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Trai"— Presentation transcript:

Slide1

Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Training

August

2014

Slide2

Resources

Office of Science Quality and Integrity intranet site,

http://internal.usgs.gov/quality_integrity/rge_edge/

RDSR portal (http://internal.usgs.gov/quality_integrity/rge_edge/RDSRportal.htm) - used to submit and access RDSR packages. Office of Personnel Management’s (OPM’s) Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) guidesUSGS Research and Development Evaluation Process Handbook - how USGS implements the OPM guidesRGE, EDGE, and ST ChecklistsTraining materialsRGE/EDGE coordinators:John Thompson and Judy Back, rge-edge@usgs.gov

2

Slide3

First-level RGE and EDGE panels

Rate

the

research/development contributions and impact of an employee using the RGEG/EDGEG and USGS checklists. Evaluate the R&D and related work in terms of scientific quality and impactAddress career progress, balance of scientific activities, relevance of work to USGS goals, & content and quality of RDSR. Identify significant work that is not R&D in nature. ID approximate % of time and impact on the position. Positions where R&D is less than 25% should not be rated by the panel.Make final recommendations for all retentions in grade, conversions to RGE/EDGE, and promotions through GS-13. Provide promotion recommendations and justifications to GS-14/15 and ST for the second-level panel

Review operational employees who request conversion to RGE/EDGE positionProvide constructive feedback to employeesAll evaluations, discussions, and documentationmust be kept confidential

3

Slide4

Research versus Development

Research (RGE)

Development (EDGE part 3)

Purpose

Extending knowledge and

understanding

New or improved

products, processes, and

techniques

Assignments

Problems to be solved:* entail relative freedom to explore promising areas in relation to organizational programs;* may stem from an intent to close gaps in knowledge in a given field, or develop new theories or explanations of phenomena; and* are difficult to define in terms of expected outcomes and measurable results. Problems to be solved:* are defined in advance or assigned;* may stem from an intent to exploit an understanding of phenomena and principles; or* have predictable outcomes or measurable results. ResultsProducts are:* papers describing new and modified theories and principles;* explanations of phenomena; and* information to improve the understanding of techniques and processes.Products are:* papers describing application of theories and principles;* design concepts, models, patents, and inventions; and* equipment, techniques, and processes.

4

Slide5

Four RGE/EDGE factors

RGE

EDGEFactor 1: Research Assignment Nature of the Assignment (scope, complexity, importance) (scope and complexity, objectives, means available, expected results)Factor 2: Supervisory Controls Supervision Received Factor 3: Guidelines and Originality Guidelines and Originality Factor 4: Contributions, Impact, and Qualifications and Contributions Stature (demonstrated ability, products,advisory activities)When scoring a package, each of the four factors is assigned a level, defined by OPM, with corresponding points. The sum of the scores you assign for the four factors determines the grade in which an employee is working.

5

Slide6

Research and Development activities to consider

Core R&D Work Peer Reviewed

Technical Assistance

Professional Society Services

Education and Outreach

Strategic Planning

Research Leadership and Direction

Societal Relevance

Science Impact

6

Slide7

1st

-level panel composition

Chair – Scientist who has served on previous panels.

Panel members – Panel members are research/development peers. The panel chair assigns members to be primary reviewers for specific scientists. RGE/EDGE coordinator – Guides panel process and takes notes during the deliberations.HR specialist – Attends panel as non-voting member to advise on procedural and interpretive issues.7

Slide8

Roles and responsibilities – panel members

Be prepared before the panel meets:

Thoroughly review the appropriate OPM RGE or EDGE (part 3) guide and the USGS RGE/EDGE/ST Factor Checklists.

They define the key elements in each factor and aid in evaluation of each factor.Thoroughly review the RDSR package for each employee.Review and consider career experience, significant contributions, and bibliography Read all publications listed under each scientist’s significant contributions (section 19)Carefully consider evidence of impact and stature of the scientistScore each package (for each of the four factors). Use the package and the appropriate guide and checklist (RGE or EDGE) to determine a degree level and corresponding score for each the four factors. If you think a GS-15 employee exceeds the maximum scores, consult the ST checklist. If an individual package does not fully meet a particular degree level, the next lower degree level must be assigned. Take notes that indicate the rationale for each factor score. Record initial scores, notes, and questions or discuss during the meeting on the Panel Member Evaluation Score Form (exhibit A-4 for RGE and B-4 for EDGE).

8

Slide9

Roles and responsibilities – panel members (cont’d.)

During the panel:

Be prepared, objective, and actively participate!

Confidentiality – nothing leaves the panel All documentation, files, and notes are returned/destroyed after panel deliberations Delete all digital files from your hard drives; destroy CDs/DVDs that contain panel informationAfter the panel:No outside discussions about panel deliberations or findingsDo not disclose identity of panel members

9

Slide10

Roles and responsibilities – primary reviewer

Prior to the meeting:

As with all other employees being reviewed, thoroughly review the RDSR and PD, read the significant publications and score the individual.

Supplement the package with additional information – As a minimum, consult the supervisor; then, consult as many references as necessary to understand the body of work. You may use, but are not limited to, the references provided at the end of the RDSR. You may not consult with the individual being reviewed. If you run into problems, contact the RGE/EDGE coordinator.Make certain you will be able to address each of the four factors and clarify scientist’s role on scientific teams/co-authored publications if it is not clearly indicated in RDSR. For each scientist you will present: Provide a digital primary reviewer write up for each of the four factors to the RGE/EDGE coordinator before the panel begins. The panel will use this to help craft the feedback, so do not quote from scientist’s PD, RDSR, or written responses from references.All communications with references must be confidential.

10

Slide11

Roles and responsibilities – primary reviewer (cont’d.)

Areas that may need clarification:

Does the scientist generate ideas/make interpretations independently?

What is the employee's role in cooperative efforts? Lead role or team member role? What were the scientist's contributions?To what extent are existing technologies/methodologies applicable to the work? Has the employee moved science/technology forward?What was the scientist's role in listed significant contributions, significance of results, and impact? Do the individual and (or) collective team contributions move the science/technology forward? Advance USGS strategic goals? Have societal impact?11

Slide12

Roles and responsibilities – primary reviewer (cont’d.)

Areas that may need clarification (cont’d):

How widely is the employee recognized and consulted in their field? Employee’s reputation among scientific and technical peers? How is leadership manifested in the employee’s record?

Have recent products been assessed by scientific/technical peers? Has sufficient time lapsed to assess the impact and relevance of findings?Does the employee perform a full range of activities in the research/development cycle (from conceiving ideas through writing interpretive reports for technical and non-technical audiences)? Communicate with non-science groups or policymakers regarding nature, impact, and application of results to societal problems?12

Slide13

During the meeting:

Present information about the assigned employee’s career, current assignment, scope of responsibility, contributions, and impact. Clarify written materials for other panel members and provide information that is lacking in the written material but is needed to make a fair evaluation.

Do not advocate. Be objective in presenting and clarifying information for the panel.

Roles and responsibilities – primary reviewer (cont’d.)13

Slide14

Roles and responsibilities – panel chair

Before the panel meets:

Assign a primary reviewer for each scientist being evaluated. May consult with Science Center Directors, National Capabilities Mangers, RDs, ADs, RGE/EDGE coordinator and other appropriate personnel on these assignments. Remind primary reviewers of their responsibilities. Strongly encourage panel members to attend training.

Emphasize to all panel members the importance of conducting a thorough and objective review and rating of each scientist before the panel meets and the importance of maintaining confidentiality. Panel members should not rely on the primary reviewer as a primary source of information.Ask primary reviewer to provide a descriptive write up for each of the four factors to the RGE/EDGE coordinator before the panel begins.

14

Slide15

Roles and responsibilities – panel chair (cont’d.)

At the start of the panel – provide guidelines and emphasize:

Importance of objective, unbiased ratings using RGEG or EDGEG

The importance of providing constructive feedback to the employeeImportance of evaluating the quality and impact of publication and non-publication contributions when scoring factor 4How to evaluate contributions that are part of a scientific teamImportance of identifying when RGEG or EDGEG is not appropriate for evaluation. Note: research/development must be at least 25% of efforts to be rated by the panel15

Slide16

During the panel:

Ensure that each panel member’s scores are posted for each of the factors before beginning discussion of employee.

Ensure primary reviewer presents information to the panel for discussion. The panel is free to consult with anyone (except for the scientist being reviewed) who has additional information relevant to a case, in person or by phone during the panel.

Lead and manage discussion. Ensure that all panel members participate and that discussion stays on track.Roles and responsibilities – panel chair (cont’d.)16

Slide17

During the panel (cont’d.):

Discuss all four factors to provide documentation for each consensus score.

Ensure that differences among scores are thoroughly discussed (especially >2 points for factors 1-3 and >4 points for factor 4).

Work towards getting consensus scores. Ensure that the final consensus score for each factor is justified in writing and that the justification correlates with the criteria for that score under the RGEG/EDGEG.Roles and responsibilities – panel chair (cont’d.)17

Slide18

At the conclusion of the panel:

Ensure that the final summary comments on each Individual Panel Results form is approved by all panel members.

Complete

and sign the Panel Recommendation Summary form. Send final documentation to the appropriate RGE/EDGE coordinator, John Thompson or Judy BackRoles and responsibilities – panel chair (cont’d.)18

Slide19

Roles and responsibilities – RGE/EDGE coordinator

Before the panel meets:

Ensure that all panel members have access to RDSR packages at least two weeks before the panel meets.

Provide training. Strongly encourage panel members to attend training.Work with panel chair to determine date and location of panel.Start of panel – provide guidelines and emphasize: Importance of confidentiality of materials and discussions – nothing is to be shared outside of the panel deliberationsThe process that will be used to discuss and evaluate each employee and the importance of reaching consensusIt is never appropriate to discuss personal/performance issues about an employee during peer-review panel discussions

19

Slide20

Roles and responsibilities – RGE/EDGE coordinator

(cont’d.)

At the conclusion of the

panel:Re-emphasize the importance of confidentiality. Panel members must not disclose any information/panel deliberations. It is line management’s responsibility to provide feedback to the employee.Send panel results (final Individual Panel Results forms for scientists not going forward to the second-level panel) to the Science Center Director/National Capability Manager.Reiterate that all digital and non-digital documentation (RDSRs, notes, etc.) from all panel members, other than the final copies of the Individual Panel Results and Panel Recommendation Summary forms must be deleted or shredded.20

Slide21

Special topics: Role of the position description

The current position description (PD), including a copy of the signed coversheet, is used by the panel to clarify the official work assignment (factor 1).

If a PD is inaccurate, the employee is responsible for drafting a statement that outlines the current work assigned by management and obtaining the supervisor’s signature to show concurrence. This should be part of the RDSR package.

Panel members should note discrepancies between the official PD of record and the assignment (section 11) and accomplishment (section 12) portions of the RDSR. Where appropriate, panel results should recommend that the PD be updated and should note the impact of discrepancies on the scientist’s career.21

Slide22

Special topics: Mixed positions

Panels only evaluate the research/development portion of the position, but need to understand the scope of the whole position to give proper weight to the

recency

of the work.Panels may have difficulty distinguishing between research/development and operational work. If necessary, Discuss questions regarding the nature of the work with the supervisor.Obtain position classification advice from the servicing personnel office.22

Slide23

Special topics: Scientific Teams

When assessing Factor 1, consider the individual’s responsibility for research priorities, guidance, and productivity and credit the assignment in the following manner:

Assessment of Team Leaders/Project Chiefs should reflect the scope and character of the entire project being conducted by the team, provided the scientist actively guides the research being done by the entire project. If the Team Leader/Project Chief does not actively guide the research of the project, his/her role should be assessed in the same manner as a team member.

Assessment of a team member’s assignment should reflect the specific project, nature, scope, and difficulty of the assignment conducted by the individual and the contribution of that work to the impact of the larger project.23

Slide24

Special topics: Assessments and Monitoring

Assessments and monitoring are mission critical for the USGS and can include a mix of operational as well as R&D components.

An employee’s R&D role in an assessment or monitoring project depends on the performance (originality, creativity, etc) of the person in the job. If a scientist is spending at least 25% of his/her time and effort conducting R&D, it is appropriate for the scientist to be evaluated by a peer-review panel.

Examples of assessment/monitoring-related research activities: Development of new assessment/monitoring protocols and toolsDevelopment of innovative decision support system toolsUse of assessment/monitoring data to develop/test hypothesesDevelopment of new predictive modelsInnovative integration of interdisciplinary assessment /monitoring resultsDiscovery of previously unknown processes or principles

24

Slide25

Special topics: Technology Transfer

Synthesizes and translates scientific findings and/or results into a more usable format for decision makers and adds value to the science through additional interpretation, synthesis, modeling, or translation.

Should be directly related to and derived from official peer-reviewed R&D accomplishments and considered as evidence of the employee’s stature in the field.

Panel members should consider: the impact of these activities on R&D in the field of expertise; contribution to the resource and the customer; and measurement of quality and quantity. Examples of science application and transfer products:Web-delivered analytical toolsModelsMaps and GIS productsDecision Support ToolsReports (USGS series and other) to actions agencies (e.g., BLM, FWS, EPA and their state and local counterparts, NGOs and the private sector)

25

Slide26

Conversion scales for RGE and EDGE

Grade

RGE

EDGE

GS-11 8-14

8-11

GS-12

16-24

13-16

GS-13

26-34 18-21 GS-14 36-44 23-26 GS-15 46-50 28-31 ST >56

>

33

26

Slide27

Where to find the information in the RDSR

RGE

1. Research assignment

a) assignment

Sections 11-12, 20 b) complexity

Sections 11-12, 20

c) importance of results

Sections 11-12, 19

2. Supervisory controls

Sections 11-13, 20

3. Guidelines a) guidelines availableSections 11-12, 20 b) originality requiredSections 11-12, 18-204. Contributions, impact and stature a) demonstrated abilitySections 9-10, 13-14, 17-19

b) products/publications

Sections 15-16, 18-19

c) advisory/consult/scientific activities

Sections 13-15, 17

27

Slide28

Where to find the information in the RDSR

EDGE

1. Nature of assignment

a) scope

Sections 11-12, 20

b) complexity

Sections 11-12, 20

c) importance of results

Sections 11-12, 19

2. Supervision received

Sections 11-13, 203. Guidelines and originality  a) guidelines availableSections 11-12, 20 b) originality requiredSections 11-12, 18-20 c) demonstrated originality

Sections 12, 16, 18-20

4. Qualifications and Scientific Contribution

 

a) Contributions of new or improved products, processes, and techniques

Sections 12, 15, 19

b) products/publications

(papers; design concepts, criteria and data;

laboratory fabrication techniques and processes;

and patents and inventions)

Sections 12, 15-16, 18-19

c) advisory/consult/scientific committee activities

Sections 13-15, 17

28

Slide29

Final panel documentation – only two panel forms are preserved:

(1) Individual Panel Results Form – for each scientist reviewed

29

Slide30

Individual Panel Results Form

Documents panel’s assessment in relation to four factors. Include final consensus score and justification (1-2 paragraphs/factor) for each factor.

Provide helpful and constructive feedback to employee.

Be accurate. Cite the scientist’s record and refer to the checklist. Substantiate panel findings.Do not quote from the scientist’s RDSR, position description, or others you have consulted for the review.Forwarded to the second-level panel when reviewing recommendations for promotions to GS-14 and above and below-grade findings.

30

Slide31

Individual Panel Results Form, cont’d.

Factor 1: Assignment

What research/development is the scientist currently doing and what is the scope of the work?

How complex are the problems being addressed?What is the importance of results/expected results?Factor 2: Supervision How is the work assigned?How much guidance is given?What level of responsibility does the scientist have?How are the results reviewed?

31

Slide32

Individual Panel Results Form, cont’d.

Factor 3: Guidelines and Originality

What guidelines (literature, procedures, instructions, precedents) are available for the current work?

What is the level of originality that the scientist brings to the work?Factor 4: Contributions, Impact, and Stature (RGE) Qualifications and Contributions (EDGE)What are the scientists publication and non-publication contributions?What impact have the scientist’s contributions had? Include evidence.What is the stature of the scientist in his/her field? In the broader scientific community? Include evidence.

32

Slide33

Final panel documentation – only two panel documents are preserved:

(2) Panel Recommendation Summary Form – for each panel

33

Slide34

Resources:

Office of Science Quality and Integrity intranet site,

http://internal.usgs.gov/quality_integrity/rge_edge/

RDSR portal (http://internal.usgs.gov/quality_integrity/rge_edge/RDSRportal.htm) - used to submit RDSR packages for review. Office of Personnel Management’s (OPM’s) Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) guidesUSGS Research and Development Evaluation Process Handbook -how USGS implements the OPM guidesRGE, EDGE, and ST ChecklistsTraining materialsRGE/EDGE coordinators:Judy Back and John Thompson, rge-edge@usgs.gov

34