Competencies for Effective PEQA in VR - PowerPoint Presentation

Competencies for Effective PEQA in VR Scott Sabella, University at Buffalo Matt “Mark” Markve, Idaho DVR Michael Shoemaker, Utah State Office of Rehabilitation

Competency Identification Competency is the ability to apply knowledge and skills to real world circumstance in order to get a desire outcome (Benner, 1982; Trinder , 2008) A knowledge base of information that serve as the foundation for understanding The skills necessary to successfully apply the knowledge The clinical judgment to apply the knowledge and skills appropriately (Dunn et al., 2000 )

Why is establishing competencies important? Stevahn & King, 2005: Improved training for new and experience professionals Enhanced reflective practice The expansion of understanding and research on evaluation The advancement of the professionalization of the field They give the basis for discussion, refinement and potential adoption by credentialing or licensing agencies Engenders public trust

Brief History of the Project Summit Group IRI on Performance Management Listserv question “What are the 5 main competencies that VR program evaluators need to be effective?” Qualitative thematic analysis conducted on the 24 respondents with 111 different unique statements

Results: Six Self-Identified Competencies

Remaining Questions Question 1: What is the relative importance of each of these self-identified competencies? Question 2: Are these themes stable or has the field of PEQA in state VR evolved? What are the principle competencies that are currently perceived as most important? Question 3: How do these competencies relate to the current competency standards within the fields of rehabilitation and program evaluation?

Methods: Mixed Approach Question 1: What is the relative importance of each of these self-identified competencies ? Method: Participants were asked to rank the six-self identified competencies; t-tests were performed to determine whether some competencies were significantly more important than others. Question 2: Are these themes stable or has the field of PEQA in state VR evolved? What are the principle competencies that are currently perceived as most important ? Method: Participants were asked an open ended question “What are the top 5 five competencies that are most important for a successful state VR agency program evaluator?” Themes were analyzed using the constant comparative method (qualitative analysis) Question 3: How do these competencies relate to the current competency standards within the fields of rehabilitation and program evaluation ? Emergent themes were compared to the CORE, CACREP standards and the Stevahn et al (2005) competencies for program evaluators.

Results Question 1 What is the relative importance of each of these self-identified competencies?

Which do you think are most important and which are least important?

What we found Weighted importance scores for the six-self identified competencies: Mean SD Data analysis and interpretation 4.53 1.29 Knowledge of VR 3.98 1.93 Interpersonal skills and communication 3.88 1.63 Judgements and recommendations 3.42 1.36 Objectivity* 3.07 1.67 Report writing and presentation* 2.12 1.28 How many of you are called upon are required to do presentations as a main part of your job? How many of you are called upon to do data analysis report writing as a main part of your job?

Results Question 2 What are the principle competencies that are currently perceived as most important? Methodology and data analysis (53 responses) Interpersonal skills and effective communication (32 responses) Project management (30 responses) Knowledge of the state VR system (30 responses) Critical thinking (24)

Methodology and data analysis Examples from the field?

Interpersonal skills and effective communication Examples from the field?

Project Management Examples from the field?

Knowledge of the state VR system Examples from the field?

Critical Thinking Examples from the field?

What did we miss?

Program Manager III:Program Quality Assurance, Policy, & Planning Selective Qualifications: knowledge of the administration of vocational rehabilitation programs including State and federal regulations governing the rehabilitation or disability determination process; knowledge of the principles of effective supervision and management of employees; knowledge of program evaluation and policy development ; knowledge of principles of organizational planning, budgeting, staffing and management; ability to communicate clearly and effectively; ability to interpret laws, regulation policies and procedures; ability to establish and maintain effective working relationships with other rehabilitation officials, State and federal legislators, employees and the general public. State of Maryland Job Openings

Results Question 3 How do these competencies relate to the current competency standards within the fields of rehabilitation and program evaluation ? 1. Council on Rehabilitation Education (CORE) 2. Council for Accreditation of Counseling & Related Educational Programs (CACREP) 3. Program evaluation competencies ( Stevahn et al., 2005)

CORE standards Yes: Methodology and data analysis, knowledge of VR No: Interpersonal skills and effective communication, project management A little: critical thinking (ethical judgement) CACREP standards Yes: Methodology and data analysis, knowledge of VR No: Interpersonal skills and effective communication, project management A little: critical thinking (ethical judgment) Program evaluation competencies ( Stevahn et al., 2005) Yes: Methodology and data analysis, interpersonal skills and effective communication, project management, critical thinking Maybe: knowledge of state VR system

Overview of Results Methodology and data analysis appears to be the most important competency Report writing and presentation is not a main PEQA VR competency Project management has emerged as a prominent theme for PEQA specialists in VR Critical thinking has also emerged and subsumed judgments and objectivity. CORE and CACREP standards include many of these competencies, but still much room for reinforcing select competencies, like project management, critical thinking, and interpersonal skills and communication outside of the therapeutic relationship

Final Thought Questions and Final Thoughts Need for well-balanced and well rounded PEQA professionals in VR How may these competencies help “accidental evaluators” ( Stevahn et al., 2005) How may these competencies help experienced evaluators? How may these competencies enhance professional’s “reflective practice ”

Thank you!!!!! Scott Sabella, sasabell@buffalo.edu , 716-645-1123 Matt “Mark” Markve, matthew.markve@vr.Idaho.gov , 208-287-6481 Michael Shoemaker, mtshoemaker@Utah.gov , 801-238-4588