Research Analysis of Faculty Peer Review Comments

Transcript:Research Analysis of Faculty Peer Review Comments:
Research Analysis of Faculty Peer Review Comments QM Conference 2014 Dr. Andria Schwegler Dr. Barb Altman Texas A&M University – Central Texas Presentation Agenda Research Overview and Background TAMUCT Internal Peer Review Process Research Design Coding Scheme and Demonstration Summary Data of Our Reviewer Comments Thematic Analysis of Our Reviewer Comments Potential Solutions to Issues Found – External Reviews? Potential Solutions to Issues Found – What does Theory Tell Us? Implications – Revisions to Peer Review Process and Training Future Research Schwegler & Altman 2014 QM Conference Presentation 2 1. 1. Research Overview & Background Goal: To empirically examine the content of the peer review comments elicited in our first round of internal Quality Matters peer reviews at Texas A&M University Central Texas. Purpose: To better understand what it was about the peer review comments that led to the most effective improvements in reviewed courses. Comments were evaluated with three frames in mind: QM standards of peer review comments that are constructive, specific, measurable, and balanced. 2. Qualitative thematic analysis of overall comments. 3. Theoretical framework of literature on effective feedback for learning from the cognitive psychology literature. Schwegler & Altman 2014 QM Conference Presentation 3 2. 2. TAMUCT Peer Review Process Details on our process and a related research project on our faculty perceptions regarding participating in peer review can be found here: Schwegler, A. F., Altman, B. W., & Bunkowski, L. M. (2013). Beliefs regarding faculty participation in peer reviews of online courses. Internet Learning, 3, 95-116. http://www.ipsonet.org/images/Westphalia_Press/Internet_Learning_Journal_2-2/3-1/8.%20Schwegler%20ILJ%203-1.pdf Schwegler & Altman 2014 QM Conference Presentation 4 2. 2. TAMUCT Peer Review Process An internal peer review process based on the Quality Matters (QM) framework was initiated over a 1.5 year period ending in Fall 2013. We had to incentivize process to overcome faculty resistance. We had to keep all reviewers internal to Colleges. Though faculty were trained on QM, we had to reinforce differences between design and delivery. Faculty completed a self-review prior to full review commencing; College Online Coordinator provided input that was voluntary to correct. Schwegler & Altman 2014 QM Conference Presentation 5 2. 2. TAMUCT Peer Review Process Across 4 semesters: 37 courses were peer reviewed 19 different faculty course developers 26 additional faculty members did not offer a course for peer review, but did volunteer to serve as peer reviewers Schwegler & Altman 2014 QM Conference Presentation 6 2. 2. TAMUCT Peer Review Process