Sadan KulturelKonak Abdullah Konak Penn State Berks Motivation Design a unified framework for assessing professional skills Teamwork is ubiquitous in entrepreneurship classes Assessing teamwork ID: 553081
Download Presentation The PPT/PDF document "Assessing Teamwork in Entrepreneurship C..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Assessing Teamwork in Entrepreneurship Classes
Sadan Kulturel-Konak Abdullah KonakPenn State BerksSlide2
MotivationDesign a unified framework for assessing professional skills.Teamwork is ubiquitous in entrepreneurship classes.Assessing teamwork knowledge, skills
, and abilities is challenging. Slide3
Challenges in Assessing Teamwork SkillsSlowly matureDeveloping tools and rubrics that can track the student development Students’ concernsSlide4
Challenges in Peer AssessmentLabor intensiveTime consumingFeedbackConfidentiality Slide5
Peer Evaluation ToolsCommunications Project Assessment Tool (COMPASS) (Thomas et al., 1999)Comprehensive Assessment of Team Member Effectiveness (CATME) (Loughry
et al., 2007)Self and Peer Assessment Resource Kit (SPARKplus) (Willey & Gardner, 2009)Slide6
Peer Evaluation & Assessment Resource (PEAR) Designed for assessing teamwork Knowledge, Skills, and Abilities Provides feedbackPreserves confidentiallyBased
on the Model of Domain Learning (Alexander et al. ,1994)Slide7
Model of Domain LearningI should
ask questions.Leading , recall/process, probing
, and rhetorical.
Cool !Attend to the next seminar.Slide8
A Sample Mapping
Expected Competency /Attitude (5-point Likert Rating scale)
I
KSACPInterested in the use of internet communications technologies to communicate
x
x
Uses internet communications technologies skillfully
x
x
x
Initiates the use of most appropriate communications technologies for specific tasks
x
x
Express a personal level interest to learn and use new internet communications technologies
x x Expresses opinions and ideas in a clear manner in electronic correspondences x x Expresses opinions and ideas in a clear and concise manner with obvious connection to topic in electronic correspondences x x
competency/attitude: “use
communication technologies skillfully” Slide9
PEAR Conceptual ModelSlide10
Creating a RubricSlide11
Creating an Assessment
Select peer and self evaluation rubrics
Enter the course data
Add teamsInstructors can select customized or standard rubricsSlide12
Conducting an Assessment
The instructor sends emails
Each student receives a unique
, encrypted URL link.Students rate themselves and their peers Slide13
ReportsSummary ReportsQuestion-based Student-basedMDL-based Export to ExcelEmail feedback to studentsSlide14
A Pilot StudyEntrepreneurial Mindset Class (22 students)Three team projects (different teams)The eleven-item rubric by Van Duzer and
McMartin (2004) and five point Likert-type scaleTotal of 2615 ratingsSlide15
Peer (212) vs Self (58) Ratings
No
Questions
PeerRatingSelf Ratingp-value1Communicated ideas clearly/effectively.4.22
4.61
0.004
2
Contributed useful ideas that helped the group succeed.
4.25
4.65
0.007
3
Delivered work when promised/needed.
4.21
4.58
0.032
4
Encouraged group to complete the project on a timely
basis.
4.16
4.58
0.011
5
Failed to do an equal share of the work (Reverse).
2.65
1.68
0.0076Had difficulty negotiating issues with members of the group (Reverse).2.712.120.0667Helped group overcome differences to reach effective solutions.4.164.520.0088Kept an open mind was willing to consider other’s ideas.4.494.650.3159Often tried to excessively dominate group discussions (Reverse).2.702.420.16010Took a leadership role in some aspects of the project.3.94
4.40
0.005
11
Was fully engaged in discussions during meetings.
4.25
4.67
0.003Slide16
Pitfalls Very high ratings, not consistent with the instructor observations Some questions were avoided Deliberate versus non-deliberate evaluationsSlide17
Sample Interest Questions Attended a speaker event about teamworkWatched a documentary or training video about teamworkWatched a video clip outside of class work about teamwork
Attended a workshop about teamworkAsked questions to an expert (professor, consultant etc.) about effective teamworkPerformed a web search to learn about effective teamwork
Read a book about teamworkRate your level of interest in attending a free workshop on teamwork.
Rate your level of interest in reading literature about effective teamwork.While you are browsing a news website, you have spotted an article called “How to be Effective in Teamwork.” Rate your likelihood of reading this article?Rate your level of willingness to take an elective course in order to improve your teamwork skills?In your institution, a renowned teamwork guru will give a workshop on teamwork skills. If you have to pay $10 for this workshop, rate your level of interest in attending this workshop.Read an online article about teamworkRead a newspaper/magazine article about teamworkSlide18
Why should interest be measured?
Variable (Cronbach's
Reliability)
GPA GroupClassStandingHave you ever had a job that requires significant teamwork?Have you ever taken a course or training on teamwork?Goal Setting (.712)-.019
.016
.154
**
.090
*
Performance Evaluation (.637)
-.010
.048
.105
*
.079
Team Forming (.827)
-.074
-.038
.100
*
.082
*
Team Coordination
-.006
-.026
.130
**.054Communication (.807)-.127**.032.120**.126**Conflict Resolution (.816)-.091*-.048.111**.067Problem Solving (.810)-.044.011.147**.089*Interest1 (.850)-.014.097*.135** .456**
Interest
2
(.803)
.011
.090
*
.037
.159
*
Interest
3
(.794)
-.006
.118
*
.170
**
.290
**
Attitude (.730)
-.245
**
.104
*
.153
*
.076
Benefit (.817)
-.191
**
.071
.119
*
.055
Based on 586 samples, we observed stronger correlations between class standing and interest than between
class standing
and self-efficacy (significance p *: 0.05 and **:0.01).
Self-EfficacySlide19
Current WorkSlide20
Conclusions PEAR facilitates peer and self evaluations. Interest should be used as an additional construct to measure students’ professional skill development (teamwork) Slide21
Acknowledgments “A Modular Assessment Framework for Professional Skills Using A Model of Domain Learning Approach”-NSF DUE-1044800
“A Virtual Incubator @ Penn State Berks (VIB) to Foster Student Innovations”-VentureWell 8804-11 Slide22
Questions?konak@psu.edu, Abdullah Konaksadan@psu.edu, Sadan Kulturel-KonakIvan Esparragoza Gül
E. Okudan KremerGareth YoderSteven Magluilo