Worth Watching: Assessment of Video Tutorials in Academic L - PowerPoint Presentation

Slide1

Worth Watching: Assessment of Video Tutorials in Academic Libraries

Sally Ziph (Librarian, Ross School of Business, University of Michigan), Jennifer Lammers Zimmer (Librarian, Ross School of Business, University of Michigan), Kelly Burr (Digital Technologies Assistant, Ross School of Business, University of Michigan)*With special thanks to Elizabeth Beers, Assessment Librarian, University of Chicago

1Slide2

OverviewBackground informationOverview of projectsResultsConclusionsGoing forward

2Slide3

Background - 2010 and 2012 SurveyNew MBA Consortium studentsAsked us to create the video tutorials in 2009Videos were intended to help complete a specific task: i.e. Build-a-list in OneSource

3Slide4

Background – Testing GroupMBA Consortium Student Group 2013:Age: 26-35 Minority studentsMet with job recruiters off-campus before ever coming to Kresge

4Slide5

Background - Creation of VideosWrote scripts based on FAQsTechnologies: Audacity for audio and Camtasia for screen captureAdvertised videos by emailVideos added to our Instruction page

http://www.bus.umich.edu/kresgelibrary/support/instruction.htm5Slide6

Overview of Two Projects2010: How do we evaluate the effectiveness of non-interactive task-based point-of-use video tutorials?2012: Can grad students (MBAs) learn a library database task through a

screencast video tutorial?In 2012, we wanted to delve deeper into student learning outcomes.6Slide7

2010 Literature SearchHow do we evaluate the effectiveness of non-interactive task-based point-of-use video tutorials?Best practices included:Pre/post TestsA/B tests

Focus groups and user interviewsSurveys Statistics7Slide8

Survey Methodology 2010Questions made up by librarians (based on our experience)Qualtrics survey emailed to MBA Consortium students

8Slide9

Screen Shot 2010 Survey9Slide10

Results 201030% students responded to email survey (13 out of 45 total)23% of respondents had used the videos77% of respondents didn’t use videos because:Didn’t know they were availableVideos were not “relevant”

10Slide11

Conclusions From 2010 ProjectList of Best Practices and pros/cons of each practiceWe concluded that future evaluation should include:Continued gathering of usage statisticsFocus groups to refine tutorials/topics

Testing methodologies that more accurately measure learning outcomes and usefulness11Slide12

Conclusions 2010, con’dClarify what the Kresge Library can do to help MBA Consortium studentsWe need to send reminders at strategic times

12Slide13

2012 Research QuestionCan grad students (MBAs) learn a library database task through a screencast

video tutorial?Same MBA Consortium Group population (same demographics)Café coupon incentive ($10)13Slide14

Survey Methodology 2012Based on paper by colleaguesQuestions vetted by UM Center for Learning & Teaching expertTested Qualtrics survey on other librarians

14Slide15

2012 Pre-test/Post-test and SurveyThe Survey questions of our Qualtrics questionairre were qualitative

i.e., different questions before and after the video tutorialPre-test questions and screenshots (before the video tutorial) were exactly the same as Post-test questions15Slide16

Screen shot 2012 Survey16Slide17

2012 Survey Video17Slide18

Results Show Learning Has Taken PlacePre-test Post-test

18Slide19

Results Show Confidence Has RisenPre-test Post-test

19Slide20

2012 ConclusionsGrad students do learn how to do library database tasks through screencast video tutorials We would like to do further statistical analysis to pinpoint statistical significance. (UM Center for Statistical Consultation and Research)

20Slide21

“Bonus Conclusions”Students’ level of enthusiasm and participation Large number of thoughtful and helpful commentsLibrarians surprised : video tutorials made such a marked difference in student learning outcomes.Slide22

2010 vs. 2012 BOTH surveys suggest students need and want reminders of Kresge Library services and resourcesVideo creation is worth our time and is effective way to instruct students

22Slide23

Further research questionsGet students back 2 weeks later—retention and transfer of learning?Learning styles?Evaluate video tutorial vs. in-person vs. podcast vs. print using A/B testing?Slide24

Translating Data into Actions Know videos are a good toolHave to decide what topics to coverHow to measure usage Advertising and placement

24Slide25

Practical ConsiderationsEach video represents about 10-12 hours of workWho will do writing, editing, recording?How to keep these updated?What are we going to do with our podcasts?

25Slide26

Questions?26