/
Amplifire Amplifire

Amplifire - PowerPoint Presentation

tatiana-dople
tatiana-dople . @tatiana-dople
Follow
374 views
Uploaded On 2017-01-29

Amplifire - PPT Presentation

Usability Test Findings and Recommendations DRAFT 1 Project Overview 2 Amplifire usability test objectives Validate usability of the overall design approach and interactions Gather feedback on specific design innovations including the progress bar transition between questions and explana ID: 515156

question participants answer questions participants question questions answer video progress review feedback introduction positive explanation recommendations answering button page

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Amplifire" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Amplifire

Usability Test Findings and RecommendationsDRAFT

1Slide2

Project Overview

2Slide3

Amplifire usability test objectives

Validate usability of the overall design approach and interactions

Gather feedback on specific design innovations, including the progress bar, transition between questions and explanations, and confidence indicators

Identify opportunities to improve ease of use:

aspects of the design that hinder task completion, cause confusion, or negatively impact user

experience

3Slide4

Project structure

Acclaro and Knowledge Factor recruited 6 participants who matched the amplifire target demographic.2

college students with some experience using online study

tools and

flashcard mobile

apps

2 recent college graduates, with work experience and an interest in graduate-level

programs

1 college

instructor

1 hospital nurse, familiar with online training coursesEach participant used amplifire during a one-on-one session with the test moderators:Molly moderatedJames took notes and analyzed dataParticipants completed a module unassisted and were interviewed about their experiences.Sessions were recorded.

4Slide5

Themes

1:

Usability findings and recommendations align

with eight broad categories

5

Dashboards are simple and easy

Welcome video and module introduction pages provide inadequate orientation

Feelings of being tested discourage honesty

Answering questions is difficult to learn

Streamlined question sets and explanation pages were liked

Progress indicators are too subtle

Review, refresh, and flashcards are on track

Going back

introduces confusion

2:

3:

4:

5

:

6:

7

:

8:Slide6

1:

Positive feedback:General

impressions were positive – simple and clean

Launching modules and changing courses

were

easy

The

new dashboards were useful

and easy to

interpret

The term “module” is intuitiveOpportunities:The Learn link is hidden when a course contains chapters The Change Book control was not noticed, so assignments were missedDashboards are simple and easy6Slide7

Positive feedback

Participants described amplifire as “clean” and “easy to follow”.

Participants were able to understand the purpose of amplifire by looking at the dashboard.

Most participants had no trouble locating the Learn link to begin modules.

All participants found and successfully used the My Courses  drop down in the header.

All

participants were comfortable with the term “Module”.

1: Dashboards

7Slide8

New dashboard mockups well received

1: Dashboards

8Slide9

New dashboard mockups well received

The new dashboards were perceived as useful and easy to interpret.

Participants

liked that estimated time remaining was

shown, and commented

that they would start a module only if they had enough time to finish it in one study session.

Participants thought that showing progress through each book (rather than for the entire course) was appropriate.

Participants understood and liked the “Knew It” / “Learned It” chart. They said it would guide them in determining what

to

study.

One participant was an instructor, so we showed him the instructor dashboard mockup. He thought the information presented would help him decide which topics he needs to address in class. He would like it to be broken down by class section.1: Dashboards

9Slide10

Opportunities

After switching courses, participants hesitated because the Learn link is hidden.Only one participant noticed the Change Book link without prompting. To ensure that assignments are not missed, the control needs to be improved.

10Slide11

Recommendations: Dashboards

If a course contains multiple books, make the list of books and method for changing books more obvious.

On

page load, open the first (or most recently-used) chapter to show modules

.

11

1: DashboardsSlide12

2:

Positive feedback:Participants thought the introduction page was informative

Participants liked the video

Opportunities:

The introduction page is too long its purpose is unclear

The

video is

shown at the wrong time and can’t be accessed again

later

Welcome video and module introduction pages provide inadequate orientation

12Slide13

Positive feedback

Participants liked the style and tone of the welcome video.Participants who read the introduction page found the information about how amplifire works to be informative.

2. Video and introduction

13Slide14

Opportunities: Introduction page

Participants said they would not read the introduction page because it is too long.

The purpose of the

page

was unclear:

Is this page about

the course content or about

amplifire

?

Will I be tested on this information?

(While answering questions) Can I go back to review the Introduction page? 2. Video and introduction14Slide15

Opportunities: Welcome video

“This is not a test” is a catchy introduction, but is presented out of context.

Few

retained the information that

the video presented, and there is no way to review the video.

One

thought the video was a commercial, and tried to close

it immediately

. She then wished she could view it again.

2. Video and introduction

15

Highlights: Participants’ reactions to the

introduction and welcome videoSlide16

Recommendations

Introduction:Improve labeling on the introduction.

Break the page into two clear

sections (or pages?)

– about this module and how online learning

works

Or, make the introduction only about the module content, and pull how online learning works into a separate opt-in tutorial.

Video:

Wait until the user has answered a few questions to show a prompt for the video. (Put the video into context and allow the user to opt in.)

Add a Help option in the global navigation to allow users to access the video and informational overlays later.

Include the video in the dashboard as a widget to access later.16

2. Video and introductionSlide17

Prompt to watch video

17Slide18

Suggested revisions to video script

Revise the video script to clarify why amplifire uses a test-like format:

Answering

questions makes your brain learn faster (line graph spiking ever upward)

And shows where you need to spend time (question mark over head, furrowed eyebrows)

And what you can skip (check mark, “got it!”, lightbulb)

So you learn

better…faster

18

Thanks, Matt and Charles!Slide19

Video widget in dashboard

19Slide20

3:

Opportunities:Participants felt like modules were tests

Participants were more inclined to guess than choose “I Don’t Know”

Feelings of being tested discourage honesty

20Slide21

Students bring a strong mental

model that any system asking questions is testing them.

Aspects

that created a test-like feeling:

The correct answer is

withheld.

Learners can’t go back to

view a previous question.

Learners can’t change their responses after learning that they missed a question

Participants wondered

how the modules would “be graded” and whether instructors would be able to see students’ grades.3. Feelings of being tested21

Opportunities: Feelings of being testedSlide22

Participants didn’t use “I Don’t Know”

Participants were more inclined to guess than choose “I Don’t Know

”.

Participants

’ past experience with test taking tells them that it’s better to guess and possibly get the answer correct than to not select an answer.

3. Feelings of being tested

22

Highlights: Participants’ feelings of being tested

“I’m not going to risk getting it wrong

”.

- Recent college graduateSlide23

Partially sure

encouraged dishonesty

Learners relied on

partially sure to make a random guess that covers 2/3 of the answers, even without any knowledge on the topic.

23

Highlights: GuessingSlide24

Recommendations

Show the prompt for the video later in the Q&A flow.Detect patterns in learners’ answer selection behavior and display a prompt to remind them that this is not a test.

If learners repeatedly answer questions by selecting two responses, encourage them to choose I Don’t Know Yet if they truly don’t know.

If a learner repeatedly selects just one answer and is wrong, ask them if they are sure before they click Submit.

How important is this problem? Does a little test anxiety improve learning?

24

3. Feelings of being testedSlide25

4:

Positive feedback:Once participants learned how to select answers and indicate certainty, they moved quickly through questions

Opportunities:

Answer selection

was difficult to learn

When

unsure,

participants wanted to select only one answer

C

onfidence

dots to the left of the Submit button seemed unnecessaryAnswering questions is difficult to learn25Slide26

Positive feedback

Once participants figured out how to answer questions, they were able to move through questions efficiently.

4. Answering questions

26Slide27

Opportunities: Answer selection difficult to learn

Most participants dismissed the answer selection instruction overlay without fully reading it. One

later wished

she could re-open it to review the

instructions.

4. Answering questions

27Slide28

Opportunities: Answer selection difficult to learn

Participants didn’t initially realize how to activate the Submit

button.

Three

participants clicked one answer and then sat, wondering why the Submit button was

disabled.

They hovered the mouse over the submit button, and nothing happened.

4. Answering questions

28Slide29

Answer selection difficult

to learnTwo accidentally discovered that clicking two responses would activate the Submit button. Those participants answered the next few questions by clicking two answers, unclear why the system “required” this.

One

participant clicked an answer twice

because she thought that was required for activating the Submit button, even

though she wasn’t sure of her

response.

4. Answering questions

29

Highlights: A participant struggling with answer selectionSlide30

Participants wanted to select one answer and “I am unsure”

Selecting two answers didn’t match the way participants think about uncertainty.

Most participants attempted to

guess

by clicking submit after selecting only one partially sure answer. The system doesn’t allow this, which forces them to choose a second

guess.

Two participants told us they wanted to

be able to select just one answer, and indicate that they weren’t sure

.

4. Answering questions

30Slide31

Confidence dots are unnecessary

None of the participants noticed the confidence

dots to the left of the Submit button.

When

asked,

all

but one said they seemed redundant with the click twice/click two answers

convention.

4. Answering questions

31Slide32

Recommendations: Answering questions

Minimum:

Add

tooltip over the select

button

Add

a

Help or Tips option in the global navigation to allow users to access the video and informational overlays later

.

Remove the confidence dots

324. Answering questionsSlide33

Recommendations: Answering questions

Bigger Picture:

Consider changing

answer selection to allow single click and

submit. Include a tip explaining that clicking an answer once means you’re unsure, and twice means you’re sure.

Detect

patterns in learners’ answer selection

behavior. If

the user is confident and incorrect more than 2 or 3 times, display a prompt – if you’re not sure, try selecting

your answer once or selecting two

answers.334. Answering questionsSlide34

Recommendations (continued)

Bigger Picture:

Consider separating confidence from answer selection.

Brainstorm additional solutions…

34

4. Answering questionsSlide35

5:

Positive feedback:Participants liked the small question sets and alternating between question and explanation modes

Opportunities:

Participants

wondered about question order and spacing

The combined answer/explanation page was understood, but page layout caused repeated scrolling

Streamlined question sets and explanation pages were liked

35Slide36

Positive feedback: Small question sets

Participants liked the small question sets and alternating between question and explanation

modes

.

The learning

experience

felt manageable

, and not overwhelming. Participants appreciated the format of “small chunks”

of learning

.

Participants appreciated that questions they struggled with were repeated frequently, while at the same time, new questions were introduced.The transition between learning and review/explanations caught some participants by surprise. But, they were not annoyed by it. One said he thought he had “finished the test”, but then realized that there were more questions when amplifire automatically moved into the next question set. This is a success in terms of not letting students stop after the first round

.

5. Question sets and explanations

36Slide37

Opportunity: Question order and spacing

Participants were aware that there was something special about question order and spacing.

Most participants expressed

interest in understanding the underlying factors behind the determined question order

.

Participants were unclear

on

how the answer choices they made affected their progress.

5. Question sets and

explanations

37

Highlights: Comments on question order and spacingSlide38

Positive feedback: Combined answer/explanation page

Participants appreciated the availability of additional learning (but didn’t use it).

Placement of the Review Correct Answers link at the end of the review cycle seemed appropriate to participants, and a few said they might use it.

Participants repeatedly scrolled up and down

between the answer choices and the explanation.

When asked

why,

one participant confirmed that it was so that he could see the correct

answer in

context and contrast it with the other answer choices

.5. Question sets and explanations

38

Highlights: Scrolling on explanation pagesSlide39

Image use in explanation was effective

Participants appreciated the use of images in the explanation, and wanted to zoom or enlarge the image (on both the question and explanation pages

).

39

Highlights: Scrolling on explanation pages

“The image in the explanation was much more helpful than the text itself”.

- Recent college graduateSlide40

Recommendations: Question sets and explanations

In the Help / Tips section of the app, provide a link to an overlay or interactive tutorial describing:

Indicating certainty

Benefits of being honest

What

determines question order

.

Add a section to the dashboard with links to topics about the science behind the learning.

40

5. Question sets and explanationsSlide41

Recommendations (continued)

Consider developing a Screen Tips mode that allows the user to click items on any screen to see an explanation.

41

5. Question sets and explanations

Thanks, Nick!Slide42

Recommendations (continued)

Add zoom functionality to images (in question and explanation views).

42

Zooming to see fabric texture on Athleta.comSlide43

6:

Positive feedback:Once participants understood the progress bar, they liked the affirmation that they were progressing toward completing the module

Opportunities:

The

number of questions and color coding in the progress bar was not understood

The overlay after completing the first question was not noticed or was

misleading

Progress indicators are too subtle

43Slide44

Positive feedback

Once participants understood the progress bar, they liked the feedback as they completed questions progressed through the module.

44Slide45

Opportunity: Number of questions and color coding was not understood

The number of questions and color coding in the progress bar was not understood

Initially,

participants were

unable to identify the meaning for the two shades of green.

One tried

to use

the progress bar to

jump back to a previous question.

Because they didn’t know how many questions were in the module, participants didn’t know what the number of tick marks in the progress bar corresponded to

.6. Progress indicators45Slide46

Opportunity: The Your Progress button wasn’t used

Only one participant found the

My Progress” overlay without prompting. When viewing it, they found it helpful, and wanted estimated time remaining to be added.

Most participants seemed not to recognize

that the button was clickable.

One participant was confused because he had answered one question correctly but the Completed number in the overlay was zero.

6. Progress indicators

46

Highlights: Reactions to the progress barSlide47

Overlay after completing the first question was confusing

Two

participants

didn’t

notice the overlay when

they

completed their first question.

(bug / cache issue?)

The language in the overlay confused participants:

One thought it meant she had completed the module. The word “completed” seemed suggest that she was finished.Another was confused because she had answered several questions correctly, so didn’t know why amplifire considered only one question “completed”.6. Progress indicators

47

Highlights: Dismissing and misunderstanding overlaysSlide48

Recommendations: Progress indicators and overlay

Improve click affordance on the Your Progress button.Include a “What does “Completed mean”

link or info icon in the Your Progress overlay.

Change the wording on the first question completed overlay to

“You answered that question correctly twice, so won’t see it again

.”)

Add estimated time remaining to the Your Progress overlay.

48

6. Progress indicatorsSlide49

7:

Positive feedback:Refresh, review, and flashcards were well

received

Opportunities:

Upcoming changes to review should be re-evaluated

Flashcards should be developed

Review, refresh, and flashcards are on track

49Slide50

Positive feedback: Refresh, review, and flashcards are useful concepts

All participants correctly guessed what refresh

and review links on the dashboard would display.

All participants thought that refresh and review would be useful.

Opportunity: Participants weren’t sure whether they would need to answer questions correctly once or twice in Refresh.

All participants except one said that digital flashcards would be useful.

7. Review, refresh, and flashcards

50Slide51

Upcoming changes to Review should be re-considered

All participants preferred the current presentation of the review

screen:

7. Review, refresh, and flashcards

51

Over the future design:Slide52

Recommendations

Review:

Keep the current design

Refresh:

Clarify for users how many times a question must be answered correctly in Refresh mode.

52

7. Review, refresh, and flashcardsSlide53

Recommendations

Flashcards:

Begin competitive analysis of flashcard apps.

Add flashcards to the upcoming feature list

.

53

7. Review, refresh, and flashcardsSlide54

8:

Positive feedback:Participants were largely successful in navigating the appOpportunities:

The

browser Back button behaved

unexpectedly

Save and Return introduces unnecessary doubt

Going back introduces confusion

54Slide55

Positive feedback

Overall, participants navigated successfully among and within modules.

55Slide56

Opportunity: Browser back button behaves unexpectedly

Participants wanted to page back through questions and explanations, and were naturally drawn to use the browser back button.

In most cases the back button crashes amplifire.

8. General navigation

56Slide57

Opportunity: Save and return introduces unnecessary doubt

It was unclear to participants whether using the Save and Return feature was the only way to save their progress. The presence of the Save and Return link seems to plant uncertainty about whether their progress is automatically saved

.

8. General navigation

57Slide58

Recommendations: General navigation

Change Save and Return to a Dashboard link.Fix the browser Back button.

Question – what would happen when a learner clicks Back when answering questions?

58

8: General navigationSlide59

Priorities

This report provided many recommendations and illustrations. The highest priorities are:

List will go here.

59

Related Contents


Next Show more