Qaiser S Durrani FASTNU Lahore Workshop on Usability Engineering Feb 2123 2011 at SEECS NUST Agenda Usability Engineering Why we need it What are its measures Where UE fits in the SDLC ID: 436447
Download Presentation The PPT/PDF document "Usability Engineering and its role in So..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Usability Engineering and its role in Software Industry
Qaiser S. Durrani
FAST-NU, Lahore
Workshop on Usability Engineering
Feb 21-23,
2011 at SEECS NUSTSlide2
Agenda
Usability Engineering?
Why we need it?
What are its measures?
Where UE fits in the SDLC
Can we integrate or map UELC with SDLC?
Experience and Emotional Measures – Role?
Case Study
Current practices in Software Industry with respect to UESlide3
W
hy Usability Engineering?
Functional perspective
User perspectiveSlide4
Usability
‘‘the capability to be used by humans easily and effectively’’
‘‘quality in use’’
‘‘the effectiveness, efficiency, and satisfaction with which specified users can achieve goals in particular environments’’
Context dependent
(shaped by the interaction between tools, problems, peoples)
A process through which usability characteristics are specified and measured throughout the software development lifecycle.Slide5
Key Research Questions in HCI
How to work with and improve the usability of interactive systems?
Guidelines for improving the usability of systems?
Methods for predicting usability problems?
Techniques to test the usability of systems?
Discussions on
how to measure usabilitySlide6
Neglecting Usability EngineeringSlide7
Usability into Software Development
When integrating usability into the system design process,
early focus on users and tasks
,
empirical measurement
, and
iterative design principles
are suggested
This integration, however, is not a trivial task, as numerous obstacles have been reported
First of all, introducing a new method into a software development organization is typically a delicate problem
User-centered design
techniques have been reported to remain the speciality of visionaries, isolated usability departments, enlightened software practitioners, and large organizations, rather than the everyday practice of software developersSlide8
Usability Engineering and Experience DesignSlide9
Models for Usability Engineering Lifecycle
Star Lifecycle Model
ISO 13407 Model
Usability engineering lifecycle
by Deborah J. Mayhew Slide10
Usability Engineering LifecycleSlide11
Requirements Analysis Phase
User Profiling – Cognitive & Non-Cognitive measures
Task Analysis
SW/HW/Environment Constraints
General Design
Principals
Usability GoalsSlide12
Design, Development & Evaluation
Conceptual Level Design
Detail Level Design
Screen Standards
Iterative EvaluationSlide13
Usability ActivitiesSlide14
Adaptation of Usability Activities into Software Engineering Development ProcessSlide15
Allocation of Usability Techniques to
Development ActivitiesSlide16
Shneiderman’s Golden Rules
R1:Strive for consistency
R2:Offer shortcut
R3:Give effective feedback
R4:Reduce Short term memory load
R5:Provide reversal of actions
R6:Design Dialogues to yield closure
R7:Provide locus of controlSlide17
Practices - MEASURING USABILITY
(Case study of 180 projects)
Measures of effectiveness
Measures of Efficiency
Measures of SatisfactionSlide18
Measures of Effectiveness
Binary task completion
Accuracy
Recall
Completeness
Quality of outcome
Experts assessmentSlide19Slide20
Comments
1- 22% of the studies reviewed do not report any
measure of effectiveness
nor do these studies control effectiveness.
Frøkjær
et al. argued that the HCI community might not succeed in trying to make better computing systems without employing measures of effectiveness in all studies
2- Research shows that measures of the
quality of the outcome
of the interaction are used in only 16% of the studies. For example,
experts’ assessment
of work products seems a solid method for judging the outcome of interaction with computers and has been used in a variety of fields as an
indicator of the quality of work products
, for example with respect to creativity. Yet, in this sample only 4% of the studies use such measuresSlide21
Comments
3-
New kinds of
devices
and use
contexts
require new measures of usability. Especially, it has been argued that the notion of task underlying any effectiveness measure will not work in emerging focuses for HCI, such as home technology
4- A number of studies
combine usability measures
into a single measure, report the combined values, and make statistical tests on the combinationsSlide22
Measures of efficiency
Time
Input rate
Mental effort
Usage patterns
Communication effort
LearningSlide23Slide24
Comments
1- Some of the efficiency measures are obviously related to the quality of interactive computer systems, because they quantify resources (e.g., time or mental effort) that are relevant in many contexts for many users
2- A second comment on the studies reviewed pertains to the measurement of time.
A surprising pattern apparent from Table is that while objective task completion time is measured by 57% of the studies,
little attention is paid to user’s experience of time
However, in this sample of 180 studies, only one study measures directly subjective experience of timeSlide25
Comments
3- The reviewed studies differ in
how task completion times, and efficiency measures in general
, are reasoned about. In the ISO definition of usability and in most of the studies reviewed, time is considered a resource of which successful interfaces minimize consumption
However, in a handful of studies higher task completion times are considered as indicators of motivation, reflection, and engagementSlide26
Comments
4- A striking pattern among the studies reviewed is that
few studies (5) concern learning of the interface.
Only five studies measure changes in efficiency over time
5- In the studies reviewed, the
median time of working with the user interfaces evaluated was 30 minSlide27
Measures of Satisfaction
Standard questionnaires
Preferences
Satisfaction with the interface
User attitudes and perceptionsSlide28Slide29Slide30
Comments
1- The measurement of satisfaction seems in a state of disarray. A host of adjectives and adverbs are used, few studies build upon previous work, and many studies report no or insufficient work on the validity and reliability of the instruments used for obtaining satisfaction measures
Another indication of the disarray is in the limited use of standardized questionnairesSlide31
Comments
2- A second comment on the satisfaction measures used is that studies
vary greatly in the phenomena
that are chosen for objective performance measures and those that are investigated by asking subjects about their perceptions and attitudes.
One question arises when users’ perception of phenomena is measured when those phenomena perhaps more fittingly could have been assessed by
objective measures
3- The review shows that in practice subjective satisfaction is taken to mean a questionnaire completed after users used the interface. Only eight studies (4%) measure satisfaction during use without using questionnairesSlide32
CHALLENGES IN MEASURING USABILITYSlide33
Subjective and objective measures of usability
Measures of usability concern
user’s perception
of or
attitudes
towards the interface, called
subjective usability measures
Other measures concern aspects of the interaction not dependent on user’s perception called
objective usability measures
Such a distinction has been argued to
simplify the nature of measuremen
t in science
Suggest using the distinction to reason about
how to choose usability measures and find more complete ways of assessing usability
Measures may lead to different conclusions regarding the usability of an interfaceSlide34
Measures of learnability and retention
Particularly
measures of efficiency
, we find it relevant to compare them to recommendations on how to measure usability
The well-known textbook by Ben Shneiderman (1998, p.15) recommends measuring (1) time to
learn, (2) speed of performance, (3) rate of errors by users, (4) retention over time, and (5)
subjective satisfaction.
Nielsen (1993, p. 26) similarly recommends measuring (a) learnability, (b) efficiency, (c) memorability, (d) errors, and (e) satisfaction
Most of the reviewed studies follow part of the recommendations by measuring
task completion time
(points 2 and b above),
accuracy
(points 3 and d),
and satisfaction with the interface
(points 5 and e):
92%
of the studies measure at least one of these;
13%
of the studies measure all threeSlide35
Measures of learnability and retention
The majority of studies make
no attempt to measure learnability or retention
This challenge is most relevant for studies or research addressing systems that
users should be able to learn quickly
or that will be intensively used
Overall, usability studies could put more
emphasis on measures of learning
, for example by measuring the time needed to reach a certain level of proficiency
In addition,
measures of the retention of objects and actions available in the interface
(i.e., the ability of users to come back and successfully use the interface) are important in gaining a more complete picture of usabilitySlide36
Measures of usability over time
The studies reviewed show that users typically interact only briefly with interfaces under investigation; as mentioned earlier the median duration of users’ interaction was 30 min; only 13 studies examined interaction that lasts longer than five hours
The brief period of interaction in the studies reviewed explains the
lack of focus on measures of learning and retention
The observation also suggests that
we know little about how usability develops
as the user spend more time interacting with the interface and how tradeoffs and relations between usability aspects change over time
From research, we need a more full understanding of how the relation between usability aspects develops over timeSlide37
Extending, validating and standardizing measures of
satisfaction
The disarray of measures of satisfaction presents special challenges
One is to extend the existing practice of measuring satisfaction almost exclusively by post-use questions;
another is to validate and standardize the questions used
Validation may be achieved through studies of correlation between measuresSlide38
Micro and macro measures of usability
Usability at a
micro level
Such measures cover tasks that are usually of short duration (seconds to minutes), has a manageable complexity (most people will get them right), often focus on perceptual or motor aspects (visual scanning, mouse input), and time is usually a critical resource
Usability at a
macro level
Such measures cover tasks that are longer (hours, days, months), are cognitively or socially complex (require problem-solving, learning, critical thinking, or collaboration)Slide39
A working model for usability measures and research
challengesSlide40
Affective Requirement
The need to make something fun, engaging, or enjoyable is usually not considered in requirements elicitation
Software requirements for these and other affective factors are never truly captured in an official manner
Juran is credited with coining the phrase "
fitness for purpose
“
If a system is intended to be a
leisure product
then the ‘fitness for purpose’ must also extend to
affectSlide41
Rebirth of Affect in Design
The idea of affect is not old but affect has re-emerged as a potentially desirable design characteristic
One of the visionaries of this re-emergence was Robert Glass from Sun Microsystems, who said:
“
If you’re still talking about ease of use then you’re behind. It is all about the joy of use. Ease of use has become a given – it’s assumed that your product will work
.” (Glass, 1997)Slide42
Summary of research into affective factorsSlide43
Exploring Affect……Theories
Three theories have each been said to contribute to computer game enjoyment
Usability:
In ISO 9241-11 (ISO, 1998), usability is characterized as consisting of three elements:
effectiveness, efficiency, and satisfaction
Grice (2000) attempted to apply these three elements to computer game design
His hypothesis was that computer games that were
enjoyable
will have high levels of
efficiency, effectiveness
, and
satisfaction
Some minor experiments conducted under his supervision seemed to indicate that this hypothesis was trueSlide44
Exploring Affect…Theories
Flow:
Csikszentmihaly describes flow as ‘the holistic sensation that people feel when they act with total involvement
In the state of flow, actions flow without conscious intervention by the actor
The term
flow was used because people in this state often
said that they “were in the flow of [the activity]”.
the
characteristics
of flow-inducing activities are:
must feel capable of completing the task
must have the ability to concentrate on task
clearly recognizes the goals of the task
receives immediate feedback about task performance
has a sense of control over their actions
has the sense of time altered: hours can seem like minutesSlide45
Exploring Affect…Theories
Heuristics for internally motivating interfaces:
Malone (1983), in agreement with Csikszentmihaly, believes that fun and enjoyment only arise from activities that are intrinsically motivated
Computer games are thought to be played because
of intrinsic motivation, with no expectation of a reward other than the activity itself
Malone and
Lepper
(1987) developed seven heuristics for the design of intrinsically motivated interfacesSlide46
Exploring Affect…Theories
The 4 major heuristics are:
Challenge-
multi-layers of challenge so that the user will feel initial success and continue to see improvements
Curiosity-
believe that their knowledge structures (or skills) are incomplete or inconsistent
Control-
interface should make the user feel that the outcomes are determined by the users own actions
Fantasy-
evoke mental images of physical or social situations
Other minor are
Competition
,
Cooperation
,
RecognitionSlide47
Results
The results being referred to are the learnability and ‘losing time’ reasons
Loss of Time
LearnabilitySlide48
Measures of specific attitudes towards the interface (Experience Design) – from 180 projectsSlide49Slide50
Current Usability Practices in Pakistan Software IndustrySlide51
Basic Software Industry Data
Number of SW industry surveyed: 26
Number of respondents: 35
Project
Type: Multiple type from Web to ISSlide52
Research Questions
Does organization include estimates for usability activities in planning phase?
Does organization involve users during SDLC phases? If yes then what kind of user involvement it has?
(a) Are usability activities integrated into requirement phase of SDLC? (b). Are usability activities integrated into design phase of SDLC?
(c)
Are usability activities integrated into implementation phase of SDLC? (d) Is usability testing done in an organization?
Does an organization collect feedback from users for a product?
Does an organization calculate return on investment for the usability activities?
Are organizations intended to introduce or enhance the UELC activities in SDLC? Slide53
Fig1:Usability Activities in Planning Phase
Fig2:User Involvement in SDLC
Fig3:User Involvement
in SDLC PhasesSlide54
Fig4: Usability Requirements
Fig5: User Profile
Fig6: User Contextual InquirySlide55
Fig9: Usability Roles
Fig7: Usability Goals
Fig8: User Interface Development platformSlide56
Fig10: Screen Design Standards
Fig11: Design Flexibility
Fig12: Usability RolesSlide57
Fig13: Detailed Design of User interface
Fig14: Usability TestingSlide58
Fig15: User Feedback
Fig16: User ExperiencesSlide59
Fig17: User Feedback
Fig18: ROI CalculationSlide60
Challenges in Measuring Usability
Subjective and objective measures of usability
Measures of learnability and retention
Measures of usability over time
Extending, validating and standardizing measures of satisfactionSlide61
Recommendations
Developers must consider user interaction from the beginning of the development process.
Practice of Usability Testing
Practice of Cost-Justifying Usability tasks
Don’t try to do a full-scale usability process from the beginning.Slide62
References
Kasper
Hornbæk
,
Int. J. Human-Computer Studies (2006),
Current practice in measuring usability: Challenges to usability studies and research
Xavier
Ferre
, Integration of Usability Techniques into the Software Development Process
Juho Heiskari, Marjo Kauppinen, Mikael Runonen, Tomi Mannisto,
Bridging the Gap Between Usability and Requirements Engineering,
2009 17th IEEE International Requirements Engineering Conference
Todd Bentley, Lorraine Johnston,
Karola
von
Baggo
,
AWRE’2002,
Putting Some Emotion into Requirements Engineering
Samia
Asloob
,
Qaiser
S.
Durrani
, Usability Engineering Practices in SDLC, Technical Report (2010), FAST-NU, Lahore Slide63
Questions?Slide64
Bottom Line benefits
Increased Productivity
Decreased user training
Decreased user errors
Decreased need of on-going technical support
Incorporating business and marketing goals while catering to the user needs (especially for
M
obile, Web and Gaming applications)Slide65
Time Constraints for the Application of Usability Activities and TechniquesSlide66
Subjective and objective measures of usability
Challenges in research are to develop
subjective measures for aspects of quality-in-use
that are currently measured by objective measures, and vice versa, and evaluate their relation
In studies of usability, we suggest paying special attention to
whether subjective or objective measures are appropriate
, and whether a mix of those two better covers the various aspects of quality-in useSlide67
Definition of Process Increments
defined seven deltas in order to get a better match with the general stages of an iterative software development process
D
1: Early Analysis
D
2: Usability Specifications
D
3: Early Usability Evaluation
D
4: Regular Analysis
D
5: Interaction Design
D
6: Regular Usability Evaluation
D
7: Usability Evaluation of Installed SystemsSlide68
Affective Requirement
Same functional requirements, underwent a similar design process by the same designers, yet the
need to convey a different affective response greatly changed the entire product
Given that requirements give the constraints on how a system should behave, then it is important to see that
‘affective requirements’
are considered a valid category of requirement
Accepting that affective factors make valid requirements raises
the following questions:
How does an organization elicit and document affective requirements?
How does an organization design to meet affective requirements?
How does an organization validate that the design elicits the required affective response?Slide69
Motivations
Research focus on how to measure usability has three motivations:
First, what we mean by the term usability is to a large extent determined by how we measure it
Second, usability cannot be directly measured so, find aspects of usability that can be measured
Which measures of usability to select is consequently central in many approaches to the design and development of user interfacesSlide70
Studies of correlations between measures
A weak understanding of the relation between usability measures gives rise to many of the issues
With a better understanding, we could make more informed choices about which usability measures to employ
Studies of correlation between measures may improve this understanding by informing us whether our measures contribute something new and what their relation are to other aspects of usability
There is need for a better understanding of the
relation between usability measures
, for which studies of correlations between measures would be one contribution