In the system concept development process, usability evaluations should serve design. Hence, we are interested in the practical effectiveness of usability evaluations on the system designed. The problem is to structure and exploit usability information so that the designers and developers are able to take the results into account in the design process. The VTT human factors group is
currently developing a new type of approach to usability evaluation and design which is called Usability Case. This is a methodology aimed at tackling the diverse challenges imposed by a longitudinal design-orientated usability evaluation process. In the following, the background to the Usability Case approach is presented (Liinasuo and Norros, 2007).
8.3.1 Background to Usability Case
The development of the Usability Case approach drew on the established tradition of Safety Case. Safety Case is a requirement in many safety standards; explicit Safety Cases are required for military systems, the offshore oil industry, real transport and the nuclear industry. Furthermore, equivalent requirements can be found in other industry standards (Bishop and Bloomfield, 1998). The latter define a Safety Case as ‘a documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment’. The idea of a Safety Case is to gather safety-related information into one document that is also usable later to demonstrate the safety of the system. Information is structured in an orderly manner that takes into account the abstraction level of the information as well as interconnections between the pieces of information.
Liinasuo and Norros (2007) claim that this way of organising information is also fruitful in the context of usability evaluation. Given the specific demands that are set for usability by the design of a complex and unique system, the existence of a clear methodology taking into account the versatile nature of usability evaluations is urgently needed.
The concept of Safety Case is widened in the proposed usability case approach by two perspectives. Firstly, while Bishop and Bloomfield (1998) focus on positive support for claims, it seems reasonable to also indicate whether evidence provides negative support for a certain claim. Hence, problems of usability are reported in the case description. Secondly, Bishop and Bloomfield (1998) aim at producing a documented final description of the good performance of the system to be used during its operation. In addition to this, we see the need for usability information during the design process to guide and give feedback to design.
Following these refinements, Liinasuo and Norros (2007) describe a Usability Case as ‘creating an accumulated documented body of evidence throughout the design that provides a convincing and valid argument of the degree of usability of a system for a given application in a given environment’ (p. 12).
8.3.2 The conceptual structure of a Usability Case
As in Safety Cases (Bishop and Bloomfield, 1998), three concepts are used to structure usability information in the Usability Case. Claims are entities that express the quality of the system in terms of usability. The abstraction level of a claim is high but various viewpoints and levels of detail are possible. Each claim comprises one or more arguments that are instrumental descriptions supporting the claim in question. Finally, each argument is based on evidence, i.e. various data. A case can be demonstrated in several ways. Kelly (1998) lists many different options to articulate evidence in arguments: describing the [safety] level through free text; tabular presentation structuring claims, claim structures that are built up from claims joined together in a hierarchical manner; traceability matrices that represent how one statement (claim, requirement, objective, etc.) is related to a series of other requirements; Bayesian belief networks that are graphical networks communicating the probabilistic causal relationship that exists between variables; and finally, goal
structuring notation that graphically presents the structure of a safety argument by using goals of several hierarchical levels with constraints and solutions (roughly parallel with evidence).
The representation of claims, arguments and evidence seems to be a clear and extensive way to describe the status of usability. Given the complexity of usability evaluations, free text seems to be a flexible means of describing the variety of usability attributes in a given system. Beyond this, tabular or hierarchical structures are also possible means to represent the Usability Case logics.
8.3.3 Characteristics of Usability Case
Usability Case (UC) describes how well the system in question fulfils the usability demands
specific to that system. In understanding these demands, both theory and practice are needed here
through applying the theory of systems usability described above.
The concept of systems usability indicates the appropriateness of a tool for an intended use. Above, we described how systems usability can be defined context-dependently by analysing the core-task demands and task performance the tool should support. We also indicated that the systems usability approach maintains that systems usability is manifested in working practices, which are generic tool-using patterns of the users established in the specific communities of practice. To elaborate tool-using practices, we analysed how the three generic tool functions described above are represented in the users’ practices. In addition, we also considered user experience as an important element of systems usability. In particular, we were interested in how promising the tool appears in future usage. Of course, traditional usability concepts, e.g. by Nielsen (1993), and corresponding breakdown of usability requirements are taken into account in our concept of systems usability. As a result of the above considerations, systems usability claims for the particular system can be produced.
When a Usability Case is produced throughout a long design process, the results of usability
evaluations are accumulated in the Usability Case. This means that in accordance with new phases
of design, the claim structure will be complemented and part of the claims must presumably be restructured to meet the new situation. In this way, the Usability Case is always up to date.
Each claim must be supported by arguments that describe how the claim is demonstrated in practice.
Each argument, in its turn, is justified by showing what data, i.e. evidence, that argument is drawn from. In usability evaluations, those data can be ergonomic standards, results from questionnaires delivered to the employees, heuristic evaluations and so forth.
The basic structure of the Usability Case is demonstrated in Figure 21. The concept and method of the Usability Case are currently being developed in three applications at VTT - nuclear power plant control room modernisation; civil emergency centre information system development; and the present case.
Figure 21: Demonstration of the basic structure of a Usability Case (Liinasuo and Norros, 2007).