In the following, we define the particular kind of usability we refer to throughout this thesis. Subsequently, we relate that kind of usability to the notions of “user experience” and
“relevance”.
3.3.1 A Definition of Usability
ISO 9241-11 describes usability as the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO, 1998). Already from this it becomes clear that usability is a difficult-to-grasp concept that can be understood and interpreted in many different ways. In particular, the usage of words such as “specified” (that appears three times in this rather short definition) and “context” shows that usability depends on a variety of external factors. It can be thoroughly evaluated only with a well-defined use case and situation in mind. Furthermore, there is no complete consensus about which sub-concepts contribute to usability. To give just one example, the above ISO definition assumes user satisfaction to be a part of usability.
Contrary, Lew et al. (2010) treat it as an independent concept in the context of their 2Q2U framework, which has been derived from the ISO 25010 standard (ISO, 2011).
1http://www.w3.org/Protocols/ (Jun. 11, 2014).
26 Chapter 3 Proposed Solution: Search Interaction Optimization
Thus, it is necessary that in the following, we clarify our understanding of usability in the context of our proposed approach. Orienting at ISO 25010 (ISO, 2011), the internal usability of a web application is measured in terms of static attributes (not connected to software execution); external usability relates to the behavior of the web application; and usability in use is relevant in case the web application involves real users under certain conditions.
Therefore, given the fact that we intend to infer usability from real users’ interactions, usability in use is the core concept we focus on. In accordance with this, Hassenzahl (2008) uses the notions of “do-goals” (e.g., booking a flight) and “be-goals” (e.g., being special) to distinguish between the pragmatic and hedonic dimensions of user experience, a concept that has a large intersection with usability (Figure 3.1). Particularly, he states that “Pragmatic quality refers to the product’s perceived ability to support the achievement of ‘do-goals’ [and]
calls for a focus on the product – its utility and usability” (Hassenzahl, 2008). Since a user’s interactions with an interface are a direct reflection of what they do, for our purpose the pragmatic dimension of usability is of particular interest.
Based on the above, in the remainder of this thesis USABILITY refers to the pragmatic (Hassenzahl, 2008) and in-use (ISO, 2011) dimensions of the definition given by ISO 9241-11 (ISO, 1998). Internal/external usability (ISO, 209241-11) and the hedonic dimension (“the
product’s perceived ability to support the achievement of ‘be-goals”’, Hassenzahl, 2008)) of usability in use are neglected.2
Definition 2 USABILITY: The extent to which a web interface can be used by real users to achieve do-goals with effectiveness, efficiency and satisfaction in a specified context of use.
(adjusted definition by ISO, 1998)
A more precise formalism for characterizing the specific type of usability one is investigating is given in Speicher (2015).
3.3.2 Usability vs. User Experience
It must be noted that user experience (UX) and usability are not interchangeable con-cepts (Law et al., 2009), although “a closer look reveals that [UX] is treated mainly as a synonym of usability and user-centred-design [sic]” (Hassenzahl, 2008). Rather, UX includes more subjective aspects while usability is more technically oriented. Furthermore, it is a misconception that usability is a true subset of UX, as is, e.g., suggested by Lew et al. (2010).
Alternatively, usability and UX should be seen as two separate concepts that share a common intersection (Figure 3.1).
First, as Hassenzahl (2008) states, “people perceive interactive products along two different dimensions”, i.e., pragmatic and hedonic quality. Already in our definition of usability above, we have explained that the pragmatic dimension is determined by do-goals while the hedonic dimension is determined by be-goals. Further, Hassenzahl (2008) argues that “the fulfilment of be-goals [...] is the driver of experience. Lack of usability might impose a barrier to the
2The following is not the final version of the definition. Based on our findings, it will be revised in Chapter 4.
3.3 Usability 27
Usability User Experience
Fig. 3.1.: Usability and user experience are not the same or have a superset–subset relation-ship. Rather, they are different concepts that share an intersection.
fulfilment of active be-goals, but it is in itself not desired.” Based on this, usability cannot be a true subset of UX, as certain do-goals, which are elements of usability, are not required to achieve a good UX. Therefore, they are to be located in the set Usability \ User Experience.
Second, to give a real-world example for this, we refer to Norman (2005). In his article, the author describes the case of musical instruments from a human-centered point of view.
Particularly, he states:
“Musical instruments are complex and difficult to manipulate and can cause severe medical problems. Musical notation is modal, so the same representation on a treble clef has a different interpretation on the bass clef. The usability profession has long known of the problems with modes, yet multiple staves have been with us for approximately 1000 years. It takes considerable instruction and practice to become skilled at reading and playing. The medical problems faced by musicians are so severe that there are books, physicians, web pages and discussion groups devoted to them. For example, repetitive stress injuries among violinists and pianists are common. Neither the instruments nor the notation would pass any Human-Centered Design review.”
Yet, musical instruments have a wonderful UX once you know how to play them, although their usability is questionable. This underpins the fact that usability—at least from a global perspective—is not a subset of user experience. In the remainder of this thesis, we take a rather technical (or objective) point of view, i.e., we aim at inferring usability from user interactions alone, among other things. Thus, we want to point out that the measurement of UX—as opposed to usability—is not a goal of this thesis.
3.3.3 Usability vs. Relevance
As has already been mentioned in the introduction, in Chapter 4 we will learn that the kind of usability investigated in this thesis comprises seven factors. Two of them, informativeness and understandability are located in the dimension effectiveness (cf. Figure 1.3). In the context of SERPs, these two factors are mainly determined by the relevance of the displayed
28 Chapter 3 Proposed Solution: Search Interaction Optimization
Usability
Relevance
Fig. 3.2.: Since the usability factors informativeness and understandability are mainly de-termined by the relevance of the results displayed on a SERP, relevance is a true subset of usability.
results. In particular, if a user is presented with results that theoretically contain the desired information, but in a language the user does not speak, they will not perceive the results to be relevant. Hence, the given SERP in theory has a good informativeness but bad understandability (although the user would most probably rate both factors to be bad).
Contrary, if the user is presented with irrelevant results in a language they understand, the SERP’s understandability is good while its informativeness is bad. Therefore, in this thesis, relevance is considered to be a true subset of usability: relevance ⊂ usability (Figure 3.2).
As an additional formalism, the fraction of usability that does not include relevance is noted in the following form: Usability \ Relevance.