“We believe that any official version of a process must continuously be updated to reflect actual development practices, or it
will soon become obsolete, and likely to be ignored and abandoned”
(Gulliksen, et al., 2006, p. 585)
The foregoing chapters answer to the research question formulated in the beginning of this thesis:
‘Is the application of UCD according to ISO 9241-210 (1998) effective, efficient, satisfactory and easy to learn for those responsible for managing hardware and software design and redesign processes?’
albeit with local validity only. Together with the analysis reported in this thesis, the fieldwork undertaken suggests an answer to the research question that is short of positive, in the sense that that the usability of user centered design, as this is described by ISO 9241-210 (2009), leaves room for improvement in the specified context of use.
In greater detail, while the user centered methodology as a process appears to be effective, and provide a the necessary groundwork for the design of systems with improved usability, it is found that the learnability and user satisfaction of applying the method scores low on a quality-in-use scale, when regarded from the perspective of the manager leading software.
An often-recurring observation in this thesis however pivots round the local validity of the findings: The hermeneutical equivalent of generalization – transferability – has not been addressed in the work done, by design, and in acknowledgement of the singular voice speaking throughout the thesis.
Transferability of the Present Results Yet, the ultimate purpose of the exercise is to answer the research question in a more general sense, and on a broader base than
allowed by the research carried out so far. I suggest this can be achieved through a number of different approaches, which include
• Validating the conveyability of the present tales by including the views and observations of the members of the Project Teams involved in the present study;
• Seeking applicability by hearing other voices of other tribesmen – other usability project managers – either directly or indirectly (through literature);
These points partly concern adding further richness, or
thickness, to the present data, which currently are confined to the views, understandings and insights of somebody being ‘responsible for
managing hardware and software design and redesign processes’, and partly to ground them in literature, for the purpose of demonstrating transferability.
Alternative Methods
Other approaches may also well provide an improved understanding of the usability of user centered design methods.
According to Pew and Mavor (2007), three different approaches appear to be directly relevant in the present case, as potential avenues to investigate the usability of user centered design methods:
1. “Evaluation of the user’s performance and satisfaction when using the product or system in a real or
simulated working environment. Also called
evaluation of “quality in use” or “human in the loop.”
2. Evaluation of the characteristics of the interactive system, tasks, users, and the working environment to identify any obstacles to usability.
3. Evaluation of the process used for systems development to assess whether appropriate HSI26 methods and techniques were used…” (2007, p. 265)
In the above list, the first point matches what is described in the section above: evaluating the user’s performance and satisfaction when
26. HSI is an abbreviation of Human-System Integration.
using the ‘product’, which in this case means the process described in ISO 9241-210 (2009). However, I suggest that the research question also may be answered, in full or in part, by applying the alternative methods.
At least, it seems likely that the findings of the latter two approaches may provide additional validity to the results provided by the first method.
Starting in reverse order, the third ‘method’ described by Pew and Mavor (2007) implies an inquiry into the work that originally went into the description of user centered design, and how the people who did it went about the necessary considerations of context-of-use, user involvement, the design of the process, the validation of the design solution and the corresponding acceptance criteria and other pertinent topics. Given that data towards this end can be obtained, this method appears to offer some potential towards answering the research question.
Further along this line of thinking, the second ‘method’ of Pew and Mavor (2007) also holds a certain promise, considering the findings of the field study undertaken and reported presently. As such, I suggest that there would be a significant benefit associated with hearing the voices of other typical roles participating in user centered design, apart from the one already documented in this thesis.
It is as such my current suggestion that a much improved
understanding of engineering practice, engineering practitioners and the maintenance of the engineering knowledge base is required to increase the satisfaction of doing user centered design. This includes gaining further and deeper insights into the mechanisms of achieving buy-in and cross-professional respect, of understanding tradition, resistance against change and the associated organizational and individual mechanisms.
The work of Gulliksen, Boivie and Göransson (2006) on the perspective of the UD – the Usability Designer – is another example of a role in the multidisciplinary team that needs to be appreciated, in order to gain a wider understanding of the team dynamics. It is
conceivable that a further devotion to this could also provide advice on necessary changes of the human factors practice, noting that human factors practitioners depends on others to transform knowledge into practical results: Human factors as a science does not have the executive arm that medicine and dentistry, as examples, does, but typically
depends on engineering practice to reach such results (Gulliksen, et al., 2006).
Central to any team is the team leader, and the multidisciplinary design team doing user centered design is unremarkable in this sense:
the team leader is indeed central. Hoping not to appear narcissistic, what is however remarkable in this context is what appears to be an almost unreasonable demand to the skills, crafts, leadership qualities, social and professional understandings and personal characteristics like patience, determination and stamina in UCD team leaders. In
agreement, Gulliksen, Boivie and Göransson notes that “Moreover, usability professionals need…a great deal of stamina and backbone”
(2006, p. 584). To use a popular metaphor, the usability team leader must be a lamb with five legs, and the point here is that this might be an Achilles heel of user centered design in an industrial context: How many of those are around, and from where do we get a fresh supply?
As Gulliksen, Boivie and Göransson (2006) suggests, there is an educational challenge in this as far as academia and the universities are concerned, but as they do (Gulliksen, et al., 2006), I believe that this is insufficient: such team leaders need the seasoning of practice, and hence, they need to survive in an industrial environment for long enough to earn the required respect and experience. This, I suggest, places an obligation on the industrial community as well, something which however needs significant research and dialogue to mature.
Stepping a step back in a sort of wrapping up of this subject, I suggest that the gathering, analysis and improved understanding of the functions and roles of the multidisciplinary team members will help assessing the width and depth of something which, based on the present data, appears to be an epistemological gap between the team members, hindering the effectiveness of the work to be performed.
Communicating User Centered Design In the context of the UD, the Usability Designer, Gulliksen, Boivie and Göransson (2006) keeps focusing on the communicative skills required.
“Communication skills are essential for the UD, given that one of the main tasks it to act as a communication channel between the users and the system developers” (Gulliksen, et al., 2006, p.
582)
and continue that
“Communication is a key factor for effective multidisciplinary teamwork and the ability to establish a common understanding
of the design problems and common ground” (Gulliksen, et al., 2006, p. 594)
Finding myself in complete concurrence, I would however like to augment this point, with a mindset towards future work: The findings of this thesis quite clearly identify that team members of a solution oriented practice are rather disinterested in communication of a theoretical nature. Yet, I suggest, transferring such know-how to all team members, and perhaps even to all stakeholders is important, or indeed essential. Finding ways and means to do so effectively, without alienating this audience, is a topic that warrants careful consideration and research.
An Update of User Centered Design Methodology Achieving transferability (in both of the reinterpreted senses) of the present findings is however not the terminal, but only a station on the line, a prerequisite for the utility of the knowledge generated: The eventual purpose of the research undertaken is to make a contribution to the increased application of user centered design, in the maritime industry as well as in a larger context, to the benefit of effectiveness, efficiency and user satisfaction.
From the scope statement of ISO 9241-210 (2009), the level of ambition of the standard is clear.
“The information in this part of ISO 9241 is intended for use by those responsible for planning and managing projects that design and develop interactive systems. It therefore addresses technical human factors and ergonomics issues only to the extent necessary to allow such individuals to understand their relevance and importance in the design process as a whole. It also provides a framework for human factors and usability professionals involved in human-centred design” (ISO9241-210, 2009, p. 1)
Let it be clear that I find that the standard achieves what it sets out to do. Sometimes, however, it is not enough to look at what is there – it is just as important to look at what is not. In the context of the research question, this translates into what I have chosen to label
‘adequacy’: Are the available descriptions of user centered design, as
expressed for instance in ISO 9241-210 (2009), adequate in terms of preparing you, the coming usability manager, for the task at hand?
Reading this, you might with some right wonder why I venture into this dimension, but I assure you it is not done on a whim: I suggest that in order to be successful, eventually, in user centered design, or indeed in any other undertaking, being prepared is invaluable – but you obviously need to prepare for the right things.
This preparedness, I further suggest, might in the present case best be achieved by using the knowledge gained from present and future research to suggest augments to the available descriptions of user centered design, of which ISO 9241-210 (2009) might be taken as an example, or to the corresponding literature: It could appear that the potential users of user centered design methods could benefit from a wider, more holistic, description of the art and craft of user centered design, the skills needed, and the pitfalls to be observed.
Future work on Methodology, Validity and Reliability As discussed above, establishing the transferability of the present results is an important next step in the work towards answering the research question, and ultimately towards better understanding of the potential barriers to a more widespread usage of user centered design in the maritime industry.
While the concept of transferability appears to be rather widely argued (Fishman, 1999; Lincoln & Guba, 1985; Polkinghorne, 2003a), the perspective seems to be a bit different in Action Research. Here, the demands and conditions needed to generalize findings appear to be less developed than in ethnography. However, Fals Borda (2001) mentions generalization in the context of participatory action research, and Greenwood and Levin subscribe to a Webrarian notion of
“Transcontextual Credibility”, which appears to match the concept of transference:
“Meanings created in one context are examined for their credibility in another situation through a conscious reflection on similarities and differences between contextual features and historical factors. They are moved from the context where the understanding was created through a collaborative analysis of the situation where this knowledge might be applied. Based on the historical and contextual analysis, AR judgements are
made about the possibility of applying knowledge from one situation in another” (1998, pp. 84-85).
From a theoretical viewpoint, however, I propose that the above argument on transferability might only be the tip of the iceberg, and that further, deeper analysis is required to examine transferability at the interface of autoethnography and self-study reflective practice. In particular, the issues of researcher positionality and of the transition from conveyance to applicability might have side effects that should be examined.
[Blank page]