Study Discussion

From the usability perspective, OSS is expected to encounter an increasingly challenging environment, with a diversity of users, both in terms of technical experience and cultural background, who possess unique needs, expectations and demands. Benson et al. (2004) emphasize the need for HCI professionals to enhance the acceptance and usability of OSS. According to Çetin and Göktürk (2008), OSS projects can only become usable through measurement and analysis.

Hedberg et al. (2007) state that it is essential to understand the user and the context of use as well as the active involvement of target users through their feedback at an earlier stage of the design process. Crowston et al. (2003) maintain that OSS relies heavily on its users’ feedback for improving its quality. Accordingly, 82% of the respondents in our survey agree that user feedback is useful in every phase of the software lifecycle,

including requirements, design, development, pre-release and post-release. Our empirical investigation also supports the hypothesis that user feedback has a positive impact on usability in OSS.

Golden et al. (2005) maintain that usability issues are not addressed at the software architecture design level, and, as a result, they identify the consequences of this omission, such as redesigning or re-architecting the whole system. Nakagawa et al. (2008) highlight the paucity of literature exploring how software architecture can influence OSS quality. Through a case study, they have shown that software architecture is positively associated with OSS quality. However, our parametric statistical analysis, PLS regression and multiple regression analyses do not support the positive relationship between the incorporation of usability issues at the architectural level and OSS usability.

Çetin and Gokturk (2007) identify the lack of user-centered design and usability

problems in an OSS development environment. Specifically, they believe that there is a need for increased collaboration between designers and developers as well as an early contribution of usability experts in order to ensure the overall quality of a project. However, Iivari and Iivari (2006) do not consider user-centered design as a “separate system development approach,” as they believe it does not entail aspects of system development either in the requirements phase or in the technical implementation phase. In our survey, 79% of the respondents support the opinion that standardized design techniques can act as a set of criteria against which software may be inspected. Similarly, in all the phases of our empirical analysis, we have found a significant relationship between user-centered design techniques and OSS usability improvement.

Crowston et al. (2003) maintain that the measurement of success and quality in an OSS project is necessary, because millions of users are dependent on OSS systems. Çetin and

Göktürk (2008) do not see usability as the main stimulus behind OSS development; rather, they believe that such development, referred to as “the freedom of the movement,” does not necessarily imply usability within software. Consequently, these authors propose a measurement framework for assessing OSS projects, which is required for self-

evaluation. In our survey, 69% of the respondents believe that usability assessment is of prime importance, especially considering the distributed environment and the cultural diversity among users. Moreover, our empirical study determined a significant

relationship between OSS usability and usability assessment.

Nichols et al. (2001) believe that the less technically-oriented users within the OSS community are challenged in reporting errors and debugging the software. These authors cite examples where users had significant problems with software behavior and

documentation even though the developers were pleased with both aspects. Similarly, 90% of the participants in our survey agreed that the proper documentation of OSS projects increases the understandability and learnability of software. Additionally, all phases of our empirical investigation support the positive impact of proper

documentation on OSS usability.

5.5.1

Limitations of the Study & Threats to External Validity

Empirical studies are always subject to certain limitations, and although we performed a number of measures to reduce the threats to external validity and increase the reliability, there are still some limitations to this study. First, while there may be other contributing factors that influence OSS usability besides the five independent variables, these factors relate most clearly to the scope of this study, which was to examine the contributing factors from the viewpoint of OSS contributors. Other contributing factors, such as usability bug reporting, interactive help features, usability learning and usability

guidelines, usability experts’ opinion, incremental design approach, usability testing and knowledge of user-centered design methods are considered in three of our other studies, which are being conducted from the perspective of OSS developers, users and the industry. Another notable limitation of this study involves the small sample size. The smaller sample size, which occurs with the number of projects and respondents, is a

potential threat to the external validity of this study. However, the number of participants is limited because few contributors consider usability and its related issues as a top priority.

In this study, we have empirically investigated the effect of key factors on OSS usability and answer the research question stated in this investigation. The empirical results of this study strongly support the hypotheses that users’ feedback, design techniques, usability assessment and documentation are positively associated with the usability of an OSS project. However, we could not find any statistical significance for the effect of usability at the architectural level on OSS usability improvement.

Chapter 6