Study Results

For the first phase of this study, parametric statistics were used by examining the Pearson correlation coefficient between the individual independent variables, the key usability factors, and the dependent variable, OSS usability, in order to test hypotheses H1 to H5. The results of the statistical calculations for the Pearson correlation coefficient are reported in Table 5.9. The coefficient between user feedback and OSS usability is positive, with a value of 0.48 at P < 0.05, and hence, hypothesis H1 is justified. The Pearson correlation coefficient of 0.21 is observed at P = 0.06 for the relationship between usability at the architectural level and OSS usability, and hence, it is found insignificant at P < 0.05. Therefore, hypothesis H2, which involves usability at the architectural level and OSS usability, is rejected. Hypothesis H3 is accepted based on a Pearson correlation coefficient of 0.48 at P < 0.05, which involves the relationship between usability design techniques and OSS usability. The positive correlation coefficient of 0.36 at P < 0.05 occurs between OSS usability and usability assessment, which indicates that H4 is accepted. Finally, hypothesis H5 is found significant and accepted due to its Pearson correlation coefficient of 0.51 at P < 0.05, which occurs

between documentation and OSS usability. Hence, the hypotheses H1, H3, H4, and H5 are found statistically significant and are accepted whereas H2 is not supported and is therefore rejected.

In Phase II, non-parametric statistical techniques are used by examining the Spearman correlation coefficient between the individual independent variables, the key usability factors, and the dependent variable, OSS usability, as shown in Table 5.9. The Spearman correlation coefficient between users’ feedback and OSS usability is positive, with a value of 0.43 at P < 0.05, and hence, hypothesis H1 is justified. For hypothesis H2, the Spearman correlation coefficient of 0.29 occurs at P = 0.01; hence, at P < 0.05, a significant relationship is found between usability at the architectural level and OSS usability. Furthermore, hypothesis H3 is accepted based on a Spearman correlation coefficient of 0.48 at P < 0.05, which occurs between design techniques and OSS usability. The positive Spearman correlation coefficient of 0.32 at P < 0.05 is also observed between OSS usability and usability assessment, which indicates that H4 is accepted. Hypothesis H5 is also found significant and thus accepted after obtaining a Spearman correlation coefficient of 0.57 at P < 0.05, which exists between

documentation and OSS usability improvement. Hence, all of the hypotheses, H1, H2, H3, H4 and H5, are found statistically significant and are accepted in the non-parametric analysis.

Table 5.9: Hypotheses testing using Pearson correlation and Spearman correlation coefficients (Contributors’ Viewpoint)

Hypothesis Usability Factor Pearson

Correlation Coefficient Spearman Correlation Coefficient H1 Users’ Feedback 0.48* 0.43* H2 Usability at the Architectural Level 0.21** 0.29* H3 Design Techniques 0.48* 0.48* H4 Usability Assessment 0.36* 0.32* H5 Documentation 0.51* 0.57* * Significant at P < 0.05. ** Insignificant at P > 0.05.

In Phase III, the PLS technique is used to overcome some of the study’s limitations and to perform cross-validation with the results observed using the approaches in Phases I and II. Specifically, the direction and significance of hypotheses H1 to H5 are examined. In the PLS method, the dependent variable of our study model, OSS usability, is

considered as the response variable, and the independent variable, the key usability factors, are considered as predicators. The test results, which contain the observed values of the path coefficient, R2 and the F-ratio, are demonstrated in Table 5.10. The users’ feedback is observed to be significant at P < 0.05, with a path coefficient of 0.76, an R2 value of 0.23 and an F-ratio of 22.68. Usability at the architectural level has path coefficient of 0.38, an R2 value of 0.04 and an F-ratio of 3.59, which is found

insignificant at P < 0.05, with an observed P value of 0.06. Usability design techniques have the same direction as proposed in the hypothesis H3, with a path coefficient of 1.03, an R2 value of 0.23 and an F-ratio of 22.86 at P < 0.05. Similarly, the variable of usability assessment conforms to the hypothesis H4, with a path coefficient of 0.52, an R2 value of 0.13 and an F-ratio of 11.39 at P < 0.05. Finally, the variable of documentation has a path

coefficient of 1.08, an R2 value of 0.26 and an F-ratio of 26.44 at P < 0.05, and, as a result, it also accords to H5. Hence in this phase, as is the case in Phase I, the hypothesis H2, which deals with usability at the architectural level and OSS usability improvement, is not statistically significant at P < 0.05.

Table 5.10: Hypotheses testing using PLS regression (Contributors’ Viewpoint)

Hypothesis Usability Factor Path Coefficient R2 F- Ratio

H1 Users’ Feedback 0.76 0.23 22.68* H2 Usability at the Architectural Level 0.38 0.04 3.59** H3 Design Techniques 1.03 0.23 22.86* H4 Usability Assessment 0.52 0.13 11.39* H5 Documentation 1.08 0.26 26.44* * Significant at P < 0.05. ** Insignificant at P > 0.05

Our objective in study model testing is to provide empirical evidence that our key factors play a considerable role in improving open source software usability. In particular, the testing process consists of conducting regression analysis and reporting the values of the model coefficients and their direction of association. The multiple linear regression equation for our study model is depicted in Equation 5.1, where OSS usability is the response variable and the key factors are predicators. The regression analysis results of the study model are displayed in Table 5.11. Specifically, the path coefficient for four out of five variables, users’ feedback, design techniques, usability assessment and

documentation, are positive and their t-statistics are statistically significant at P < 0.05. However, the path coefficient for usability at the architectural level is negative, its

negative t-statistics and its P value of 0.09 at P > 0.05 render this independent variable statistically insignificant.

Table 5.11: Multiple Linear Regression Analysis of the Study Model (Contributors’ Viewpoint)

Model coefficient Model coefficient Coefficient value t-value

Users’ Feedback β1 0.32 3.13* Usability at the Architectural Level β2 -0.16 -1.69** Design Techniques β3 0.17 1.51* Usability Assessment β4 0.19 2.16* Documentation β5 0.47 5.02* Constant β0 0.59 0.10* * Significant at P < 0.05. ** Insignificant at P > 0.05

The R2 and the adjusted R2 vales of the overall study model are 0.50 and 0.47 with an F- ratio of 14.24, which is significant at P < 0.05.

Recapping Equation 5.1 by inserting model coefficient values, we get:

5 4 3 2 1 0.16 0.17 0.19 0.47 32 . 0 59 . 0 Usability OSS = + v − v + v + v + v

where v1, v2, v3, v4 and v5are the five independent variables.