Hypotheses Testing and Results

To test hypotheses H1-H5 of the research model, as shown in Figure 3.1, parametric statistics are used to examine the Pearson correlation coefficient between individual independent variables, the usability factors, and the dependent variable, OSS usability. The results of the statistical calculations for the Pearson correlation coefficient are displayed in Table 3.9. “In statistical hypothesis testing, the p-value is the probability of obtaining a test statistic. The lower the p-value, the less likely the result is if the null

1

Minitab _{is a statistics software package , and is often used in conjunction with the implementation of Six Sigma, CMMI and other}

hypothesis is true, and consequently the more "significant" the result is, in the sense of statistical significance” (Wikipedia-f).

The Pearson correlation coefficient between the users’ expectations and OSS usability is positive (0.22) at P < 0.05, and hence, hypothesis H1 is justified. For H2, the relationship between usability bug reporting and OSS usability, the Pearson correlation coefficient is 0.37 at P < 0.05, and hence, it is found to be significant as well. Furthermore, the

hypothesis H3 is accepted based on the Pearson correlation coefficient of 0.22 at P < 0.05, which represents the relationship between the interactive help features and OSS usability. The positive correlation coefficient of 0.49 at P < 0.05 is also observed between OSS usability and usability learning, which means that H4 is accepted. However,

hypothesis H5, which denotes the relationship between usability guidelines and OSS usability, yields a Pearson correlation coefficient of 0.11 at P = 0.27, and thus, this hypothesis is statistically insignificant and consequently, it is rejected. Hence, as observed and reported, hypotheses H1, H2, H3 and H4 are found to be statistically significant and are accepted, whereas H5 is not supported and is therefore rejected.

In the second phase, non-parametric statistical testing is conducted by examining the Spearman correlation coefficient between the individual independent variables, the usability factors, and the dependent variable, OSS usability, as displayed in Table 3.9. First, the Spearman correlation coefficient between the users’ expectations and OSS usability is found to be positive (0.28) at P < 0.05, and hence, hypothesis H1 is justified. For hypothesis H2, which examines the relationship between usability bug reporting and OSS usability, the Spearman correlation coefficient of 0.37 is observed at P < 0.05, and hence, this hypothesis is significant. Moreover, the hypothesis H3 is accepted based on the Spearman correlation coefficient of 0.27 at P < 0.05, demonstrating a statistically significant relationship between interactive help features and OSS usability. A positive Spearman correlation coefficient of 0.42 at P < 0.05 is observed for the fourth hypothesis, which represents the relationship between OSS usability and usability learning, indicating that H4 is also accepted. For hypothesis H5, which involves usability guidelines and OSS usability, the Spearman correlation coefficient of 0.03 is observed at P =0.79. Since no

significant relationship is found between the usability guidelines and OSS usability, H5 is rejected.

Hence, as observed and reported, H1, H2, H3 and H4 are found to be statistically significant and are accepted, whereas H5 is not supported and hence rejected in both parametric and non parametric analysis.

Table 3.9: Hypotheses testing using parametric and non-parametric correlation coefficients (Users’ Perspective)

Hypothesis Usability Factor Pearson

Correlation Coefficient Spearman Correlation Coefficient H1 Users’ Expectations 0.22* 0.28* H2 Usability Bug Reporting 0.37* 0.37* H3 Interactive Help Features 0.22* 0.27* H4 Usability Learning 0.49* 0.42* H5 Usability Guidelines 0.11** 0.03** * Significant at P < 0.05. ** Insignificant at P > 0.05.

In the third phase, the PLS technique is used to perform the cross validation of results obtained in Phase I and Phase II. Specifically, this method examines the direction and significance of hypotheses H1–H5. In the PLS technique, the dependent variable of the research model, OSS usability, is the response variable, and the independent key usability factors are the predicators. The test results containing the observed values of the path coefficients, R2 and the F-ratio are shown in Table 3.10. The first variable, users’ expectations, is significant at P < 0.05, with a path coefficient of 0.22, an R2 value of 0.49 and an F-ratio of 5.15. Furthermore, the variable of usability bug reporting has a

path coefficient of 0.58, an R2 value of 0.35 and an F-ratio of 15.57, and hence, it is significant at P < 0.05. The next variable, interactive help features, has the same direction as those proposed in hypothesis H3, with a path coefficient of 0.46, an R2 value of 0.50 and an F-ratio of 5.26 at P < 0.05, and so it is also significant. Moreover, the variable of usability learning conforms to hypothesis H4, with a path coefficient of 0.62, an R2value of 0.47 and an F-ratio of 32.82 at P < 0.05. Finally, the last variable, usability guidelines, has a path coefficient of 0.22, an R2 value of 0.12 and an F-ratio of 1.20 at P = 0.27. Hence, in this phase, as in Phases I and II, hypothesis H5, which deals with usability guidelines and OSS usability, is not found to be statistically significant at P > 0.05, and thus, it is rejected.

Table 3.10: Hypotheses testing using Partial Least Square regression (Users’ Perspective)

Hypothesis Usability Factor Path Coefficient R2 F- Ratio

H1 Users’ Expectations 0.22 0.49 5.15* H2 Usability Bug Reporting 0.58 0.35 15.57* H3 Interactive Help Features 0.46 0.50 5.26* H4 Usability Learning 0.62 0.47 32.82* H5 Usability Guidelines 0.22 0.12 1.20** * Significant at P < 0.05. ** Insignificant at P > 0.05

The multiple linear regression equation of our research model is depicted in Equation 3.1. For this statistical test, the testing process includes regression analysis, which yields the

values of the model coefficients and their direction of association. In this case, OSS usability is considered as the response variable and the usability factors are the predicators. As shown in Table 3.11, the path coefficients for all five variables are positive, whereas the t-statistics for four out of five variables, including users’

expectations, usability bug reporting, interactive help features and usability learning, are statistically significant at P < 0.05. In contrast, the t-value for the usability guidelines is observed as 0.40 at P = 0.68, thus making the variable of usability guidelines statistically insignificant in this research model.

Table 3.11: Multiple Linear Regression Analysis of the User Model Model coefficient

Name

Model coefficient Coefficient value t-value

Users’ Expectations f1 0.22 2.55* Usability Bug Reporting f2 0.20 1.79* Interactive Help Features f3 0.19 1.69* Usability Learning f4 0.38 4.40* Usability Guidelines f5 0.04 0.40** Constant f0 3.92 0.45* Significant at P < 0.05. ** Insignificant at P > 0.05

The R2 value and the adjusted R2 value for the overall research model are observed as 0.32 and 0.29 respectively, with an F-ratio of 9.10, which is significant at P < 0.05.

Recapping Equation 3.1 by inserting the model coefficient values, we get:

5 4 3 2 1 0.20 0.19 0.38 0.04 22 . 0 92 . 3 Usability OSS = + v + v + v + v + v

where v1, v2, v3, v4 and v5are the five independent variables.