Data Analysis

As indicated earlier, the data collected through the questionnaire was keyed into PASW Statistics 18. Then, it was analyzed by using the aforementioned software. Descriptive statistics was applied in addition to dimension reduction technique – factor analysis - where the principle component analysis was used. The aim is for data reduction, as explained in the following section.

3.3.8.3.1 Factor Analysis

―Exploratory factor analysis is the most widely used statistical methods in the psychological research‖ (De Winter, Dodou, & Wieringa, 2009). This method was used in this study to extract factors out from the variables of the questionnaire to simplify the analysis and grouping of related variables. The major intention of the researcher was data reduction, therefore the principal component analysis was used as a suitable extraction method (Preacher & MacCallum, 2002) quoted by (Treiblmaier & Filzmoser, 2010), in addition to the fact that normal distribution is not prerequisite (Reimann, Filzmoser & Garrett, 2002) quoted by the same source.

The sample size has influence on the analysis, where the larger samples will have less probability of errors, more accurate estimates and generalizability (Treiblmaier & Filzmoser, 2010). Several recommendations for sample size are there in the literature, ranging from absolute sample size to ratio between subjects and variables (Treiblmaier & Filzmoser, 2010). However, according to MacCallum et al (2001) quoted in Treiblmaier & Filzmoser (2010) this could be oversimplifying of the issue as the ―population factors in data can be adequately recovered if communalities are high‖ (Treiblmaier & Filzmoser, 2010). They go further in saying that researchers usually recommend larger sample size than usual, when the communalities are low (Treiblmaier & Filzmoser, 2010).

The sample size for this questionnaire; as shown above is 57 with 48 valid cases. TThis is considered a small sample size as established in the literature, where the minimum accepted size of 50 is considered poor (De Winter, Dodou, & Wieringa, 2009). Small sample size is treated cautiously by researchers when using factor analysis. However, small sample size should not be of high concern to researchers and reviewers as indicated by Preacher & MacCallum (2002), if the communalities are high, number of factors is relatively small and model error is low. In this study, the communalities of all items were above 0.6 as shown in Table 7.4, and the number of factors extracted was six as shown in Table 7.2

Hogarty, Hines, Kromrey, Ferron, & Mumford (2005) and (Treiblmaier & Filzmoser, 2010) asserted that the sample size depends on both communality of the variables and overdetermination of the factors based, on MacCallum, Widaman, Zhang & Hong (1999) and MacCallum, Widaman, Preacher, & Hong (2001) studies. Overdetermination ―refers to the degree to which the factor is clearly represented by a sufficient number of variables‖ (Hogarty et al 2005) where it is considered so if it has ―high loadings on at least three to four variables and exhibit good simple structure‖

(Hogarty et al 2005). Similar recommendation can be found in De Winter, Dodou & Wieringa (2009), where they suggested that the ―lower sample sizes were needed when the level of loadings (λ; therefore the communalities) was high, the number of factors (f) small, and the number of variables (p) high‖ (De Winter, Dodou & Wieringa, 2009), which is in line with MacCallum et al (1999) theoretical framework as indicated by the same source. In addition to the above, several researchers have used small sample sizes in their studies regardless whether they employed factor analysis or not. Examples can be seen in the work of Rovai (2001), Ifinedo (2006), Henson & Roberts (2006), Van Raaij & Schepers (2008), Bangert & Easterby (2008), Yu (2009), Abedin, Daneshgar & D‘Ambra (2010a), and Abedin, Daneshgar & D‘Ambra (2010b), where the sample sizes were 20, 72, 60, 45, 53, 49, 47 and 40 respectively.

De Winter, Dodou, & Wieringa (2009) reported that in certain conditions, sample size was less than number of variables, although some studies and factor analysis guidelines argue that this should not be the case. On the other hand, they reported that Marsh and Hau (1999) proved that surpassing the equality barrier has no negative effect on the simulation results. In their own study, De Winter, Dodou, & Wieringa (2009) reported a similar outcome, while even going further in suggesting that increasing the number of variables was beneficial, even when it exceeds the sample size. They supported their argument and results with the proof by Robertson and Symons (2007), that this case is valid for maximum likelihood factor analysis, despite that such method considers such case as ―impossible because the covariance matrix turns nonpositive definite‖ (De Winter, Dodou, & Wieringa, 2009). They recommended going as far as possible in increasing number of variables provided that it does not challenge the overall quality of the set. In regards to the number of factors to be decided on, they recommend to go for most appropriate number of factors rather than the correct number (De Winter, Dodou, & Wieringa 2009). In addition to this, Preacher & MacCallum (2002) suggested that

decreasing the number of factors has a negative impact on the communalities, while increasing it, will compromises interpretability. In conclusion of the use of factor analysis, it has been asserted that ―considering that models are useful unless they are grossly wrong (MacCallum, 2003) and a small sample size factor analytic model is not per definition grossly wrong, applying factor analysis in an exploratory phase is better than rejecting EFA a priori.‖ (De Winter, Dodou, & Wieringa, 2009)

3.3.8.3.2 Lecturer Evaluation

To support and supplement the evaluation results of the model by the students; another evaluation, as a feedback from the participating lecturers, was used. It consists of open-ended questions to guide lecturers in the evaluation process. The form can be found in the appendix. The use of both quantitative and qualitative data – triangulation - would improve the validity and reliability of the results. In addition, conducting the evaluation by all the parties involved i.e. students and lecturers, will improves the credibility and validity of the evaluation, as it shows two different perspectives.