Quantitative analysis: Academics

Consistent with the mixed methods approach undertaken, the data analysis techniques employed in relation to the academic interviews were also a mix of quantitative and qualitative analyses. The key considerations to data entry for the information collected from academics is the first issue to be discussed in this section, followed by a summary of each of the quantitative and, in section 4.7, the qualitative examination techniques used.

4.6.1 Data entry

Both the face-to-face and telephone academic interviews were digitally recorded paper and pencil interviews (de Vaus, 2002), where the investigator recorded answers on paper questionnaires but also recorded the interview. The written responses were subsequently checked and verified by referencing the digital audio recording of each interview and the

93 respective interview transcripts. Respondents’ answers to the quantitative component of the survey were then manually entered into the online version of the instrument previously constructed and prepared on SurveyMonkey34.

4.6.2 Missing data

Missing data can be problematic for analysis, therefore to enhance the reliability and validity of the results, each item (missing or not) was coded (Allen & Bennett, 2012; de Vaus, 2002). The benefit of recorded and transcribed interviews, which include survey responses, is that the reasons for omitted data can be accurately identified and coded to increase the quality of the dataset. For the academic surveys there were three main reasons for missing data: the interviewer missed the question; a definitive answer could not be ascertained; or the question was not applicable.

Table 4.5 sets out each reason and the unique missing value code assigned to discriminate between the different reasons for the missing numerical data. Cases with a missing code of zero will be examined in detail with the qualitative responses, as respondents generously provided explanations where they were unable to definitively answer a question.

Table 4.5 Missing value codes

Code Reason Example

-1 Interviewer missed the question •Missed in error; or unable to complete due to time restrictions imposed by interviewee.

0

9

A definitive answer could not be ascertained

Question not applicable

•Where the context would result in different answers. Eg. ‘It depends…’

•Passed over if not relevant Eg. Some

coordination questions were not relevant for casual tutors

34 _{SurveyMonkey is a cloud based application providing survey software and data storage} online. It can be accessed at http://www.surveymonkey.com/

94

4.6.3 Univariate and bivariate analyses

Utilising the statistical software package for social science (SPSS), version 20, univariate descriptive analyses are conducted to present a distribution of the frequency of occurrence for each of the demographic variables for academics, and secondly to construct a table of frequency, central tendency and dispersion for each of the ordinal variables contained within the Likert scale questions. Given that Likert scales are ordinal in nature, the statistical significance of the descriptive values and data are analysed using Chi-square tests (de Vaus, 2002).

The key concept of bivariate tests is to simultaneously summarise data on two variables. Given the small sample size and the categorical nature of the data, nonparametric cross-tabulations are used to empirically indicate the direction, strength and significance of bivariate correlations (Coakes, Steed & Ong, 2009).

4.6.4 Principal component analysis

To examine the main issues for academics, regarding group work in accounting education, and to investigate the factors that contribute to their conceptions of group work, a principal component analysis (PCA) was conducted. Factor analysing techniques are among the most utilised statistical methods for Likert-scale questionnaires (Fabrigar & Wegener, 2012), and PCA is one form of data reduction that is commonly used to summarise a larger number of variables into more meaningfully focused key components. PCA differs from exploratory factor analysis (EFA). Fundamentally, ‘PCA is based on a different underlying mathematical model’ (Fabrigar & Wegener, 2012, p. 31), and unlike EFA, PCA is not designed to identify underlying latent constructs. Its purpose is essentially to describe both the unique and common variances of a group of variables. In other words, PCA is ‘a model in which a small set of principal components are constructed from the measured variables, and the ability of these components to predict the measured variables is assessed (as indexed by the principal component loadings)’ (Fabrigar & Wegener, 2012, p. 32). The PCA technique is therefore the most appropriate method to highlight the key components of group work for the academics in the current study, as identified by their collective responses to the initial rough scale35 items. To help clarify what variables belong together, and to ensure each component is clearly

35 _{Rough scale items refer to the initial construction of scales, given the need to subsequently} test their applicability and reliability for what they purport to measure (de Vaus, 2002).

95 interpretable, the final component list is extracted using the widely accepted, orthogonal varimax36 _{rotation procedure (Coakes et al., 2009; de Vaus, 2002). Finally, the non-} parametric alternative to the univariate between-groups analysis of variance (ANOVA), the Kruskal-Wallis test, was used to identify any significant differences in the demographic characteristics of academics, which may have influenced the outcome of the PCA.

4.6.5 Limitations

Given the relatively small number of academics surveyed in this study, inferences cannot be confidently generalised from the sample. In the same way, it is commonly agreed that PCA and univariate statistics are bound by several underlying assumptions that need to be addressed when conducting a quantitative analysis of the data (Allen & Bennett, 2012; Coakes et al., 2009), such as sample size. Conversely, recent research has questioned the validity of specific benchmarks and rules of thumb, such as needing a minimum five subjects per variable for factor analysis, as recommended by Allen and Bennett (2012) and Coakes et al. (2009) (de Winter, Dodou & Wieringa, 2009; MacCallum, Widaman, Zhang & Hong, 1999). Inconsistencies regarding absolute numbers have been apparent for some time (MacCallum et al., 1999), however the necessary conditions for what constitutes an acceptable ‘small’ number, particularly in relation to factor analyses, continues to be examined (de Winter et al., 2009). The latest evidence suggests that small samples, even those well below the standard benchmark of 50, are capable of producing good estimates if the data are well-conditioned37 (de Winter et al., 2009; Fabrigar & Wegener, 2012; MacCallum et al., 1999).

MacCallum (2003, p. 124) explains that ‘when communalities are high, meaning unique variances are low, sampling error will have a relatively low impact on results. But when communalities are low, meaning unique variances are high, the impact of sampling error will be high’. This means that for a sample size equal to the number of academics surveyed in the current study (23), high loadings (λ) of .8 can cohabit with a higher number of factors (f = 4), and a larger number of variables (p = 24), to produce a reliable

36 _{An orthogonal based rotation determines that the resultant components are uncorrelated with} one another (Coakes et al., 2009). Varimax maximises the variability in component loadings (Fabrigar & Wegener, 2012).

37 _{Research suggests that well-conditioned data include high loadings, a high number of variables,} and a small number of factors (Fabrigar & Wegener, 2012).

96 solution at a 95% confidence level (de Winter et al., 2009). Without dismissing the overall relevance of size and the fact that increasing the absolute number in the sample remains beneficial (de Winter et al., 2009), it is important to note that the limitations normally associated with small sample sizes can be embraced. Furthermore, it is reassuring to note that factorising models are an artificial approximation of reality (Fabrigar & Wegener, 2012; MacCallum, 2003), and therefore acknowledging their imperfections (as long as they are not grossly incorrect), can usefully make interpretations that help to clarify the nature of the phenomena of interest (MacCallum, 2003).