4. An Overview of the Approach and Details of the Selected Programmes
4.7 Data Analysis
Statistics are normally used to confirm or validate the data to a certain degree. Descriptive statistics assist in better understanding of the data (Miller, 1995). Quantitative analysis of data gathered from the student survey was carried out using the software tool called SPSS (Statistical Package for Social Sciences; SPSS/PC, 2010 version 18) (IBM, 2011). It is important to prepare the survey data in a consistent format before analysis, therefore, the student responses received on email were first stored in ‘Snap’ survey management software (Snap, 2010) which was then exported to SPSS after refining them to align with SPSS data fields. Due to the comparatively large number of potential students involved in the online survey (188) compared to total number of interviewees (40), it is appropriate to use statistical analysis methods for data obtained from the student questionnaires. The data acquired from the rest of the stakeholders through interviews were analysed qualitatively.
Berk (2005) describes twelve potential sources of evidence to measure teaching effectiveness as: (a) student ratings, (b) peer ratings, (c) self-evaluation, (d) videos, (e) student interviews, (f) alumni ratings, (g) employer ratings, (h) administrator ratings, (i) teaching scholarship, (j) teaching awards, (k) learning outcome measures, and (l) teaching portfolios. The author has used more or less all these sources to measure effectiveness of WBL delivery as
‘student ratings’ - through student questionnaires and interviews with students ‘administrator ratings’ - through programme leader and support services interviews ‘teaching portfolios’ - through tutor interviews and students’ feedback on tutors ‘employer ratings’ - through employer interviews
‘peer ratings’ - through employer interviews and students’ feedback on peers ‘self-evaluation’ - through tutor interviews
‘videos’ - through tutor and support services interviews and students’ feedback on
content and use of technology
‘alumni ratings’ - through interviews of passed-out students, academics, employers,
representatives of PBs and support services who were ex-WBL students
‘teaching scholarship’- through programme leaders and support services interviews ‘teaching awards’ – through programme leaders, tutors and support services
interviews
‘teaching portfolios’ – through tutor interviews
‘learning outcome measures’ - through tutor interviews and students’ feedback on
learning contracts
The WBL support staff (University management, Library, IT Services, FEE WBL administrators, WRLS and the Student Union) added more insight into most of the features above.
The current study used three variables as main building blocks or enabling factors for a good learning experience as:
quality (learning materials, delivery, teaching/tutoring, acceptance/credibility)
access (programme content, programme leader, tutors, mentor at workplace, peer
students, university services)
support (University, Employer, PB, family, peer students)
The effectiveness of these variables was evaluated on the basis of how different stakeholders perceive their experience in activities such as learning, tutoring, mentoring,
administration, support and accreditation. Furthermore, the effectiveness of WBL was explored in the context of the delivery of programmes including the online learning aspects.
Effectiveness can only be measured if there are clear benchmarks and criteria being defined in terms of delivery of WBL. As mentioned above, the expectations of various stakeholders in a learning environment are very different from each other (Liyanage et al., 2010a). As a result, the angle of perceiving effectiveness is different among various stakeholders. For example, learners may perceive effectiveness from the quality of learning material, ease of accessing the content, the level of interactivity in the content, level of support by programme leaders, tutors, and employers, importance and acceptance of qualifications acquired through WBL for professional registration, and difficulties faced during studies. The tutors may perceive effectiveness from the student performance, university support for improvement of quality of learning materials, and opportunity for learning and using new technology. Further, employers may perceive effectiveness in terms of employee productivity, Return on Investment (RoI), and the support from the university whilst professional bodies may have a different view on effectiveness may be in terms of ease of accreditation of WBL programmes compared to face-to-face university programmes and also flexibility of professional registration of members through WBL qualifications.
Boulay, Coultas et al. (2008) in their review on how compelling is the evidence for the effectiveness of e-Learning in WBL place emphasis on employees’ skills. They concluded that despite the limited amount of empirical research on the effectiveness of WBL through e-learning programmes, they are still being adopted within organisations. Lewandowski (2007) in his investigation into the effectiveness of explicit instruction in language learning further confirms that learners’ ultimate attainment will be determined by a combination of values and consequently, the level of ultimate attainment is likely to vary from learner to learner as in all education. This further clarifies the researcher’s point of
view on effectiveness which can vary from person to person upon each individual’s
perception.
Although the research does not compare these WBL programmes explicitly against a face to face FT delivery provision, the researcher expected from stakeholders in their feedback to compare WBL with face-to-face FT delivery. Thus the effectiveness will be
evaluated on this basis of the stakeholders’ perceptions.
Tables 4.3, 4.4 and 4.5 show the matching of each question with variables identified.
Table 4.3 Mat hing of uestions in online students’ uestionnai e ith a ia les
Question Characteristics Variable
1 What is the distance/online WBL programme you are following?
Discipline vs WBL Demogra -phics (Indepen- dent variables) 2 What is your age? Age vs WBL
3 What is your gender? Gender vs WBL 4 What is your highest Educational
qualification?
Background Knowledge/ academic experience vs WBL 5 What is your main employment? Experience/ status/
responsibilities/ Support from superiors/ sub-ordinates vs WBL 6 On average, how many hours a
week do you spend on your main employment?
Time available for studies vs WBL
7 Who is sponsoring your studies? Self-financing commitment/ employer support/ Grants vs WBL
8 Where do you usually live? Distance/ Blended learning/ Hands-on part/Physical access to university resources vs WBL 9 From where do you access your
online programme/s?
Access/network support/technical compatibilities vs WBL
10 The main reasons why you have chosen online WBL
10.1 I can learn at my own pace Flexibility/ time/ pace/ beneficial for weaker and stronger students
access 10.2 I prefer self-learning Motivation/ Maturity/
background/ Quality of material/ Online tutor support
access
10.3 I do not need to go to university for studies
Distance/ time/ work place time access 10.4 I can learn anytime when I am
free
Flexibility/ time management/ online hours
access 11 An Induction at the beginning of
programme made / will make me comfortable using the
Blackboard e-Learning Portal (eLP)
Technology/ learner support/ Building confidence/ not all can attend (geography)/ Blended Learning
quality/ access
12 The eLP is very user-friendly Technology/ LMS sophistication/ learner support/ communication interface
access/ quality 13 Online discussions / chat
provided in eLP help me to share different views posted by others, as if I were in physical
classroom
Communication/ interactivity/ knowledge sharing/ community feeling/ difficulties in
synchronisation
access/ quality
14 I still prefer the inclusion of some face to face sessions in the programme because I miss the classroom environment
Blended learning/ physical
interaction/ community inclusion
Table 4.4 Matching of MCQ questions with variables contd.
15 The programme leader is very supportive and accommodating
learner support/ university standards
support
16 The feedback I get from my module tutors for queries and submissions is timely and responsive
learner support/ online
standards/ university standards
quality
17 My module tutor's subject knowledge and expertise to deliver online are of high quality
learner support/ expertise/ university standards
quality
18 My Mentor at my workplace supports me in my studies
Employer inclusion and
commitment/ WBL requirements / recognition of WBL/ Mutual benefits
support
19 My employer has supported my studies
Employer inclusion and commitment / WBL specific/ Industry standards/
recognition of WBL
support
20 The following university services are supportive and responsive 20.1 IT learner support/ university
standards/ technology
support
20.2
Library learner support/ university standards/ resources
support
20.3
Finance learner support/ university standards/ affordability
support
20.4
Student services learner support/ university standards
support 21 There is an effective monitoring
mechanism to ensure I progress through my programme in a timely manner
learner support/ university standards/
completion within deadline
support
22 Do you feel that the university is concerned when you are in difficulties?
Learner support support
23 I would like the inclusion of learning elements (Eg: quizzes, animations, graphics, audio and video clips, simulations,
illustrations, diagrams etc) to aid the understanding of subject content
Quality standards/ learner support/ university standards/ Interactivity/self-learning specific
support/ quality
24 The quality of learning materials of my programme is very high, relevant, and up-to-date
Quality standards/ learner support/ university standards/ university support to tutors/ technology
quality
25 My curriculum is very relevant to my role, duties and
responsibilities
Learner support/ employer commitment/ PB guidance/ WBL specific/Tailoring
Table 4.5 Matching of MCQ questions with variables contd.
26 After completion of this programme, my professional status should be upgraded and more employment prospects should be opened up
PB inclusion/ employer guidance and commitment/ future benefits for students/ WBL recognition/ Accreditation
support/ quality
27 This programme helps me with my personal development through increased knowledge and skills
future benefits for employee and employer/ university commitment
quality
28 I am likely to continue education in distance/ online WBL mode after this programme
Motivation/ satisfaction/ university quality standards/ acceptance and recognition of WBL qualifications
quality
29 It was difficult to adjust to online learning initially
Self-learning/ online learning specific/ paradigm shift/ background/ past experience
access
30 Have you benefitted from APEL?
WBL specific/
PB/ Employer/ University support
support/ access 31 Have you benefitted from
APCL?
WBL specific/
PB/ Employer/ University support
support/ access The collected data were analysed using narrative and statistical analysis whilst document analysis was used with the collected resources. Narrative analysis of rich, detailed, qualitative interview data focused on emerging theory, using the inductive
analysis process to arrive at an understanding of the ‘WBL’ phenomenon under
investigation. The process of qualitative analysis involves:
Comprehending the phenomenon under study
Synthesizing a portrait of the phenomenon that accounts for relations and links
within its aspects
Theorising about how and why these relations appear as they do
Recontextualizing, or putting the new knowledge about phenomena and relations
back into the context of how others have articulated the evolving knowledge (Morse, 1997). This is another stage where researcher’s own experience in analyzing qualitative data had an influence in contextualising new knowledge with already accumulated knowledge and extracting what is coming out of data from this study.
Interview data were analysed through content analysis using NVivo software (QSR, 2011). NVivo gives an easy way to categorise answers of all the respondents for particular questions by allowing for removal of redundant data and summarising them according to the same theme and strong points under each theme. Some of the NVivo screens showing
how the data was manipulated within the software are given in the Appendix VI. The
‘Autocode’ function in NVivo was the most useful among all to achieve the above task.
The collected student survey data was subjected to univariate and multivariate statistical analyses such as frequency, chi square (χ2), Classification And Regression Tree (CART) analysis and factor analysis. The dataset was tested for sampling adequacy to show whether the dataset is suitable for factor analysis (Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) which should be between 0 – 1 and closer to 1 will be an indication of suitability) (Field, 2005).
4.7.1 How Triangulation was Accomplished?
Qualitative and quantitative data analyses have resulted in outputs in narrative form, form of codes and also in statistical form. Collating all forms of data to triangulate provides insights into the relationships, similarities and differences from the research findings. Triangulation is achieved not only between qualitative and quantitative outputs
but also between different quantitative analysis techniques such as frequency analysis, factor analysis, CART analysis as well as Spearman Brown correlations as explained in chapter 6 and 7. Accordingly, across these analyses, the main themes coincided and were encapsulated into 3 main categories namely quality, access and support.