• No results found

CHAPTER 3. Research Method and Design

3.3 Rationale for whole-of-study approach and research

3.3.2 Research intervention design

The CoLeCTTE framework was used to guide the development of the learning environment in this study. Adapted from the activity theory model (see Section 2.5), CoLeCTTE presents the areas of the model where critical thinking and reflective processes take place during group work tasks as they are performed within an activity-based authentic learning environment. In addition, technology is used to enable collaboration during group work.

Case groups and participants were selected via participant selection and who performed collaborative tasks were working within a technology-enhanced environment (see of Figure 3.1). Case 1 was an undergraduate group, and Cases 2 and 3 drew participants from a postgraduate cohort based on the findings and questions derived from Case 1. Psychometric test measures and questionnaires comprised the quantitative data in this study to derive a profile of participants in terms of their eductive ability, demographical data, technological self-efficacy, learning approach and motivation, and critical thinking measures during group work (see of Figure 3.1).

Semi-structured interviews were conducted for the qualitative investigation in this study to determine what van Manen (1990) calls “the essence of lived

experience” and to investigate participant experiences in the use of collaborative and learning technologies during group work. Only Cases 2 and 3 participants were interviewed (following findings from Case 1) to determine participant perceptions, specifically in gauging their own performance on critical thinking tasks, the influence of learning and collaborative technologies, and approaches to their study and learning. The interviews were aimed at exploring the impact of technological self-efficacy, and the influence of learning and collaborative technologies on their study and learning behaviour in relation to critical thinking activities and tasks. This part of the research allowed further investigation into cause and effect especially from outliers in the group.

Finally, triangulation of quantitative data (measures of eductive ability, critical thinking, cognitive learning style and motivation, and critical thinking) and

qualitative data (interviews and observations) were used to determine relationships

conduct of the research study involved seven important steps to ensure the validity and reliability of results:

1. Selection of participants from:

a) Class 1: Core Project Implementation (Case 1 group);

b) Class 2: Industrial Electrical Power Distribution (Case 2 group); c) Class 3: Applied Thermodynamics (Case 3 group); and

2. Task analysis and design of critical thinking tasks within the learning context of the subjects using the CoLeCTTE framework;

3. Measurement of learner skills and profiling of case groups and

participants based on eductive ability, demographic data, technological self-efficacy, and cognitive learning style and motivation;

4. Performance measurement of critical thinking behaviour during group tasks;

5. Semi-structured interview of participants (Cases 2 and 3 only). 6. Inter-rater reliability testing; and

7. Interpretation, and statistical and case analysis of both qualitative and quantitative data.

Steps 1, 2 and 6 are further explained below while 3, 4, 5 and 7 are dealt with in Chapters 4, 5 and 6, respective to the three cases under investigation in this research.

3.3.2.1 Participant and activity design criteria

Referring back to Figure 3.1, participants were selected from undergraduate (aged 18 – 25) and postgraduate (aged 21 – 40) student cohorts with a selection frame defined as students coming from single degree programs ( ). They were

included based on High School or undergraduate approved qualifications and no international students were included in the study. These criteria were used to

purposefully select student participants who were homogenous with respect to age,

belonged to the same class, social, environmental and study load factors. To guide the purposeful selection of participants, university pivot tables (Bibby & Fuller, 2005) from which the participants were selected indicate a total of 2,348 students that fell into these criteria. Although stratification and clustering of the population meant a total target population N of 331 from which participant selection could be derived, it is important to note that representative sampling from a true N cannot be calculated “nor would we gain any analytic leverage by so doing” (Gerring, 2007). The selection of cases was justified according to the principles for case study selection outlined by Gerring (2007), where the participant criteria represented the variables which were typical of the “other cases” or the broader population to be targeted. Case groups or classes in this study were sampled from a different population (different technology subjects) and are unique within their own context, thereby providing internal validity because of their non-comparability, and therefore, are qualitatively different.

Overall, class groups were selected purposively to represent the generation of students that are technologically savvy, therefore, minimising technological proficiency as a factor of bias. This approach allowed other causal factors to be highlighted in the analysis. It was presumed that students from the same discipline and class would have similar prior knowledge bases and come from the generation which Oblinger and Oblinger (2003; 2005) defined as Millennials or the Net Generation, whose characteristics offer distinct differences to Generation X and Baby Boomers who are from previous generations. It was anticipated that there

could be a high level of interest amongst these students to using collaborative technologies to complete learning tasks or activities in this study; namely Vyew, Teamspot, Tablet PC and EDSA Power Analytics modelling tool (EDSA) (see of Figure 3.1). The study investigated the learning experiences of participant volunteers who belonged to one of the three case groups investigated in this study. These cases are briefly described and more detailed descriptions will be provided in Chapters 4, 5 and 6.

In the activity part of the schematic diagram (see of Figure 3.1), Case 1 is comprised of undergraduate students selected from the course Core Project Implementation. These students worked in teams to develop a business plan and implement the proposal outlined in the business plan. Case 2 comprised students from the course Industrial Electrical Power Distribution. These students worked in teams to create an accurate working model of an industrial power system that illustrated all the significant aspects of a typical power distribution system. Case 3 comprised postgraduate students from an Applied Thermodynamics course within a Masters of Engineering (Power Generation) program. These students worked in teams to develop a thermal efficiency project and produce a Justification Report for presentation to a mock board.

Finally, the CoLeCTTE framework provided for the integration of

participants, activities and technology. Participant and technology interaction were organised in accordance with the principles of activity theory (Ratner, 1996; Ratner, 1997; Vygotsky, 1978). Learning activities were capstone activities, contextualised and authentic, and critical thinking analysed based on participants demonstrating behavioural and cognitive manifestations of critical thinking during the activities. In developing these tasks, the following initial principles were applied: (a) the need for

students to interact with a problem set by the lecturer; (b) the need for students to identify resources and information of relevance which might be available and shared electronically; (c) the need for students to engage in critical discourse and perform higher order skills; (d) students collaboratively synthesised a position or response in relation to the lecturer’s challenge; and (e) responses and ideas were shared

electronically (Barnes, 1979, January; Kurfiss, 1988; Marzano & Others, 1988). Internal validity in this study was maintained by using a naturalistic approach and centring activity design towards facilitating deep learning and critical thinking. Cases 1 and 2 were not provided with direct instructions on how to behave critically. Rather, learning tasks were complex, case-based and problem-based which were considered to be authentic to the specific disciplines of participants in the study. As such, they were ill-defined and/or ill-structured so students had to define the tasks and sub-tasks needed to complete the activity which, therefore, educed the critical thinking behaviours (Bennett, Harper, & Hedberg, 2001; Herrington, Oliver, & Reeves, 2003). Hong (1998, pp. 3-4) describes ill-defined problems as not having a clear solution strategy but may allow single correct answers about which qualified experts would agree, while ill-structured problems have various solutions and solution paths so that experts in the domain do not agree on a particular solution. So these types of problems educe critical thinking

behaviours which are characterised as: defining, describing the critical issues or problems, determining accuracy and relevance of the information, identifying alternative ways of looking at problems and solutions by considering possibilities, making inferences, determining similarities and differences between issues, justifying and evaluating the merits or demerits of an assertion or belief and identifying causes of an event.

In addition, Cases 1 and 2 were not instructed on how to collaborate or share their learning artefacts. Dillenbourg (2002) posits that supporting

collaborative effort too much (specifically, scripting collaboration) raises the risks of damaging collaboration by destroying natural interactions, disturbing natural

problem solving processes, increasing cognitive load, ‘didactising’ interactions, and interfering with goal setting. To this end, the term implicit intervention for

extracting critical thinking behaviour is introduced as a non-prescriptive instructional approach from which learner performance could be derived. This type of

intervention approach allowed the researcher to observe critical thinking behaviour that arose as a result of technology integration and learner skill bases in the activity. Case 3, however, drew on feedback from Cases 1 and 2 to determine the necessary degree of intervention and changes to the CoLeCTTE framework. Clear and step- wise explicit intervention (provision of explicit and prescriptive instructional

guidance or scripts for performance of learning skills and tasks) were integrated into the design of the learning tasks for Case 3 to draw out critical thinking behaviour.

To ensure that the implementation of CoLeCTTE could be conducted in each of the cases, the instructional design principles underlying the CoLeCTTE framework were negotiated and an agreement reached between the researcher and course facilitators (i.e., lecturers, tutors or course coordinators). When no alignment could be reached between applying the principles of CoLeCTTE and course design, the research was not pursued. A total of four courses were considered for inclusion in this study. Only one subject called Project Delivery could not be included since the Course Facilitator felt that the approach to be used in the study would be in conflict with the objectives of the course. The researcher worked with each of the course facilitators to design their courses (from which Cases 1, 2 and 3 were derived)

which are discussed in Chapters 4, 5 and 6. In particular, Sections 4.2, 5.2 and 6.2 discuss how the design of course and activities in Cases 1, 2 and 3, respectively, aligns with the CoLeCTTE framework.