Evaluations Internet Based Learning Environments

Findings from three studies: Collaborative Learning Environments conducted by Lockyer (2000); Assessing Students’ Perceptions of Synchronous Internet-based Learning

Environments by Teh (2001) and Building a Profile of the Web Based Learner by Young (2006) presented at the Australian Association Research in Education (AARE) conference were selected because their research designs, analysis processes and findings were useful when developing the design for this study of information technology teacher’s perceptions of online oncampus instructional material. The research strategies followed in each of these studies are also of interest because they provide examples from both the Scientific and Constructivist paradigms and suggest, with respect to this study, where each is likely to be appropriate.

Study 1

An investigation of collaborative learning environments conducted by Lockyer (2000) examined health education delivered via the Internet. The report of findings was presented to the AARE conference by the designers of the course. They began their study believing that online learning would have advantages for collaborative learning, specifically that it promised:

• an increase in the variety of resources available • the elimination of timetable constraints

• an increase in the range of group discussion and collaborative learning options • increased exposure to diverse opinions and cultural perspectives.

Their strategy for moving from face-to-face to online learning began with the identification of material to be covered. This included gaining consensus and a shared understanding of what was to be achieved among “learning groups” (p. 2). The learning groups, comprising four to five students, were subsets of the larger tutorial groups. The criteria for the selection of participants are not recorded. This makes it difficult to determine if participants were self- selected and if this factor equated with greater skill or even confidence in the use of online resources than would be the case in a random selection of students. Not withstanding, this is an examination of a pedagogical model within which an Internet based learning model was designed and a prototype developed, and is of interest. The idea of using learning groups to identify, develop and trial learning material appropriate to an online situation arose from the belief that “it was not possible to move directly from the face-to-face teaching and learning environment . . . without careful re-conceptualisation of how the pedagogy can be adapted . . .” (p. 2). The specific focus of this prototype was to develop processes for matching

collaborative learning activities to appropriate technologies. The methods employed in this study are considered in the Methodology, Chapter 3.

Outcomes reported in Lockyer’s (2000) study indicated that while learners contributed fewer individual reports to groups, individual contributions showed far greater detail in the group discussions that were web or chat room based. The overall conclusion from his study was that design and pedagogy must be considered together and as such be re-evaluated. The Lockyer (2000) study presents an evaluation of numerous specific features of a collaborative online learning environment. The author argues that his study has two major findings for those organizing online education. First, skills in studentship and in group management need to be taught with particular regard to facilitating group decision making and secondly, that students need adequate access to lecturers.

Study 2

Teh’s (2001) study Assessing Students’ Perceptions of Synchronous Internet-based Learning Environments is an examination of the survey used to evaluate the relationship between a university computer-lab environment and student outcomes. It was presented as a report at the AARE 2001 conference. The course is described as a prototype comprising learning activities conducted in real-time [synchronous] and utilising “web-based mediated [tutor controlled] conferencing and telecomputing approaches” (p. 1). The course being evaluated operates under the jurisdiction of Singapore’s Ministry of Education and within “the context of The Masterplan . . . a blueprint for the integration of information technology in education . . .” (p. 1).

In Teh’s (2001) study, Moo’s (1974) social scale, shown in Table 2.5, was used to provide a framework for the evaluation of the prototype. The survey was specifically evaluating the quality of the information, the external links recommended in the course notes and students’ perceptions of learning activities. An interesting feature in his description of the course was the ‘electronic reserve shelf’. Teh (2001) describes it as mirroring the function of a class folder on the reserve shelf of a traditional library, but with all the benefits of instantaneous electronic and remote access. There was also an important additional feature for those administering online courses called the “Web guide”. It serves to “point students in the right direction . . . keep their research or online interaction on track . . . [and] prevent information overload” (p. 3). The evaluation instrument resulting from Teh’s (2001) study is a survey. A selection of categories from that survey is shown in Table 2.5 below.

Table 2.5 Social scale -computer-lab environment - student outcomes Scale Name

Teh’s (2001)

Description Moo’s Description

Investigation Extent to which skills and processes of inquiry

are used in problem solving and investigation. Personal Development

Innovation Extent to which the teacher plans new and varying activities and techniques and encourages students to think creatively.

System Maintenance [change dimensions]

Resource availability Extent to which the computer hardware and

software are adequate. System Maintenance

The findings from Teh’s (2001) study were analysed statistically within a scientific methodology. He stated in the conclusion that this survey has been ‘cross-validated’ as an instrument for “assessing student perceptions of internet based learning classroom

environments” (p. 6). Teh (2001) concluded that it could be used with confidence. As in the previous two studies’ focus on online learning they are of interest and considered in the Methodology, Chapter 3.

Study 3

Young (2006) presents interim findings from an ongoing study that aims to build a profile of the young web-based learner. This report is based on a case study of five participants using “observation, think-aloud protocols and post-activity interviews . . .” (p. 1). Young notes that in the years 2000 to 2003, “9-12 year olds in the USA recorded a growth in web-based exploring activities from 22% to 58%” (p. 1). Drawing on this trend and citing other statistics from similar research, Young contends that the web affects informal learning and this in turn will change formal learning. In the development of this argument, Young identified the web as a cognitive tool which will influence the individual learner . . . and “[in turn be] influenced by the individual’s actions . . .” (p. 4). Young’s study is conducted within a theoretical framework derived from three components of Cognitive Learning Theory: situated cognition; distributed cognition and activity theory. They are described by Young as follows:

First, Situated Cognition: the relationship between the students and their environment can be used to explain how learning occurs. In the theory of situated cognition the knowledge gained is “bound by . . . the materials and environments (i.e. tools, rituals and physical spaces . . .” (p. 6) making up the learners’ environment. Barab and Plucker (2002) concur with the conclusion that “effective learning occurs when it is situated in authentic activity . . . . Where authentic activity is the “ordinary practices of culture . . . ordinary people doing ordinary things” (p. 6).

Secondly, Distributive Cognitive theory: knowledge does not exist as an entity but evolves from the interaction between “tools, rules, values, artifacts” and the individuals inhabiting the learning environment. Through a sequence of culturally developed tools and activity the learner completes an action which “leads to new cognitions of what, how and why one needs to know” (p. 7). In this scenario, the web is a culturally developed tool and can be viewed as “forming part of the learner . . . rather than an external” (p. 7). The Internet is in a co-existent role with the learner.

Thirdly Activity theory: derives from the notion that learning is not a study of individual’s “learning in isolation, with only their minds to guide them [but rather] . . . individuals learning with a wide variety of tools to assist them to carry out goal oriented activities” (p. 8). It is socially organised practical activity and tool mediated activity that unites the mind with real world activities and events. Young’s (2006) research uses the “Activity Theory triangle” shown in Figure 2.1 to examine the relationship between the web and the individual learner.

Figure 2.1 Activity Theory Triangle

Young (2006) believes the web, is a tool which provides an opportunity to observe both interactive learning and the dynamic changes in the tool itself as cognition progresses. Young’s research identifies activities, curriculum and “interpersonal relationships as elements of the learning environment” (p. 9). The study targets these elements specifically for

examination believing they are reciprocally affected by the changes technology causes.

The strategies and methods of a naturalistic enquiry are used by Young (2006) to uncover themes. The preliminary findings from these processes are presented here as emergent

categories. The five categories which emerged are: societal and cultural influences; situational boundaries; goals and outcomes; activities undertaken by individuals; the web as a cognitive tool and attributes of individual learners. Young notes as a concern, that these initial emergent categories “roughly related to the elements of interest raised through the integrated theoretical underpinnings of the study” (p. 11). Subsequent cycles of analysis eventually led to the individual learner being declared the unit of analysis and the following three robust emergent categories: participant citizen; tool mediated citizen and adaptive citizen.

In Young’s (2006) study these three emergent categories lead to an examination of the learning environment comprising society, tools and individuals. They are listed as primary categories for the analysis process of her research. These primary categories and the secondary categories that emerged in subsequent analysis are shown in Table 2.6. The web based learner profiles for each is shown in Tables 2.7 to 2.9.

Figure 2.1 Application of Activity Theory to Web-Based Learning

was removed to comply with copyright.

The image can be viewed on page 6 of Young’s Study

Building a Profile of the Young Web-Based Learner

www.aare.edu.au/you04191.

Table 2.6 Profile of the Young Web Based User

Participant Citizen Young (2006) Tool-Mediated Citizen Young (2006) Adaptive Citizen Young (2006)

Global Citizen Commercial Citizen Communicative Citizen Abiding Citizen Technician Security Guard Integrator Design Analyst Efficient Worker Researcher Director

Table 2.7 Profile of the Young Web Based User: Descriptors of a Participant Citizen

Participant Citizen Young (2006)

Global Citizen

Understanding of their community in Australia and Australia as part of the world Awareness that language variations occur across cultures

Awareness of the impact of global distances on activities Awareness of the digital divide

Understanding that different community groups throughout the world use the web for different purposes

Commercial Citizen

Understanding of economic values

Consumerism: exposure to online advertising;

knowledge of brand names and commercial organisations; online shopping activities

Communicative Citizen Use of email

Use of chat rooms and messenger services Understanding iconic representations Abiding Citizen

Rules set by location boundaries Engaging in socially accepted practices

Table 2.8 Profile of the Young Web Based User: Descriptors of a Tool-Mediated Citizen Tool-Mediated Citizen Young (2006)

Technician

Competence (master v. apprentices) Efficient practices

Security Guard

Using logins and passwords Awareness of safety issues Integrator

of various human senses

of different online and offline activities of available hardware and software Design Analyst

Analysis of various web site layouts

Interpreting displays and graphical representations Understanding of use and limitation of the web

Understanding of skills and knowledge required or facilitated through web usage

Table 2.9 Profile of the Young Web Based User: Descriptors of an Adaptive Citizen

Adaptive Citizen Young (2006)

Efficient Worker

Scanning for key words Skimming texts Evaluating material Automaticity in response Researcher

Search for information Navigate through sites

Comparison of tools for purpose Director

Flexibility Persistence Focus

Personal preferences

Self regulation/ Self Monitoring

Young (2006) illustrates the difficulty of making judgments about a ‘learner’s success’ in her study by comparing the notion of competence in the past and present using the example of the Adaptive Citizen: Efficient Worker (Table 2.9). Competence in the past was “generally claimed when the learner demonstrated both accuracy and fluency with a given task or situation [in a given time]. . . ” (p. 16). In web based learning, Young believes the concept of speed is different and can be viewed in numerous forms including: scanning for key words; skimming for context; interpreting graphics; evaluating material towards goals and

automatically responding to tool (Internet) related functions. Young (2006) concludes from her study that participants’ speed has evolved in line with their “personal engagement” with the web and that “if children are automatically responding rapidly in the web based learning environment . . . we must try to understand how this technique has changed learning and what . . . educators need to do to facilitate growth and maximise experiences” (p. 17).

This section has outlined the findings from a variety of research reports of Internet based learning. The reports examined and elaborated the previously discussed pedagogies in the context of e-learning environments. They also provided a variety of strategies and instruments for evaluating the effectiveness of online oncampus courses.