Enhancing Students Level of Critical Thinking through Interaction in Problem Based Learning and Computer Supported Collaborative Learning Environment

Enhancing Students’ Level of Critical Thinking through Interaction in Problem

Based Learning and Computer Supported Collaborative Learning Environment

Siti Mastura Baharudin

Universiti Sains Malaysia Penang, Malaysia sitimastura@gmail.com

Jamalludin Harun

Skudai, Johor, Malaysia jharun2013@gmail.com

Abstract— The purpose of this study is to investigate the potential impact of PBL-CSCL learning environment on students’ level of critical thinking skills and cognitive performance. This study will be carried out in ten weeks among 37 students who enrolled in Programming Language Concepts and Paradigms course. The independent variable of this study is the pattern of interaction types while the two assessment techniques which are the students’ critical thinking skills and cognitive performance are the dependent variables. The study seeks to identify the best pattern of interaction that occurs in the PBL-CSCL learning environment that helps to maximize students’ critical thinking skills and cognitive performance. Further investigation involves in finding if there correlation between students’ level of critical thinking and subject specific cognitive performance with types of interaction occurs during learning. This is a quantitative research design that involves the use of quantitative and qualitative data and adopts one group pre-test and post-test design. The findings of the study will help instructors in understanding students’ preference of interaction types in a real world learning activities in order to enhance their critical thinking and cognitive performance.

Keywords— critical thinking; interaction; PBL; CSCL I. PROBLEM STATEMENT

Technical and information and communication technology (ICT)-based courses are constantly changing with their syllabus undergoing alterations to keep up with the developing technology. Various newly developed techniques will be adapted to the course syllabus in accordance with the changes in technology. This proves that problem-solving techniques in technical and ICT-based courses are closely related to real-world problems in which the problem-solving techniques for these courses are also constantly changing and updated from time to time [1]. In order to empower students to understand the concept of a course and later, to be able to apply these concepts in real world problems, students need to have high level of critical thinking skills [2] where they would be able to process and apply the information they have received.

To achieve this, students are required to master higher-order thinking skills such as critical thinking as emphasized in the recent teaching and learning (T & L) process. Thus, student-oriented learning approaches are important whereby students act as the source of information and not just the recipients. Critical thinking skills have an important role in activating the process of

problem solving and decision-making for a student during the learning process [3].Critical thinking skills can help graduates apply the knowledge they acquired while in university to real-world working situations. Unfortunately, according to [4], the level of critical thinking skills among students are still low, thus, appropriate measures should be taken to assist students in improving their level of critical thinking skills. Since critical thinking skill often cited as learning outcomes, it is considered as an important issue [5].

According to [6], critical thinking skill can be improved through appropriate learning approach. PBL is said can be an approach that is believed can help students in improving their critical thinking skill level. Problem-based learning (PBL) is the use of a real world problem or situation as a context for learning and uses problems as the foundation for learning and focuses on activities while learning [7]. This is because in this approach, learning occurs when students are driven by their curiosity and desire to find the solution to all the problems that have been given. PBL is an approach that can improve collaboration and student interactions during the learning process [2]. Indirectly, applying PBL as part of the teaching process can help improve students’ critical thinking skills [1].

However, any effective learning strategy would be meaningless in the absence of an effective medium to help during T&L. Additionally, [8] and [9] have noted that the effectiveness of critical thinking skills may be enhanced with online learning where conversations and discussions can take place via the web. [5] indicated that online discussions can promote to the enhancement of critical thinking skill among learners.

Online learning initially begins with the focus on making courses available for distributed students both individually and collaboratively using computer [10]. Despite of its rapid growth, the effectiveness of online learning remains questionable. It is reported that online learning is a complex learning compared to face-to-face learning [11]. Since, online learning is mostly a text depended online learning environment, it is prone to misinterpretation and resulting in failure in message delivery between learners as well as instructors [12].

Access to education should not mean merely access to content rather; it should mean access to a learning environment that provides opportunity for interaction and connectedness [13]. It requires students to be highly

2014 Fifth International Conference on Intelligent Systems, Modelling and Simulation

responsible for their learning, independent learning and requires an amount of self-motivation [12]. However, due to the difficulty in building effective interaction between learners in online settings, students might feel isolated and would result in lack of motivation for participation in learning. Thus, it is important that students are learning in an online setting where students’ learning performance can be assessed and observed through their cognitive engagement. Furthermore, a successful online learning setting is measured through its ability to enhance students’ cognitive performance as well as interaction aspects [2].

Originating from collaborative learning, computer supported collaborative learning (CSCL) is the process where students work in small groups to achieve common goal and knowledge is transmitted among students either understanding of a subject or a solution to a problem, mediated by computers [14][15]. However, students working in small groups does not guarantee the effectiveness of learning and it is indicated that students’ online discourse are mostly information-sharing statement which is categorized as surface level of critical thinking. It is evidence that students often do not pose critical analysis with peers during online discourse [5].

The addressed issue is due to lack of meaningful interaction in CSCL environment. In order to achieve higher order thinking in online discussion, collaborative tasks has to be structured to promote meaningful interaction. Interaction between individuals and collaborative learning activities are the important facets in CSCL [17][18]. Productive interaction is an indicator of a successful of a collaborative learning is. Thus, interactions and learning activities has to be designed effectively to ensure the successful of a CSCL and to achieve learning outcomes [16]. Upon defining meaningful interaction for CSCL environment, the effectiveness of the environment and the successfulness of learning can too be measured through students’ performance in tests and their cognitive engagement.

This study believes that effective interaction characteristics in online learning environment particularly CSCL combining with PBL approach will effect positively on students’ level of critical thinking and cognitive performance. This study argues that interaction should not be considered as a whole and should be investigated further to seek different types of interaction and its pattern. Therefore, this study seek to investigate different types pattern of interaction and how each of it effects students’ level of critical thinking and their cognitive performance.

II. RESEARCH FRAMEWORK

This study consists of three main stages with regard to the ADDIE model that proposes five phases for developing online learning environment. The first stage of this study involves the analysis phase where students' perception of the learning environment will be taken into consideration. Students’ perception has been outlined based on the collaborative learning characteristics as proposed by [17] (i.e., students are divided into small

groups, students work together to achieve the same goal) and [19] (i.e., students are responsible of their learning, students are able to reflect on their assumptions and thought processes and learning must take place in an active mode) as well as the effective interaction characteristics as proposed by [20] (i.e., students are able to give feedbacks to group members, students are able to exchange information or materials among group members and students provide guidance to group members).

Meanwhile, the second stage of this study is a combination of the design phase and the development phase. The learning environment is developed based on the characteristics of the CSCL proposed by [21] (i.e., small size groups consists of three to four members, teacher will act as facilitator, tasks given are ill-structured and authentic, students will be evaluated based on their performance, provide opportunity for students being responsible of their own learning and interactions are mediated via computer) and PBL as proposed by [22] (i.e., problem is the starting point of a learning, problems represent real world situations, learning is student-centered learning process, activity-based learning is the central part of learning process and students learning are build from their experiences and interest). Additionally, the collaborative and interaction features that have been compiled from the findings in the first stage, based on students’ perception, will be incorporated in this stage. Once the design phase is completed, the newly developed learning environment will be refined in order for it to be applied to students.

Finally, the third stage of this study is a combination of the implementation and evaluation phases. After the development of the learning environment, it will go through the implementation phase and be used by students. This step will be followed by the assessment phase which will be conducted on the students. During this phase, the effectiveness of the learning environment will be evaluated through performance tests (pre-test and post-test). Another aspect that will also be evaluated is the critical skills among students using the model [9] that indicate four levels of critical thinking skills (i.e., triggering events, exploration, integration and resolution). Then, five types of pattern interactions (i.e. centralized knowledge exchange, distribute knowledge exchange, group development impediment, ability impediment and partial knowledge exchange) that were posed by each group will be identified using IBIS model proposed by [23].

III. RESEARCH QUESTION

The study investigates the developed PBL-CSCL learning environment to help students improve their level of critical thinking and their performances in a subject specific test. It also aims to identify pattern of interactions that occur during learning. Hence, the study posed the following inquiries:

1) What is students’ perception on suitable online learning environment?

2) Is the developed learning environment really effective in terms of:

a) Improving student's performance in a test? b) Increasing the level of critical thinking skills

among students?

3) Does interaction among students help to improve their critical thinking skills?

4) Which pattern of students’ interaction can help improve critical thinking skills among students?

IV. RESEARCH DESIGN

This study adopts quantitative research design that involves quantitative and qualitative data. To find the outcome of the study, one group pre-test and post-test design is used to gain and analyze the data. Quantitative and qualitative data gained will triangulated to identify the association between students’ pattern of interaction and their level of critical thinking in the learning environment. Instruments will be used to collect data for the study includes performance tests, log data, students’ portfolios, online transcripts, interviews and questionnaires. Quantitative data will be analyzed via descriptive analysis, inferential analysis and data mining while the qualitative data will be analyzed using content analysis technique.

V. DATA COLLECTION

Data collection processes in this research are divided into two phases:

A. Phase 1: Collecting Initial Data

Prior in developing PBL-CSCL learning environment, students’ perceptions on suitable online learning characteristics are first gathered.

Instrument used for this phase is questionnaire that are administered to one hundred fifty-nine second year students who were selected using stratified sampling method. The questionnaire consists of thirty-six Likert Scale questions and is divided into two sections: students’ preferences on collaborative learning and effective interaction in order to develop the PBL-CSCL environment. For students’ preferences on collaborative learning section, it consists of five constructs (i.e., students are divided into small groups, students work together to achieve the same goal, students are responsible of their learning, students are able to reflect on their assumptions and thought processes and learning must take place in an active mode). While for effective interaction characteristics section, it consists of three different constructs (i.e., students are able to give feedbacks to group members, students are able to exchange information or materials among group members and students provide guidance to group members).

Additionally, there are ten open-ended questions to obtain further insight on their perceptions.

B. Phase 2: Collecting Data for Actual Study

The actual study was carried out in ten weeks, involving thirty-seven students enrolled in the Programming Language Concepts and Paradigms course. This group of students was selected using cluster sampling. They are divided into small groups of three to

four students respectively. Overall, ten groups were formed.

Five authentic real-world tasks were given through the PBL-CSCL learning environment using the Moodle LMS. Each task was structured based on [22]`s principles. In order to motivate the students to think critically and optimize their interactions, the learning activities were designed using the principles by [17][19][20]. Three programming language were used in the tasks are JavaScript, Visual Basic and C++. Students were required to compare and enhance the five tasks from certain dimensions as a compiler point of view while instructor monitor and scaffold students’ interactions in the PBL-CSCL learning environment.

Students were asked to work in group while all interactions and learning activities are mediated by PBL-CSCL environment. Learning tools used in the learning activities are chat, forum, journal and blog. Additional features like notes, hyperlink, videos and announcement are embedded into learning activities as well. Students’ level of critical thinking is evaluated using model adapted from [9]. Also obtained from the learning environment were students’ portfolios, online transcripts and log data to identify the pattern of interaction and their level of critical thinking while learning. Students’ patterns of interactions will be identified using model from [23], that concern on critical thinking skills in online interaction.

To obtain the qualitative data, 10 students were selected using purposive sampling to take part in the interviews. 20 semi-structured interview questions were prepared for each student. Interviews were conducted to gain in-depth understanding of the influential effect of PBL-CSCL learning environment with students level of critical thinking as well as their pattern of interaction that promote their critical thinking skills. Specifically, the questions were divided into inquiries related to understand students’ critical thinking skills and activities related to their interaction patterns during learning. Log data gained from PBL-CSCL learning environment will be evaluated in order to identify students’ learning pattern. The results will suggest the process of a student`s involvement in learning activities and their patterns of using the learning environment.

VI. DATA ANALYSIS

Data analysis in this study adopts descriptive and inferential analysis. Quantitative and qualitative data obtained from data collection phase will be triangulated to get the findings of the research.

A. Phase 1: Data Analysis for Initial Data

To answer first research question, data gained from first phase will be evaluated. Questionnaire consists of thirty-six Likert scale questions and ten open ended questions will be analyzed.

Likert scale questions on collaborative learning with five different constructs adapted from [17] and [19] and effective interaction constructs with three different constructs adapted from [20] are measured to obtain their mean values.

While, open-ended questions consists of five questions on collaborative learning and five questions on effective interaction. Qualitative data from these questions are analyzed using content analysis technique to extract recurring themes.

B. Phase 2: Data Analysis for Actual Data

The actual study will be analyzed to answer research question two, three and four which intend to investigate students’ level of critical thinking and pattern of interaction in PBL-CSCL environment.

To determine students’ subject specific cognitive performance, marks from both pre-test and post-test will be run by paired sample t-test to determine the significance difference.

Students’ online learning portfolio and interaction transcripts will be evaluated using content analysis technique after an inter-rater reliability procedure. Using the indicators from model [6], the portfolios and interaction transcripts will be coded into four levels of critical thinking skills (i.e., triggering events, exploration, integration and resolution). All learning tools used in the learning activities as means of interactions i.e. chat, forum, journal and blog will be coded as different learning tools according to level critical thinking involved.

The same procedure will be adopted to analyze pattern of interactions gathered from the portfolios and interaction transcripts. Students’ online transcripts and portfolios will be evaluated using descriptors and indicators from model [16]. The online transcripts and portfolios will be coded into nine types of interactions (i.e. issues, positions, arguments, group development, response, acceptance of response, objection to response, support request, and conflicts). Once the interaction types are coded, the results will be further categorized into five different pattern of interactions (i.e. centralized knowledge exchange, distribute knowledge exchange, group development impediment, ability impediment and partial knowledge exchange). All coded results will be analyzed by frequency counts, forming the quantitative data point of the study. Nonlinear correlation (eta) will be used to identify inter-relationships between types of pattern interactions and students’ level of critical thinking.

The qualitative data on the other hand, will be analyzed via content analysis in order to extract recurring themes. Finally, quantitative data will be analyzed using decision tree analysis by data mining.

VII. SIGNIFICANCE OF STUDY

The result from the study will identify the correlation between certain patterns of interactions that occur in the PBL-CSCL environment in addition to determining students’ level of critical thinking. Consequently, the best pattern of interaction that can help to improve students’ level of critical thinking can be concluded. The study serves as a guideline for lecturers on how to successfully conduct online classes for technical courses that can also improve students’ level of critical thinking. Course designers can also benefit by gauging ways to design learning activities in an online learning environment

specifically concerns on interactivity effectiveness in learning.

VIII. EXPECTED FINDINGS

It is expected that the proposed learning environment may help to improve students’ performance and their understanding in the Programming Language Concepts and Paradigms course and also improve their level of critical thinking skills. At the end of the study, a framework of a learning environment on how to effectively conduct PBL-CSCL environment will be proposed. This framework will suggest the best interaction pattern that can help the lecturer to assist students’ learning with.

REFERENCES

[1] S. Sendag and H. Odabasi. Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills. Computers and Education, vol. 53, no.1, pp. 132-141, 2009.

[2] H. Bai. Facilitating Students’ Critical Thinking in Online Discussion: An Instructor’s Experience. Journal of Interactive Online Learning, vol. 8, no. 2, pp. 156-164, 2009.

[3] C. E. Leach, M.A. Kilfoil and T.A. Little. Weaving Innovation into the Fabric of Transformation: An e-Training Approach to Innovation. Privacy, Security, Trust and the Management of e-Business. CONGRESS '09. World Congress on, vol. no., pp.11-20, 25-27 Aug. 2009.

[4] S. E. Walker. Encouraging the disposition to critically think. Athletic Therapy Today, vol. 10, pp. 42-44, 2005.

[5] C. Perkins and E. Murphy.Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, vol. 9, no. 1, pp. 298-307, 2006.

[6] A.C. C. Chiang and I.P.W. Fung. Redesigning chat forum for critical thinking in a problem-based learning environment. The Internet and Higher Education, vol. 7, no. 4, pp. 311-328,2004. [7] M. R. de Villiers and Q. R. Queiro. Real-world problem-based

learning: a case study evaluated. South African Journal of Higher Education, vol. 17, no. 1, pp. 112-122, 2003.

[8] E. Derwin. Critical Thinking in Online Vs. Face-to-Face Higher Education. Media Psychology Review, vol. 2, no. 1, 2009. [9] D. R. Garrison, T. Anderson, and W. Archer. Critical thinking,

cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, vol. 15, no.1, pp. 7 — 23, 2001.

[10] M. Ryan, K. H. Carlton and N. S. Ali. Evaluation of traditional classroom teaching methods versus course delivery via the World Wide Web. The Journal of Nursing Education, vol. 38, no.6, pp. 272-7, 1999.

[11] C. Angeli, N. Valanides and C. J. Bonk. Communication in a web- based conferencing system: The quality of computer-mediated interactions. British Journal of Educational Technology, vol. 34, no.1, pp. 31–43, 2003.

[12] S. W. Lee and C. Tsai. Students’ perceptions of collaboration, self-regulated learning, and information seeking in the context of Internet-based learning and traditional learning. Computers in Human Behavior. vol. 27, no. 2, pp. 905–914, 2011.

[13] A. P. Rovai and K. T. Barnum.On-Line Course Effectiveness: An analysis of student Interactions and Perceptations of learning. Journal of Distance Education. vol. 18, no.1, pp. 57-73, 2003. [14] C.C. Tsai, Lin, and S. M. Yuan. Developing science activities

through a networked peer assessment system. Computers and Education, vol. 38, no. 1, pp. 241-252, 2002.

[15] L. Lipponen, M. Rahikainen, J. Lallilmo and K. Hakkarainen. Patterns of participation and discourse in elementary students computer-supported collaborative learning. Learning and Instruction, 13, 487–509, 2003.

[16] J. W. Strijbos, T. A. Ochoa, D. M. A. Sluijsmans, M. S. R. Segers and H. H. Tillema. Fostering interactivity through formative peer assessment in (web-based) collaborative learning environments. In C. Mourlas, N. Tsianos, & P. Germanakos (Eds.), Cognitive and emotional processes in web-based education: Integrating human factors and personalization (pp. 375-395). Hershey, PA: IGI Global, 2009.

[17] D. W. Johnson and R. T. Johnson. Learning together and alone: Cooperative, competitive, and individualistic learning (4th ed.). Boston: Allyn & Bacon. 1994.

[18] K. Kreijns,

P.A. Kirschner and Jochems. Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research. Computers in Human Behavior. vol. 19, no. 3, pp. 335–353, 2003.

[19] P.A. Kirschner. Using integrated electronic environments for collaborative teaching/learning. Learning and Instruction, vol. 10, no. 1, pp. 1-9, 2001.

[20] D. W. Johnson and R. T. Johnson. Cooperation and the use of technology. In D. H, Jonassen, (Ed.), Handbook of research in educational communications and technology. New York: MacMillan, 1996, pp. 1017- 1044.

[21] B. H. Lee. Conditions affecting Computer Supported Collaborative Learning in Higher Education in the UK and South Korea. Doctor of Philosophy Thesis, University of Birmingham, 2010.

[22] E. de Graaff, and A. Kolmos. Characteristics of Problem-Based Learning. International Journal of Engineering Education, vol. 19, no. 5, pp. 657-662, 2003.

[23] C. Liu, and C. Tsai. An analysis of peer interaction patterns as discoursed by on-line small group problem-solving activity. Computers & Education, vol. 50, no.3, pp. 627–639, 2008.