Can one version of online learning materials benefit all students?

(1)

Symposium Presentation

UniServe Science Proceedings Visualisation 152

Can one version of online learning materials benefit all

students?

Alexandra Yeung and Siegbert Schmid, School of Chemistry, University of Sydney, Australia Roy Tasker, School of Natural Sciences, University of Western Sydney, Australia

[email protected] [email protected] [email protected]

Introduction

Computers have had a significant impact on teaching and learning in recent years. When used as cognitive tools, computers can enable students to develop higher levels of cognitive processing by displaying information as both text and graphics to facilitate retention and transfer (Kozma 1987). For many students, chemistry is a subject that involves a novel set of terminology and symbology, and an array of abstract concepts and mental images not consistent with their observations and experiences of the world (Rusay 2003). Information and communications technology (ICT) offers the opportunity to help students develop understanding of these abstract concepts by illustrating them with multimedia simulations, thereby making them more concrete. ICT instruction can be reviewed multiple times, allowing the learner to control the pace of learning (Tissue, Earp and Yip 1996). Furthermore, students can access online pre-laboratory work at any time thereby allowing them flexibility whilst offering the university a cost effective means of delivery.

Chemistry and laboratory work

Traditionally chemistry courses at university level incorporated a significant amount of time spent in laboratory classes and it has been argued that laboratory work is an integral part of chemistry education (Johnstone and Al-Shuaili 2001). Laboratory work allows students to acquire technical skills such as handling glassware and equipment, which is a necessary preparation for many careers involving chemistry. At the same time, it also offers the chance for students to develop more ‘theoretical’ skills in areas of experimental design and method, observation and interpretation as well as analysis of results. These skills cannot always be gained through lectures and demonstrations alone (Abraham, Cracolice, Graves, Aldhamash, Klhega, Palma Gli and Varghese 1997). However, first year chemistry laboratories are a place where large amounts of new information are presented to students. It is unrealistic to expect first time, unprepared learners to be able to process all laboratory experiences with understanding, and indeed students with little or no prior knowledge find it difficult to derive meaning (Berry, Mulhall, Gunstone and Loughran 1999). As a consequence, instruction needs to take account of existing knowledge of students if integration is to be facilitated.

It has been shown that conceptual understanding developed prior to the laboratory session influences students’ ability to process information in the laboratory. Thus, it is desirable to have some kind of pre-work to prepare students for the laboratory session. The principal reason for using pre-work is that exposure to related theoretical concepts and experiments increases students’ learning and performance in the laboratory (Johnstone 1997). Furthermore, pre-work eases the transition into new experiments by allowing students to familiarise themselves with the experiment and gain a clearer understanding of what is expected of them (McKelvy 2000). Effective preparation also reduces anxiety while increasing student confidence. This has been shown in a study where students who used a virtual laboratory program were more positive about the program contributing to their confidence and reducing their anxiety in the lab (Dalgarno, Bishop, Bedgood Jnr and Adlong 2004). This produces productive and positive learning experiences (Koehler and Orvis 2003).

The aim of this study was to examine the influence of two online chemistry modules on academic performance in a laboratory titration assessment or final examination of first year chemistry students.

(2)

The effectiveness of the module as pre-laboratory work was investigated by determining whether students who completed the module performed better in a laboratory titration assessment than those who did not. The effectiveness of the modules as a learning tool was investigated by determining whether students who completed the modules performed better in the final examination. Particular attention was paid to the level of prior knowledge and whether the modules were suitable for all students.

Pre-laboratory work modules

Two online modules used in this study are from Bridging to the Lab: Media Connecting Chemistry

Concepts with Practice (Jones and Tasker 2002). Each module is set in the context of a real life

problem. Students make decisions regarding experimental design, observe simulations of reactions (both at macroscopic and molecular levels), record and interpret data, perform calculations, and draw conclusions from their results. Each module is divided into sections and students can only progress to the next screen once the learning activities have been completed correctly. Feedback is provided to assist students when they make mistakes and they are free to navigate backwards through past sections. A summary is provided at the end of each module, followed by a ‘self test’ where students can test what they have learned by applying their freshly gained knowledge to new situations.

The module, Concentration: Preparing a Standard Solution (hereafter called the standard solution module), requires familiarity with concepts of stoichiometry. Students ‘prepare’ a standard solution of a particular concentration, including selecting appropriate glassware. The module, Reaction Types:

Treatment of Copper(II) Waste (hereafter called the copper(II) waste module), is designed to develop

students’ qualitative understanding of redox and precipitation chemistry, as well as revising solubility rules. Students are required to make observations and draw conclusions.

Methodology

Participants

Three different groups of first year chemistry students participated in this study. All students were enrolled in one of the semester 1 chemistry units of study (UoS) available to students undertaking mainstream science qualifications. All three units cover similar material, but differ in the level of assumed prior knowledge, and the level at which material is presented. CHEM1001 (‘Fundamentals’) students have either not completed chemistry for the Higher School Certificate (HSC), i.e. university entry level, or achieved poor results at that level. CHEM1101 (‘Mainstream’) students have satisfactorily completed HSC chemistry, whilst CHEM1901 (‘Advanced’) students have achieved a HSC chemistry mark above 80. Students from all three groups were asked to complete both online pre-laboratory work modules in their own time. 720 (63%), 779 (68%) and 704 (62%) students completed the copper(II) waste, standard solution, or both modules, respectively. These numbers were much lower than for other online pre-work tasks.

Interviews

Sixteen students were interviewed using stimulated recall techniques (O’Brien 1993) with students from each UoS involved in the study. These interviews focused on students’ rating of the modules (clarity, usefulness) and learn about their experiences using the online pre laboratory work modules. The interviews were semi-structured and students were shown the modules on a computer screen to stimulate their memory, so they could recall their experiences during completion of the online modules as accurately as possible. Interviews were about 30 minutes in length and were videotaped, which allowed recording of the visual data on the computer screen that were used to stimulate recall.

(3)

Results and discussion

The results of two assessments were collected for data analysis – end of semester examination and a laboratory titration assessment. Examination marks ranged from 14 to 98 with a standard deviation of 14.9. Titration marks range from 3 (answer within 1% of correct value) to 0 (answer more than 5% wrong). There were large variations in titration marks across individual courses; however, student performance was higher in Mainstream than in Fundamentals and higher still in Advanced. The titration assessment is common to all UoS, but each UoS has a different end of semester examination. The effectiveness of the modules as pre-laboratory work

Effect on Advanced and Mainstream students

Independent samples t-tests showed that there was no significant difference in mean titration mark for Advanced students who completed the module and those who did not (t141=0.036, p=0.971). Independent sample t-tests also confirmed that there was no significant difference between the mean titration mark of Mainstream students who completed the module and those who did not (t601=-1.625,

p=0.105) (see Figure 1).

Effect on Fundamentals students

There is a more than 30 % increase in titration mark for Fundamentals students who completed the module compared with those who did not (see Figure 1). An independent sample t-test was conducted, which confirmed that this increase was not due to random chance and that the difference in mean titration marks for students who completed the module and those who did not (t354=-4.133,

p<0.001) was statistically significant.

Completion of the online standard solution module was not associated with improved titration marks for Advanced students and only a slight improvement for Mainstream students. Most of these students would have completed titration exercises prior to entering university, since a pre-requisite to enrol in their UoS is completion of HSC chemistry or equivalent and titrations are widely used in high schools. Given these students would be familiar with the techniques involved the additional pre-laboratory work would merely act as a revision exercise for those students. They would be unlikely to suffer working memory overload whether or not they complete the online module. Thus, these students would cope well with the laboratory work and be well trained for the assessment exercise.

For Fundamentals students, however, the outcome was very different. The performance differed dramatically for those students that completed the online module. Fundamentals students have a weak chemistry background and are unlikely to have much (if any) experience in titration technique. The standard solution module is closely related to the laboratory assessment. In addition to

1.53 1.95 2.35 2.02 2.10 2.34 0.0 0.5 1.0 1.5 2.0 2.5 3.0

Fundamentals Mainstream Advanced

M e an t it rat ion mark/3

Did not complete module Completed module

N=116 N=240 N=185 N=418 N=107 N=36

Figure 1. Mean titration marks for first year chemistry students (considering module completion)

(4)

facilitating understanding, the module exposes students to the techniques involved in preparing a standard solution, the type of equipment that is used and helping them to understand the various steps involved before going to the laboratory. Animations and simulations incorporated in the module allow students to have a visual representation of the techniques involved, such as quantitatively transferring the solution to the volumetric flask. When these students complete the module their prior knowledge improves by a large amount providing both a scaffold to which new information can be linked to and minimising working memory overload which otherwise may lead to information being discarded. Consequently these students are not confused by the many new concepts or techniques and can instead focus on getting the practical work done with minimal error.

A similar result was shown previously where students with a weak chemistry background performed to a level comparable to that of students who have a much better background and experimental ability after watching videos showing titration techniques (McNaught, Grant, Fritze, Barton, McTigue and Prosser 1995). In that study, all students participated in the study and self-selection was not an issue. In the present study, students encountered difficulties in accessing the modules on computers at university. Therefore completion rates were much lower than usual but do not necessarily indicate lower motivation of these students. Since both independent empirical studies show the same trend we conclude that motivation does not account for the difference in performance.

The improvement in the titration marks for Fundamentals students is even more significant, given the amount of preparation all students undergo as well as the fact that all Fundamentals students in semester 1, in the previous year (when no additional online module was offered) achieved a mean mark of 1.63 (292 students). The benefits of the module were apparent to a Mainstream student, Angela (pseudonym), who said: ‘I also think that it is particularly good for people who hadn’t done chemistry before. But because I have done it I found it relatively easy’. This comment indicates that the module may be of more benefit to Fundamentals students, as seen from the perspective of another student, and it also emphasises the fact that Mainstream students (and Advanced students by inference) have previously encountered the concepts presented in the modules and do not necessarily perform better in the titration assessment as a consequence of completing the module (this student got 3/3 marks for the titration assessment).

Effectiveness of the modules as a learning tool

In addition to acting as pre-laboratory work, the online modules can serve as a learning tool to supplement current student resources. The standard solution module is more of a technical module showing techniques required to prepare a standard solution, whereas the copper(II) waste module is more of a theoretical module covering many common chemistry concepts. The copper(II) waste module requires students to visualise substances on the macroscopic, symbolic and molecular levels. Both of these modules may have benefits for most students as a learning tool - the standard solution module in teaching students calculations that are used in chemistry and the copper(II) waste module on concepts such as precipitation and redox chemistry.

Effect on Advanced Students

As shown in Figure 2 completion of the modules does not appear to improve examination marks for first year Advanced students. One-way ANOVA was conducted to explore the impact of completing the online modules on final examination mark. Subjects were divided into treatment groups depending on the number of modules they had completed: no module, one module or both modules. There was no statistically significant difference in academic performance between the three groups [F2,144=0.821, p=0.442]. Post-hoc comparisons using the Tukey HSD test also did not indicate any significant difference in the mean marks between groups, i.e. Advanced students who completed the modules did not perform significantly better than those who did not.

(5)

Effect on Fundamentals and Mainstream Students

In contrast, the mean examination marks for Fundamentals and Mainstream students appear to be strongly correlated with the number of modules completed (see Figure 2). One-way ANOVA was conducted for each of Fundamentals and Mainstream chemistry to explore the impact of the online modules on final examination mark. Similar to the Advanced students, subjects were divided into treatment groups depending on the number of modules they had completed. There was a statistically significant difference in academic performance between the three treatment groups for Fundamentals [F2,374=12.025, p<0.001] and Mainstream students [F2,616=10.698, p<0.001]. Post-hoc comparisons using the Tukey HSD test indicated significant differences in the mean examination marks for both Fundamentals and Mainstream students who completed two modules compared with those who completed no modules. Fundamentals students who completed two modules performed better than students who did not complete any modules by an average of about eight marks, while the corresponding difference for Mainstream students was an average of about five marks. Average examination marks for students who completed one module only in every UOS have a much higher standard error of the mean owing to the small number of students in those groups.

While there was no difference in academic performance for Advanced students as a result of completing one or both online modules, Fundamentals and Mainstream students who completed both modules did perform significantly better than students in their respective units of study who did not complete the modules or only completed one. This result clearly demonstrates that the modules are associated with improved outcomes for first year chemistry Fundamentals and Mainstream students. These findings again show that the one version of the online chemistry modules is of benefit as a learning tool for certain groups of students but not all students, depending on their prior knowledge.

Natalie(pseudonym) comments ‘computers are a lot more fun and a lot more interactive. It gives you feedback straight away’. Using online modules not only provides an exciting way of presenting information by incorporating the use of simulations at the macroscopic level and offering visual representations, but it also makes abstract concrete more concrete and allows instant feedback to be given. This provides students with a scaffold that facilitates learning, and helps students develop a better understanding of material, which can explain the associated increased performance in the examination. The scaffolding provided could also be a way of developing students’ metacognitive abilities by requiring students to monitor their answers before selecting an alternative one based on the feedback given. This type of activity can assist students in self-regulation and control over their mental processes (Puntambekar 1995). Learners are able to explore and experiment, which consequently allows them to construct their own knowledge and mental pictures. James(pseudonym) says: ‘I really liked the interactions, the questions and how it took you through step by step and at the end of each of the sections it reinforced what you just learnt’. Instant feedback given to students may

51.2 54.9 70.9 54.7 55.2 65.1 59 60.3 71.8 40 45 50 55 60 65 70 75

Fundamentals Mainstream Advanced

Mean exam mark/100

0 1 2

N=195

N=128 N=34 N=215 N=386 N=38 N=39 N=5

(No. of modules completed)

N=103

Figure 2. Mean examination mark compared with the number of modules completed for each UoS

(6)

also prevent misconceptions from developing as corrections to their mental models can be made immediately. Furthermore, by making the modules more interesting, students will have a greater chance of being motivated, thus increasing learning.

Students are in control of their own learning as they are able to regulate the pace in which they complete the module. Davidψ enjoyed completing the modules because of ‘the simplicity. Even though it sounds pretty weird, but how it uses big font and big graphics, it makes you, I don’t know but it psychologically makes you think that it’s easy. So you would have a go at it… It just gives you the concise tiny bit of information in big writing that tells you there’s not much to know.’

It is clear that the modules have significant benefits for first year Fundamentals chemistry students who do not have a good background in chemistry. It seems likely that the inclusion of more modules of this type may be associated with even greater improvements in student performance, and should be carefully investigated for potential implementation. However, there were no observable benefits for Advanced students who have a strong chemistry background.

Conclusions

This study investigated the effectiveness of online modules for pre-laboratory work and as a learning tool. The performance of Fundamentals students, who have a comparatively weak background in chemistry, dramatically improved in a laboratory titration assessment closely related to the module. These students are least familiar with the concepts of titrations, i.e. their improved performance indicates that completing the module improves their prior knowledge in a way that provides an effective scaffold which new information can be linked to and thereby reduces working memory overload during the laboratory sessions. Since the increase in performance can be attributed directly to completing the standard solution module, the regular implementation of modules of this kind would be expected to further improve learning in chemistry, albeit for students with a weaker background.

The effectiveness of the online modules as a learning tool was also investigated. Advanced students did not benefit from completing these particular modules, however, Fundamentals and Mainstream students who completed both modules performed significantly better in the examination than those who did not.

This study has provided empirical evidence concerning future implementation of these or similar modules into the teaching program. Variations to the modules need to be made to benefit all students. For students with a stronger background modules need to be designed differently and targeted more towards areas with more difficult concepts rather than skills development, which is one of the main purposes of the titrations. When such changes are implemented, many more students will benefit. Acknowledgements

The authors would like to thank all University of Sydney students who agreed to participate in this study. Data collections described in this work were authorised by The University of Sydney Human Research Ethics Committee, project reference number 04-2004/7353.

References

Abraham, M.R., Cracolice, M.S., Graves, A.P., Aldhamash, A.H., Klhega, J.G., Palma Gli, J.G. and Varghese, V. (1997) The Nature and State of General Chemistry Laboratory Courses Offered by Colleges and Universities of the United States. Journal of Chemical Education, 74(5), 591–594.

Berry, A., Mulhall, P., Gunstone, R. and Loughran, J. (1999) Helping Students Learn from the Laboratory. Australian

(7)

Dalgarno, B., Bishop, A.G., Bedgood Jnr, D.R. and Adlong, W. (2004) What Factors Contribute to Students’ Confidence in Chemistry Laboratory Sessions and Does Preparation in a Virtual Laboratory Help? In Proceedings of the

Scholarly Inquiry into Science Teaching and Learning Symposium. Sydney, Australia: UniServe Science, 15–21.

Graesser, A., Wiemer-Hastings, K., Wiemer-Hastings, P. and Kruez, R. (2000) AutoTutor: A simulation of a human tutor. Journal of Cognitive Systems Research 1, 35–51.

Johnstone, A.H. (1997) Chemistry Teaching - Science or Alchemy. Journal of Chemical Education, 74(3), 262–268. Johnstone, A.H. and Al-Shuaili, A. (2001) Learning in the Laboratory; Some Thoughts from Literature. University

Chemistry Education, 5, 42–51.

Jones, L. and Tasker, R. (2002) Bridging to the Lab: Media Connecting Chemistry Concepts with Practice: W.H. Freeman and Company.

Koehler, B.P. and Orvis, J.N. (2003) Internet-Based Prelaboratory Tutorials and Computer-Based Probes in General Chemistry. Journal of Chemical Education, 80(6), 606–608.

Kozma, R. (1987) The Implications of Cognitive Psychology for Computer-Based Learning Tools. Educational

Technology, 27(11), 20–25.

McKelvy, G.M. (2000) Preparing for the Chemistry Laboratory: An Internet Presentation and Assessment Tool.

University Chemistry Education, 4(2), 46–49.

McNaught, C., Grant, H., Fritze, P., Barton, J., McTigue, P. and Prosser, R. (1995) The Effectiveness of Computer Assisted Learning in the Teaching of Quantitative Volumetric Analysis Skills in a First Year University Course.

Journal of Chemical Education, 72(11), 1003–1007.

O'Brien, J. (1993) Action Research Through Stimulated Recall. Research in Science Education, 23, 214–221.

Puntambekar, S. (1995). Helping Students Learn ‘How to Learn’ from Texts: Towards an ITS for Developing Metacognition. Instructional Science, 23(1-2), 163-182.

Rusay, R.J. (2003, unknown) Chemistry/Memory/Learning. Retrieved 29/5/04, from http://ep.llnl.gov/msds/ chem120/learning.html

Tissue, B.M., Earp, R.L. and Yip, C.W. (1996) Design and Student Use of World Wide Web-Based Prelab Exercises. The

Chemical Educator, 1(1), 1–13.

The authors assign to UniServe Science and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to UniServe Science to publish this document on the Web (prime sites and mirrors) and in printed form within the UniServe Science 2008 Conference proceedings. Any other usage is prohibited without the express permission of the authors UniServe Science reserved the right to undertake editorial changes in regard to formatting, length of paper and consistency.

Yeung, A., Schmid, S. and Tasker, R. (2008) Can one version of online learning materials benefit all students? In A. Hugman and K. Placing (Eds) Symposium Proceedings: Visualisation and Concept Development, UniServe Science, The University of Sydney, 152–158.