LEARNING TO IMPLEMENT SCHOOL EXPERIMENTS IN A
BLENDED LEARNING APPROACH: AN EVALUATION
STUDY
Thorid Rabe1, Olaf Krey1and Franco Rau1 1University of Potsdam, Germany
Abstract: This paper reports about the redesign of a traditional course on school experiments in physics that is part of the teacher education program at the University of Potsdam. A blended learning approach was chosen to improve the course in the sense of (1) a stronger focus on aspects of pedagogical content knowledge related to experiments, (2) an increase of time on task, and (3) fostering communication processes. The face-to-face lab course existent before was combined with online activities implemented by means of a mixxt-community. The implementation of the blended learning scenario was accompanied by an evaluation study that aimed at surveying students` perception of their gains in competencies and of the communication processes related to the course. The results presented in this paper are based on data collected with a questionnaire consisting of scales on perceived increase in subject matter knowledge, in experimental competencies, in nature of science knowledge and in aspects of pedagogical content knowledge. Results indicate that students appreciate the course in total, because it facilitates increase of competencies in all facets that were surveyed by the questionnaire. Yet, at the same time, students rate the influence of the online activities as put into practice in this course as rather low. The time on task effect shows up to be relevant in particular for the acceptance of the online activities. Consequences for an improved blended learning design and further evaluation are discussed.
Keywords:blended learning, hybrid course, school experiments, teacher education, evaluation study
BACKGROUND, FRAMEWORK, PURPOSE
The University of Potsdam holds a strong tradition in physics teacher education. As an important part of their studies in physics education students have to take a course in which they learn to conduct and to stage school experiments in the domain of physics. Experiences with this course - being a mixture of seminar and laboratory work - were twofold, so far. On the one hand, students are enthusiastic and motivated while conducting the experiments. They usually increase their abilities in staging and conducting experiments indeed. On the other hand, the authors found students showing up unprepared and struggling with content knowledge in physics, which we assumed to be already understood by the students. Hence, students paid little attention to topics related to pedagogical content knowledge in the field of school experiments (PCK) (Shulman, 1986) and did not develop the competencies we intended.
For improvement a blended learning approach was chosen (Bonk & Graham, 2005), sometimes also referred to as a hybrid course (Garnham & Kaleta, 2002). By definition “Blended learning arrangements combine technology based learning with face-to-face learning” (Kerres and de Witt, 2003). General expectations associated with blended learning relate to the improvement of students’ learning outcome and of quality of teaching. Access to
learning opportunities and flexibility of learning are supposed to be increased while costs are reduced (Graham 2005).
The intention for our course was more specific and threefold. We aspired (1) to increase students' time on task, (2) to establish a continuous learning process and (3) to give additional opportunities for communication about experiment related PCK. One of the competencies students should have developed as a result of the course was to give reasons, objectives and purposes for using a certain experiment in a specific classroom context. Evidently, these are reasonable objectives or even key competencies for future physics teachers which have received quite some research attention within the science education community (e.g. Hart et al., 2000; Jacobsen, 2008 and the review article Lavonen et al., 2004).
To support our objectives, we used a social community (www.mixxt.com) as a supplement to our lab classes which, obviously, required physical attendance. The structure of the resulting hybrid course is shown in figure 1. All online activities are marked with an asterisk (*). The face-to-face course continued to focus on practical aspects of preparing and staging experiments for school physics. Direct interaction between students as well as between students and instructors took place here. The online activities in preparation of the laboratory work consisted of tasks to be worked out by the students. E.g. the students had to report on relevant pupils’ misconceptions in the content area, they had to specify learning goals related to their experiments and they were asked to exemplify one experiment in greater detail. Furthermore, students were encouraged to comment on the work of the others in the sense of a feedback culture. The instructors of the lab course were asked to give online feedback as well and to give advice for improvement of the work done.
It was pointed out to the students that the online activities played an important role in the preparation of the face-to-face course and the assessment at the end of course. Anyhow, participation was voluntary. See figure 2 for an exemplary screenshot taken from our mixxt community.
Figure 1: Structure of the blended learning scenario implemented in a lab course on school experiments in physics
The implementation of the blended learning scenario was accompanied by an evaluation study to gain an insight into the outcome of the design. Our interest
-besides aspects of
practicality or convenience of the hybrid format -concerned the students’ acquisition of competencies. In particular, we wanted to learn about students’ competencies in the subject matter and experimentation, their pedagogical content knowledge with reference to school experiments and knowledge of the nature of science as a result of the
newly added online
activities.
Due to the small number of students and reasons of fairness it was not possible to conduct a comparative study between a course with the blended learning structure and a course in the
traditional format.
Additionally, at that point of time no reliable instruments for direct measurement of the competencies mentioned before were available. Therefore, we decided to survey the increase of competencies as perceived by the students on the basis of overall course on one hand and cause by the online activities on the other hand.
METHODS
A multi method approach was chosen to evaluate our course. Observations were conducted by one of the authors not directly involved in the teaching of the face-to-face course. Qualitative data from interviews with the teaching staff member and students were collected. Furthermore detailed protocols of students' and teachers' online activities are available (Rau et al., 2012).
Last but not least, a questionnaire including open ended questions and closed likert scale items (4-level, 0=no agreement, 1= little agreement, 2= some agreement, 3=full agreement) was employed. It is mainly the quantitative data resulting from this questionnaire that is presented and discussed in this paper. Scales were designed for our special evaluation interests as mentioned above. The factorial structure (main component extraction, promax rotation) and reliability of the chosen constructs was analysed and scales were identified. These scales were, then, used for analyses of correlation and variance. Our study included 19 teacher students (6 females), which explains why the final scales consist of very few items. As a first result, we were able to identify the scales shown in table 1 (together with their reliability coefficients (α), mean values (M) and standard deviations (SD). Moreover the questionnaire collected some personal data of the students and the estimated time they spent on online activities per week.
Scale caused by overallcourse caused by onlineactivities SMC
perceived increase in subject matter competencies (4 items)
e.g. “I do understand the topics of physics.” α=.74M=2.12, SD=0.41 α=.80M=.75, SD=0.54
ExpC
perceived increase in experimental competencies (3 items)
e.g. “I can show specific phenomena by means of
experiments.” α=.66M=1.81, SD=0.37 α=.95M=0.36, SD=0.53
NoSK
perceived increase in nature of science knowledge (3 items)
e.g. “I can explain the interrelation between models,
experiments und laws in physics.” α=.71M=1.95, SD=0.51 α=.88M=0.60, SD=0.61
PCK
perceived increase in experiment related pedagogical content knowledge (4 items)
e.g. “I can anticipate problems in the realization of a
certain experiment.” α=.67M=1.80, SD=0.55 α=.76M=0.61, SD=0.48
CommPA
perceived positive influence of mixxt on communication during physical attendance (5 items)
e.g. “The online activities fostered the interaction with
the instructor.” α=.75M=1.13, SD=0.57
CommOn
perceived positive influence of mixxt on online communication (3 items)
“e.g. “The online activities encouraged the subject
related interaction between students.” α=.76M=1.11, SD=0.64
MixxtC
competence using the mixxt community (4 items)
e.g. “The use of the online platform was easy to learn.” α=.80M=1.88, SD=0.63 Table 1: Statistics of Scales
RESULTS
The handling of the mixxt community was not much of a problem for our students, as the mean value (see table 1, MixxtC) indicates. In contrast to our intention, the online activities were not perceived as very supportive for communication processes – neither online nor during the on-site time (see table 1, CommPA, CommOn).
The highest perceived increase in competencies was found for subject matter competencies (see table 1, SMC). This effect is independent from students’ prior knowledge in experimental physics as an ANOVA (students have been assigned to a low, medium and high achieving group based on their grades in experimental physics) shows (F(2;16)=1.06, p=.369). This result can be explained with reference to what Riese (2010) found, namely that school related content knowledge is central for successful teaching but does not play an adequate role in physics teacher education. The perceived influence of the online activities on this increase in subject matter competencies is not a strong one, but it is existing independently from prior knowledge (F=(2,16)=.68, p=.51) (see table 1, SMC). A corresponding result can be found for the perceived increase in experimental competencies due to overall course (F(2,16)=2.96, p=.081) and due to the influence of mixxt (F(2,16)=1.27, p=.308) (table 1, ExpC).
Further, students perceived an increase of their NoS knowledge and - as we intended - also their experiment related pedagogical content knowledge (table 1, NoSK). Here again, the overall increase as well as the impact of the mixxt community activities are perceived, but not very strongly. As one would expect based on the literature (Kunter et al., 2007), subject matter proficiency (SMC), NoSK and PCK influence are highly correlated with each other, at least -and this is somewhat remarkable - for the part that was attributed to the online activities: r(SMC,PCK)=.48*, r(SMC, NoSK)=.71**. The same applies to reported gains in subject matter competencies and the experimental competencies: r(SMC, ExpC)=.68**.
Of little surprise is the fact that the time on task – as estimated by the students themselves -provides a good predictor for the perceived increase in subject matter competencies. An ANOVA shows that the perceived increase is different (strong effect) for two contrast groups (split half): While students in group 1 investing M=3.33 hours per week (SD=.78) into course related activities on average perceived a reasonable increase (M=1.94 SD=.32), students in group 2 in average investing M=6.57 hours per week (SD=2.94) perceived a higher increase (M=2.43, SD=.37). This difference is statistically substantiated (F(1;17)=9.15, p=.008**, ²=.35). The same tendency can be found for the increase that was perceived due to online activities (F(1;17)=7.67, p=.013**, ²=.31). This also allows the conclusion that a positive effect of mixxt community activities is dependent on some dedication to that activity.
CONCLUSION
Students appreciate the course as being successful in offering learning opportunities and state a slight, but existent impact of online activities on the increase of competencies. Against our expectations, the influence of the blended learning approach was rather small or the questionnaire was not a commensurable instrument for measurement. Time on task was enhanced for at least some of the students. This leads us to generally following up the chosen approach. Still there is room and need for further improvement concerning the following aspects. We have to find ways of activating more students to take part in online activities to a full extent and on a deeper level of reflection. A stronger linkage between online activities and face-to-face communication could help to reach the full potential of a social community in the above described sense (continuity of the learning process and opportunity for communication). At first glance, qualitative data suggest that the (online) activity of the
teacher trainer might be a crucial factor to foster students' activity as well. Therefore, it seems promising to train and brief the teacher trainers, to tie together the parts of the hybrid course and to support the continuity of online activities of students. Furthermore our instruments should be refined and supplemented e.g. by a scale measuring the motivational impact of mixxt activities. A design including control groups could help to objectify the subjectively perceived increase in competencies and direct methods of measurement of competency gains should be considered.
FUTURE PROSPECTS
The first insight into the implementation of a blended learning scenario in the teacher education program as presented in this paper is encouraging enough to follow up the chosen approach. Still, some lessons have been learned.
Concerning the technical realization we will substitute the mixxt community by wikis. We chose wikis, for substantial reasons as well as pragmatic ones. First of all our improved course design changes from voluntary documentation, reflection and discussion (for which the blog tool of mixxt.org was fitting well) to obligatory tasks that are constructed to prompt an in-depth study of certain aspects that play an important role in the process of preparing, conducting and reflecting a school experiment. The product of our students' learning effort is a wiki page which looks at a certain experiment from many perspectives and which has been improved and refined many times before it is finalized. The wiki based implementation also allows building up a database of physics school experiments. Finally the administrative effort (account setup, user and group management) is reduced.
To increase the students’ engagement in online activities, and thereby, the time on task, a minimum of online activity was defined as a requirement for lab course attendance. Tasks for the students are specified and the format and complexity of the expected answers are clarified to the students. A joint project of all students in the course, the development of a database for school experiments in physics, is meant to increase the collaborative character of the online tasks. Future instructors of the lab course will be trained in attending the online activities in a supportive way. Here, an essential issue arises in that not only students’ time on task is enlarged but also teachers’ “time on work” is increased.
References
Bonk, C., & Graham, C. (2005). Handbook of blended learning: Global perspectives, local designs. San Francisco, CA: Pfeiffer Publishing.
Garnham, C., & Kaleta, R. (2002, March). Introduction to hybrid courses. Teaching with Technology Today, 8
(10).
Graham, C. R. (2005). Blended learning systems: Definition, current trends, and future directions. In C. J. Bonk & C.R. Graham (Eds.),Handbook of Blended Learning: Global perspectives, local designs. San Francisco, CA: Pfeiffer Publishing.
Hart, C., Mulhall, P., Berry, A., Loughran, J., & Gunstone, R. (2000). What is the Purpose of this Experiment? Or Can Students Learn Something from Doing Experiments? Journal or Research in Science Teaching, 37(7), 655-675.
Jacobsen, L. B. (2008). Investigating the Need for Teachers and Students to understand the Purpose of Experimental Work in Physics. In The 9th Nordic Research Symposium on Science Education: Symposium program and abstracts. red. / Kristján Ketill Stefánsson. Stofnun Árna Magnúrssonar á Íslandi, 2008, 87 Kerres, M., & De Witt, C. (2003). A Didactical Framework for the Design of Blended Learning Arrangements.
Learning, Media and Technology,28(2-3), 101-113. doi:10.1080/1358165032000165653
project. In M. Prenzel (Ed.), Studies on the educational quality of schools. The final report on the DFG Priority Programme (pp.32-52). Münster: Waxmann.
Lavonen, J., Jauhiainen, J., Koponen, I. T., & Kurki-Suonio, K. (2004). Effect of a long-term in-service training program on teachers’ beliefs about the role of experiments in physics education. International Journal of Science Education, 26(3), 309-328.
Rau, F., Krey, O., & Rabe, T. (2012). Experimente für den Physikunterricht begründet einsetzen lernen - ein „hybrid course“. In S. Bernholt (Ed.),Jahrestagung der GDCP in Oldenburg 2011. Münster: LIT Verlag. Riese, J. (2010). Empirische Erkenntnisse zur Wirksamkeit der universitären Lehrerbildung,PhyDid (9)1, 25-33. Shulman, L.S (1986). Those who understand: Knowledge growth in teaching.Educational Researcher,15(2),
4-14.
van Dijk, E. M., & Kattmann, U. (2006). A research model for the study of science teachers’ PCK and improving teacher education.Teaching and Teacher Education, 23, 885-897.
