DOES OUR INITIAL TRAINING PROGRAM FOR PRIMARY TEACHERS REALLY WORK?
DESIGN, IMPLEMENTATION AND ANALYSIS OF A QUESTIONNAIRE
María Martínez Chico1, M. Rut Jiménez Liso1 and Rafael López-Gay2
1 University of Almería, Spain
2 High School Nicolás Salmeron y Alonso, Almería, Spain
Abstract: As teacher preparation programs contribute to the quality of instruction, the evaluation of teacher preparation programs can provide useful information for the improvement of teacher preparation and practice. Nevertheless, progress in research on science education contrasts with scarce number of articles and open forums existing about specific evaluation tools and methods to know the initial teacher training courses’
effectiveness. In this paper, we present the design and analysis of a Likert-type scale questionnaire used to measure how effective is a Primary teacher training course in changing preservice teachers’ conceptions about science and how it works, science teaching and learning. After validating and pilot testing, the questionnaire was
implemented and the statistical treatment of data collected through the questionnaire aimed to determine Reliability, Significant Differences and Effect size (which quantifies the observed differences between pre and post-test). Results show significant differences and moderate-large effect size were found between pre (N=199) and post-test (N=188), which seems to demonstrate our training program produce a progress in student teachers’
conceptions. Specifically, the highest effect size found corresponds to the category called
“Teacher and teaching” implying that the greatest conception change occurred in this category.
Keywords: Initial teacher training, conceptions change, evaluation of training proposals, questionnaire
INTRODUCTION
The current “scientific literacy” and “science for all” approaches require new teaching strategies focused on helping students to learn about scientific inquiry but also understand science concepts (Osborne & Dillon, 2008). Initial teachers’ training plays a fundamental role in achieving this change in sience education, however, the huge volume of research related to strategies for science teacher preparation in recent years (Schwarz, 2009), contrasts with the lack of information about what we are doing in initial teachers’ training and how we evaluate it. As the National Research Council committee charged with reviewing teacher preparation programs concluded, there is practically “no systematic information on the content or practices of preparation programs or requirements for science teachers” (NRC, 2011).
The situation is similar in Europe; while there is some research on what might be effective in preservice education little is known about what is actually offered. Moreover, methods used by trainers to evaluate their training courses and their effectiveness in achieving their objectives also seem to be scant. Often the maelstrom of daily preparation and
development of classes distracts from awareness of what our students are achieving and we are guided by perceptions such as students' enjoyment and not by evidence obtained from systematic evaluation.
Nevertheless, as teachers’ trainers, we should ask ourselves and be able to answer a question: Does our initial teacher training program really work? In order to answer, in this study we show the design, implementation and validation of a questionnaire designed to measure the effect of an initial training course on preservice teachers’ conceptions about science and about teaching and learning. New data and the tool which we add to the literature may be useful for understanding and comparing the effectiveness of other teachers’ training program.
RATIONALE
According to educational research, prospective teachers have very persistent attitudes, conceptions, etc. about science and how science works, science teaching and learning that joined to their content knowledge can work as obstacles for an effective change in science teaching (Tobin & Espinet, 1989). Therefore, to overcome these obstacles, teachers’
training and its evaluation should be focused on, among other things, changing pre-service teachers’ conceptions (Porlán & Martin del Pozo, 2004), which requires paying attention to the following issues:
- Science content knowledge, helping prospective teachers to be able to explain natural phenomena and support the use of models in Primary Education;
- conceptions about science, how inquiry works, and justification of science teaching, questioning the preparatory purpose of science education or reductionist
conceptions about science, and promoting “scientific literacy for all” approaches;
- conceptions about learning, overcoming simplistic views and moving towards the social constructivist conception;
- and conceptions about science teaching and teacher’ role, questioning a model focused on subject content and teacher's explanations, and promoting a student- centered education.
Given these dimensions, we have designed an initial training program focus on a modeling-centered inquiry approach, (Martínez-Chico et al., in press). This instruction approach allows us to work on the dimensions mentioned, engaging learners in scientific inquiry whose focus is on the construction and revision of scientific models that can be applied to understand and predict natural phenomena (Schwarz, 2009). To evaluate the training program a questionnaire obeying a pre-experimental design pretest-instruction- posttest, has been designed, implemented, validated and analyzed, considering the conceptions’ change outlined above.
METHODS
Questionnaire development and validation
To discover the degree of impact of our program on changing preservice teachers’
conceptions, we designed and developed a Likert-scale questionnaire (1=Strongly disagree; 2=Disagree; 3=Neither agree nor disagree; 4=Agree;5=Strongly agree), according to these categories:
a) Science, scientific activity and justification of science education b) Students and Learning
c) Teacher, teaching and assessment
For each category, ‘big ideas’ were chosen from science education literature. 46 items were written up, a positive-negative pair for each main idea, which were selected randomly to develop two questionnaires, with the aim of checking the consistency in participants’
responses.
Both questionnaires, applied with forty minutes apart to avoid comparison between positive and negative items, were tested with a pilot sample of Primary School preservice teachers (20) with the purpose of modifying items or not. After checking consistency in each participant’ responses (individuals who responded to 15% items inconsistently were excluded), a criterion for changing items was established: if more than 20% of the sample responses on an item were contradictory, it must be rewritten. Only 2 items were changed.
The final questionnaires can be found at:
https://www.dropbox.com/s/7msv2e0d4exk74k/TEST.docx.
Sample
The sample comprises students in the 2nd academic year of Primary School Teacher’s Training Degree in the University of Almeria (divided into 4 groups). They experienced the training program we designed (whose effect on student teachers’ conceptions we intend to measure).
The pre-test was administered to 199 students in the first session of the science education program after explaining the purpose of our research briefly. The post-test was
administered to 188 students in the last session, following the same application procedure.
RESULTS AND ANALYSIS
The statistical treatment of data collected through the questionnaire aimed to determine Reliability, Significant Differences and Effect size.
Table 1.
Statistical analysis Reliability
Ability to demonstrate stability or consistency of scores
Cronbach’s Alpha coefficient has been calculated, obtaining a value of α= 0.851
Acceptable values are those above 0.70 Significant Differences
between pre and post-test Degree of certainty that there are differences rather than just chance
Nonparametric tests have been performed (as the data do not conform normality), obtaining significant differences in every item
Effect size
Quantifies the observed difference between pre and post-test
It was calculated using the Wilcoxon signed-rank test (a non-parametric statistical hypothesis test used when comparing two related samples) obtaining a substantial effect in 83% of items According to research on educational innovation,
effect size values upper than 0.15 are considered relevant (Valentine & Cooper, 2003)
Figure 1 shows the average values of students’ responses in pre and post-test for each item.
On the x-axis (in brackets) the effect size of each item is shown, for example, the highest effect size found corresponds to item B8 with a 0.76 value. This is an item related to the order of teaching contents: it should be from the most concrete and closest contents to the most general and abstract ones.
We used this parameter (the effect size) because, in addition to ensuring the existence of significant differences, we think it is very important for the validation of our training program not only to know whether differences are statistically significant but also to know their size. Moreover the average values of the effect size have also been calculated as an indicator of the training course’ effect on each general dimension. They are shown in the upper area of the graph (figure 1), the highest value being 0.41 which is related to
“Teacher and teaching” implying that the greatest conception change occurred in this category.
Figure 1. Difference between pre testing and post testing.
Another item with a very high effect size (0.51) is related to the view of science and scientific work, and it refers to the importance of communicating ideas in the process of generating scientific knowledge (item B22N).
Regarding to the category Students and learning, the idea "children have their own explanations about phenomena before learning the scientific explanations" established in the item A5 refers to the need to question and change the naive views about science
learning. In this case, we also obtained good results as the high effect size found (0.53) indicates.
Furthermore, two items in which the effect size has been very high too, are included in the category Teacher and teaching: “Children explanations are persistent and do not change because the scientific explanations are shown clearly and orderly” (A1), so they should do scientific activities in class: as engage in problems, formulate and justify their own hypotheses, look for evidence…” (B6). In these ideas related to questioning traditional conceptions about teaching and the teacher’s role we found a effect size of 0.56 and 0.52 respectively.
CONCLUSIONS
We present a questionnaire for measuring the effectiveness of an initial teachers’ training course (following a model-based inquiry approach) in changing preservice teachers’
conceptions. After being developed and validated, the questionnaire was implemented with a large group of preservice Primary teachers.
The analysis undertaken suggest that our training program was effective of, taking into account that pre-test responses were already positive, the course lasted only 40 hours and that modest effect size values are significant in educative innovation.
We have also extracted some evidence of our training program working. Concretely a comparison of pre and post-test results reveals significant differences in 22 of the 23 ideas and high-very high effect size, what means an improvement in the students’ responses:
more agreement in items expressed in positive sense, and more disagreement in items expressed in negative sense.
It strongly suggests a progress in student teachers’ conceptions about science, science teaching and learning, along every item; being the highest average value of the effect size calculated in the category “Teacher and teaching”, what implies that the greatest progress in preservice teachers' conceptions occurred in this dimension.
Despite the good results obtained with the information that the questionnaire allowed us to collect on students' conceptions, we have to go deeper in our study oriented to evaluate the effectiveness of the initial teacher training.
Until now, at best, we can only provide evidence of an instrument’s efficacy in measuring what it is designed to measure, in our case, conceptions. Therefore, these results will be triangulated with information collected through tools, in order to achieve a more complete evaluation of our teachers’ training program. With the aim of performing that more complete evaluation of the training course' effectiveness in changing conceptions, we will analyze also qualitative data collected by interviews with students, tasks performed by students and online forums developed along the course.
REFERENCES
Martínez-Chico, M., López-Gay, R. & Jiménez-Liso, R. (2013). Design and implementation of an initial primary teachers training course through model-based inquiry. In Proceedings of The international Conference New Perspectives in Science Education (Florence, Italy, 8 -9 March 2012). Florence. http://conference.pixel- online.net/npse2013/conferenceproceedings.php
National Research Council (2011). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Committee on a Conceptual Framework for New K-12 Science Education Standards. Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
Osborne, J. & Dillon, J. (2008). Science Education in Europe: Critical Reflections. A Report to the Nuffield Foundation. Available from:
http://www.nuffieldfoundation.org/sites/default/files/Sci_Ed_in_Europe_Report_Final .pdf (accessed 3rd of February 2011)
Porlán, R., & Martin del Pozo, R. (2004) Conceptions about the teaching and learning of science. Journal of Science Teacher Education, 15(1), 39–62.
Schwarz, C.V. (2009). Developing preservice elementary teachers’ knowledge and practices through modeling-centered scientific inquiry. Science Education, 93(4), 720- 744.
Tobin, K. & Espinet, M. (1989) Impediments to change: application of coaching in high school science teaching. Journal of Research in Science Teaching. 26 (2), 1989, 105- 120.
Valentine, J. C. & Cooper, H. (2003). Effect size substantive interpretation guidelines:
Issues in the interpretation of effect sizes. Washington, DC: What Works Clearinghouse.
