The aim of this study was to explore the current state of Grades 4 to 8 science education in Ontario. Specifically, this study was designed to explore the attitudes and experiences that J/I teachers bring to their science teaching practices. Teaching strategies being implemented, available resources, and time allotment for science instruction were also examined.
This chapter describes the methodological approach to the study. A two-phased mixed methods research study was conducted that integrated both quantitative and qualitative data. The instruments for data collection consisted of an online survey (that collected quantitative and qualitative data) and follow-up interviews (that collected qualitative data). A mixed methods research design was selected as most appropriate to answer the research questions, which involved collecting quantitative descriptive and narrative data. Casting a wide net for data collection was done intentionally to elicit a large sample size allowing maximum variation across participants. Data collection in this explanatory study aimed to describe the status of various phenomena currently impacting the effectiveness of elementary science education in Ontario.
Research Context: Elementary Science Education in Ontario
The range of Grades 4 to 8 is considered to be a critical time to foster students’ intrinsic interest in science and to build their self-confidence (Bressler & Bodzin, 2013; Fitzgerald et al., 2013). Positive and enriching experiences with science from an early age enable students to see themselves as pursuing science-related careers and/or involved with science (Bressler & Bodzin, 2013). For students to meet this potential they need teachers who can effectively facilitate and scaffold the learning of science content and
skills. Investigative research has identified teachers’ academic background and personal experiences with science as playing a major role in preparing them to deliver quality science instruction (Appleton & Kindt, 2002; Bulunuz & Jarrett, 2010). Additionally, factors such as resources, funding, time allotment, and most importantly, teachers’ attitudes and self-efficacy influence the effectiveness of science education, and ultimately, student success (Clark & Linn, 2003; Morrison, 2013; Thomson & Gregory, 2013).
Research Questions
The research design used both an online survey that collected both quantitative and qualitative data as well as follow-up interviews. This methodology is well-suited to address the research questions outlined below because it allowed the complementary strengths of both qualitative and quantitative data collection to account for large-scale survey and small-scale interview input (Patton, 1990). Both quantitative and qualitative data sets were used to uncover the specific factors that are currently influencing the state of Grades 4 to 8 science education in Ontario from the perspective of J/I teachers. The five research questions that guided this study were:
What attitudes do J/I teachers have toward teaching science?
What science-related academic and professional experiences do J/I science teachers have?
What teaching strategies are J/I science teachers currently using to teach science?
What support resources are available to teach science in elementary schools?
How much instructional time is allocated for science in elementary schools? These research questions elicited data that were analyzed and used to provide insight into the level of preparedness and the amount and variety of resources that J/I science teachers
have to teach science. Additionally, these questions were designed to prompt teachers to reflect on the supports and limitations involved in their current science teaching practice.
Research Design
The use of a mixed-methods research design involves reporting on a research problem through the collection and analysis of both quantitative and qualitative data (Creswell, 2012). A “concurrent triangulation method design” as described by Creswell, Plano Clark, Gutmann, and Hanson (2003) was used for process of data collection, analysis, and presentation of results. This method involved a triangulation of quantitative and qualitative data collection, separate data analysis, and an integration of the two throughout the interpretation and discussion stages of this report. In the case of this study, using a mixed-methods design allowed for the weaving of various strengths and
weaknesses of quantitative data collection (i.e., large N, trends, generalized themes) with those of qualitative data collection (i.e., small n, details, comprehensive) (Patton, 1990).
This study followed a two-phased explanatory design model described by Creswell and Plano Clark (2007). The notation of this study can be written as QUAN (qual) qual. These methods were used in sequence beginning with an online survey (N = 219) that collected predominantly quantitative data with some qualitative data (open-ended questions), followed by semi-structured individual interviews with participants (n = 6) purposively sampled from survey participant pool. Using this model to guide the research design enabled the researcher to gather initial insight into the current state of elementary science education in Ontario and then complement that information with detailed
personal accounts from the experiences of individual teachers that allowed for greater depth of investigation than if only one type of data had been collected.
Instrument Development
This section provides a brief rationale for the development of both the survey and interview tools used to collect data for this study.
Survey. The development of the survey involved a review of several surveys used in large-scale research to collect data about science teachers and science instruction in the United States. Several questions used in the survey for this study were modified from the 2004 and 2007 editions of Horizon’s Pan-Canadian Assessment Program (PCAP)
Teacher Questionnaire (CMEC, 2004, 2007) as well as the 2012 National Survey of Science and Mathematics Education Science Teacher Questionnaire (Horizon Research, 2012). Other questions were developed by the researcher to elicit information specific to the research questions or to the educational context in Ontario. The survey questions were imputed into the SurveyMonkey software. The survey was comprised of 39 questions broken down into four sections requiring approximately 20-30 minutes to complete. A final question asked if participants would be willing to participate in a follow-up interview.
Prior to data collection, a critique of the survey questions was conducted. The purpose of this portion of the survey critique was to determine estimated completion time, confirm clarity of question wording for participants, ensure the question order was logical, and ascertain whether all of the questions applied reasonably to the intended participant population of Ontario teachers. One principal, two full-time teachers, and two occasional teachers known to the researcher reviewed and provided feedback for this tool. The researcher’s supervisor and committee also reviewed and provided feedback on this tool. This process resulted in a few redundant questions being removed, altering the
wording of a few questions for clarity, and changing the Likert scale options on a couple questions.
The second portion of the critique was conducted with the purpose of ensuring that the online survey interface worked properly based on question order and participant response (question logic). This portion of the critique involved three full-time teachers and two occasional teachers completing a mock version of the survey online. This process resulted in several minor text edits, and a couple changes to the response question logic.
Interview. The interview protocol included 19 questions and was designed as an extension of the survey. The content questions outlined in the interview protocol follow the same themes as the survey (e.g., teacher attitudes and experiences, teaching strategies, resources, and time, etc.) with the intention of collecting data of greater detail from participants regarding their personal stories and experiences. The interview was semistructured and was conducted over the phone for easier participant access.
The interview protocol refinement process was conducted to test the interview questions for clarity, order logic, and the degree to which they would elicit the intended type of answer. The interview questions were sent to five people including one principal, two full-time teachers, and two occasional teachers with request for feedback and then reviewed by the researcher’s supervisor and committee. The result was that a couple questions were removed, and two questions were re-written for clarity. Lastly, a pilot interview was conducted with a colleague of the researcher with a similar academic background. This interview was conducted to ensure clear recording technique, expected interview length, appropriateness of question order, and clarity of questions, and to increase the overall validity of the research tool. The result of this portion of the pilot was
a couple questions being broken into several smaller questions and one question was made more specific.
Data Collection
As the investigation involved human participants, ethical clearance from the degree-granting university was required prior to beginning this research. The researcher decided to independently distribute the survey rather than involve specific school boards as gatekeepers so that participants from a wider geographical range in Ontario could participate. Both STAO and Youth University are educational organizations in Ontario that supported the distribution of the survey by posting the link on their websites and/or sending the link and information to their listserv of Ontario teachers. An outline of the methodology, potential risks and benefits of participating in the study for participants, measures taken to reduce any risks and maintain anonymity of participants, and a list of educational organizations that would act as gatekeepers was submitted to the Research Ethics Board (REB). Additionally, both the survey and interview question sets, as well as anticipated correspondence between the researcher and future participants were submitted for review. The ethical clearance for this study was modified once in order to include a second educational organization who sent the survey link out to their listserv. Research Ethics clearance (file # 14 – 046 - FAZIO) was obtained in September 2014. Information about the study and the link to the online survey were sent to the approved educational organizations for data collection to commence at the beginning of October 2014. Survey Data
The survey was designed to elucidate both the demographic data as well as data that will describe the attitudes and teaching strategies used by J/I science teachers.
Nominal and ordinal data were collected in addition to textual data from the open-ended response questions. Survey questions can be seen in Appendix A. Several opportunities for additional comments were provided throughout the survey. These qualitative data provided the researcher with additional detail to support and expand on quantitative findings. The survey questions were designed to gather information that helped the researcher to explore the experiences, attitudes, perceptions, and feelings of preparedness that J/I teachers in Ontario have toward teaching science.
The final question of the survey asked participants whether they would be interested in being contacted to participate in a follow-up interview. Initial analysis of both the quantitative and qualitative data collected from the online survey served as a support for the progressive development of the semistructured interview protocol. Quantitative data analysis procedures are discussed in a following section. Interview Data
Participants selected for a follow-up interview were contacted via email to establish an agreeable date and time for an interview. To allow for a larger geographical range of interview participants, and to eliminate travel time and location bookings, interviews were conducted over the phone. The interview required approximately 30 minutes to 1 hour for completion. The interview process followed a semistructured format, which began by requesting the same type of demographic information as in the survey. An interview protocol involving the 19 open-ended interview questions (as seen in Appendix B) was used to ensure a moderately consistent format with question asking and anecdotal note taking. These questions were developed to elicit responses that expand on and provide additional detail through personal accounts to the data collected
from the survey. Using open-ended questions allowed for participants to freely express their perceptions, lived experiences in schools, and their attitudes toward teaching
science. These responses provided contextual evidence that either aligned or contradicted with the data obtained from the survey. Each interview was audio recorded and
transcribed by the researcher. Anecdotal notes were made throughout each interview with the purpose of keeping note of specific details or additional points that provided notable information. Audio recording files and transcription files were stored electronically and backed up. Transcriptions were member checked by each respective interviewee. Participants
Participants in this study can be described as J/I teachers who currently teach science to students in Grades 4 to 8 in Ontario. This is a demographically diverse population as teachers from school boards across Ontario represent rural, urban, and suburban areas, in low, medium, and high socioeconomic areas. It is due to this diversity that casting a wide net across all of Ontario was necessary by sending out survey links and posting the link in virtual locations made the survey accessible to a wide range of Ontario teachers. Access to participants for this study was facilitated through
organizations like the Science Teachers’ Association of Ontario (STAO) and Youth University at Brock University, online Ontario teacher forums, and snowballing.
The chance to win one of four $25 Chapters gift cards was used as an incentive for complete survey participation in the online survey. Participants who wished to participate in the draw were asked to leave their emails at the end of the survey. The emails of the participants (n = 162) wanting to be entered into the draw were assigned a number (1 to 162). A random data-generating tool was then used to select the four
participants who would win a gift card. It was stated in the consent form that completion of the survey was the only requirement to have a chance to win the gift card that that indication of willingness to participate in a follow-up interview would not impact the results. Additionally, it was noted that participant emails for this question would be confidentially destroyed by June 2015.
Survey participants. Participation in this study was voluntary. Advertisement for participation indicated preference for elementary teachers who teach science at the
Grades 4 to 8 level. A total of 249 people responded to the survey. Participants who indicated that they taught either below or above the Grade 4 to 8 range or who did not complete more than 20% of the survey were removed from the data set prior to data analysis. Two-hundred and nineteen survey participant data sets were used for analysis. Upon clicking on the survey link potential participants would have to agree to the terms included on the consent form prior to having access to the survey questions.
Interview participant selection. Participants who completed the survey had the option to indicate willingness to be contacted for a follow-up interview. Of the 219 participants who completed the survey and met the required conditions outlined above, 55 (30.73%) said that they would be willing to participate in a follow-up interview; however, only 46 left their emails and only 36 had also completed the survey to a satisfactory degree. Participants were purposively selected with maximum variation sampling in order to collect data from participants representing the widest range of chosen characteristics (Creswell, 2013). The participant selection process was dependent on a spectrum developed by the researcher to represent a range of lowest to highest scores relating to the participants’ use of various instructional resources (Appendix A –
Q. 24) and reported frequency of various science instructional strategies (Q. 28). Both of these questions involved a frequency scale with responses that were scored from 1 to 5, with 1 being never and 5 being always or very frequently. Both of these questions were comprised of multiple components. The score for each component of each question was used for the score calculation. The aggregate score for each potential interview
participant was determined based on these scales. Higher frequency of resource use and higher frequency of instructional strategy use resulted in a high score. Potential interview participants were then ordered from lowest to highest total score creating a spectrum that was then divided into three groups of 15 or 16 people to correspond with low, medium, and high scores. This categorization was done to indicate teachers who had a low,
medium, or high relative self-efficacy to teach science to students with various needs and relative level of pedagogical variation.
Participants’ rating of feelings of capability to teach science (Q. 21: I consider myself: a specialist, very capable, capable but prefer other subjects, or not comfortable) and their years of teaching experience (Q. 6) were then used to select two interviewees with a range of attitude toward teaching science and years of experience from each low, medium, and high category. This selection process was used so that the researcher would be able to gather data from participants with the widest ranges of attitudes, experiences, and practices. Those who were selected for an interview were contacted via email with the interview consent form attached in order to establish an agreeable date and meeting time. While the survey was used to identify big picture themes and factors that are currently influencing the state of elementary science education, the follow-up interviews
were used to refine, explain, and support the identified themes and factors in greater detail.
Data Analysis
For this study, data collection, analysis, and recording were conducted in an integrated pattern. Quantitative data gathered from the survey were statistically analyzed using the statistical functions on the SurveyMonkey program and MS Excel. Qualitative data collected from the open-ended question in the survey and follow-up interviews were analyzed by hand through an inductive and deductive coding process. Both types of data were compiled and analyzed indicating themes and patterns that were used to develop responses to the overarching research questions.
Validity of the data collected and the themes elicited from the findings in this study were ensured to the greatest degree possible. Strategies to support validity include triangulation and member checking. This process was conducted to confirm that the verbatim transcripts accurately expressed the views of each participant and to allow for any necessary clarification. These validation processes ensured that the findings are grounded in the data, that coding themes selected were appropriate, and that inferences that were made align logically with the data (Creswell, 2002).
Quantitative
Nominal and ordinal data were collected during this study. The majority of the quantitative data are presented through descriptive statistics. These data include demographic information that assists the researcher in knowing the breakdown of participant representation based on several characteristics (e.g., age, gender, years of teaching experience). Descriptive statistics also provide a breakdown of the proportions
of survey participants’ attitudes, teaching strategies, use of resources, time spent on science instruction, and the availability of various support systems.
Nonparametric statistics do not assume normal distribution and work well for a sample size above approximately 30 (Salkind, 2008). A chi-square test for independence was performed on several sets of nominal data from the survey. These sets included the explanatory variables B.A. and B.Sc. The response variables included capability levels, degree of science as a personal priority, professional experiences, and teaching strategies. A chi-square goodness-of-fit test was used on the frequency distribution of time spent on science instruction by survey participants. The results of these analyses will provide broad data trends that assisted the researcher with answering the research questions. Qualitative
Qualitative data were collected from both the survey and the follow-up interviews. As suggested by Creswell (2012), the researcher began this aspect of data analysis by reading through the survey data and transcriptions several times. This process allowed the researcher to develop a deeper and more comprehensive understanding of