Instrument Development and Study Protocol

4.4.1 Program Attendance

I attended five of the eight visits to KIOSC that involved year seven and eight students from the two schools participating in the research. My attendance at these visits served

primarily to build a familiarity with the students, teachers and staff prior to conducting interviews. Once this relationship was established it was not necessary to attend students’ second visit to KIOSC for the year.

Those present were aware of my role as I had been previously introduced to the students and teachers from the two participating consortium schools when originally recruiting students for the study. This introduction was repeated on student visits to KIOSC when necessary, as these took place several weeks after my initial visit. During the visit I did not take part in the activities and my interactions with students and teachers were largely limited to pleasantries and discussion of the activity. My presence during the visits helped guide subsequent interviews with students, teachers and staff, for example by asking about a particular activity or event that had occurred to prompt reflection. A level of familiarity with individuals also assisted the flow of conversation during interviews and enabled me to probe in more depth on particular points.

A concern of observation research is that the presence of the observer affects what is being observed and that the researcher’s own bias and subjectivity will affect their observation (Merriam & Tisdell, 2015). My presence may have modified participant behaviour, for instance by encouraging students to perform in a more socially acceptable manner. However after the initial introduction I believe my presence was minimally disruptive as throughout the day staff, teachers and students were all more concerned with proceeding in a timely fashion through the activities and in interaction amongst themselves. As the students were in a novel setting and in the presence of unfamiliar adults (KIOSC staff) it is likely that the addition of one more unfamiliar adult did not dramatically affect their participation or overall behaviour. I attempted to further minimise any impact of my presence by remaining to the back of the room during staff teaching and only interacting with students during the activities when it was clear that social interaction was taking place, e.g. a group of students and teachers gathered around a table discussing a particular activity.

4.4.2 Interviews

Interviews are key tools for exploring the internal motivations and perceptions of individuals; how they interpret or make meaning of their experiences (Rubin & Rubin, 2011). This makes them a common component of qualitative research (Ritchie, Lewis, Nicholls, & Ormston, 2013) and a necessary tool here in this research exploring

stakeholder interpretations of a non-formal science education program. Consequently for 52

this research the main data gathering was performed through a series of semi structured interviews.

Semi-structured interviews were used to elicit participant responses on particular topics which were of interest while also allowing freedom to explore certain responses in depth or to follow relevant tangents. The questions asked were specific to participant type, either staff, teacher or student (see Appendix D for a complete list of questions). Teachers, staff and students were interviewed in the weeks after each KIOSC visit that they were involved in. This meant that some teachers were only interviewed once while most of the staff were interviewed multiple times. Students were generally interviewed twice with very similar questions, however a few students were not present at school when

interviews took place and so only took part in one interview. Repeated interviews usually took place months apart which was expected to minimise any priming effect from

repeated exposure. Participants who were interviewed for a second time were also asked about any changes in their opinions since the previous interview.

The interviews took place in an empty classroom or office, and also occasionally outside in a quiet space at the school (teachers and students). All interviews were recorded with an audio recording device placed next to or between the researcher and the interviewee and this recording was later transcribed by the researcher. Notes of participant names were taken at the beginning of the session as well as occasional notes to track students’ responses. On average the interviews with teachers and staff took around 30 minutes, while interviews with students ranged up to 20 minutes depending on the number participating and the general nature of the group.

Interviews with teachers and KIOSC staff were typically one-on-one, however due to time constraints towards the latter half of the year there was one group interview with all four staff and also several individual written responses to the interview questions. While this format reduced the amount of depth individuals went into in their responses, it still provided useful information on participant perspectives.

Students were interviewed in small groups of usually two or three, often within the same friendship group, in order to establish a comfortable environment for them. This

arrangement provided students with a more natural setting which encouraged the sharing of ideas amongst peers that they might otherwise have been reluctant to share with the researcher, an unfamiliar adult stranger. With usually only two or three students present in each group interview it was possible to explore each student’s interpretation of their

KIOSC experience in some depth and take the time to ensure each student had a chance to respond to each of the listed questions.

While there is criticism that this type of interview can affect the data obtained as individuals are pressured by the group to conform to a socially acceptable view point (Ritchie et al., 2013) it can also provide insight into the social negotiation of values and roles. Peer influence can be a strong influence on student perception of science

(Riedinger, 2015) and so in this research, insight into the socially constructed perception of KIOSC amongst groups of students was useful for understanding student perspectives.

4.4.3 Survey

After students had completed their visits to KIOSC for the year they completed a reflective survey (see Appendix F for a copy of the survey). The survey was initially designed using the online software Survey Monkey and students completed a paper version of this and the responses recorded manually by the researcher into the Survey Monkey software. A total of 44 students took part in the survey however due to incomplete responses only 42 surveys were useable.

The survey included questions on student family background and other demographic data in order to place the research in a wider context and also to inform research question four: Is there an interaction between student background and students’ interpretation of the non-formal science education program? This information is pertinent to

understanding student interpretation of their non-formal learning experiences as student attitudes and expectations can affect their approach towards, and the subsequent impact of, external learning experiences (Falk & Storksdieck, 2005). Additionally student

background, more specifically their science capital, heavily influences students’

participation in science learning activities in and out of school as well as influencing their trajectories into post-compulsory STEM study (DeWitt & Archer, 2015). Hence several questions concerning student science participation and family background were adapted from the UK ASPIRES year eight survey (equivalent to Australian year seven) and also from the US ‘Is Science Me?’ study (Aschbacher, Li, & Roth, 2010) in order to ascertain whether student background had any effect on how they responded to non-formal learning experiences. Students were also asked about how they currently felt about their science teacher and how they thought their friends felt about science. These questions were included as science teachers have been shown to be an important school based factor in students’ continuation into post-compulsory STEM education (Lyons & Quinn, 2010a) and

the attitude of a students’ friendship group can also affect student participation in science (Barton, Tan, & Rivet, 2008).

The survey also included a series of items based on the Expectancy Value of Motivated Behavioural Choice (EV-MBC) framework (Eccles, 2009) which measured students’ current and changed attitudes towards learning science. The EV-MBC has been previously used as indicators of students’ motivation to continue with post-compulsory STEM study and pursue a STEM related career (Andersen & Ward, 2014). Subsequently this research drew from several survey items used by Andersen and Ward (2014) which asked students to evaluate how they felt about science compared to their other subjects. These items included questions which assessed students’ expectancies of success in science along with their subjective valuation of learning science (relative interest, cost, utility and

attainment). The EV-MBC framework was also used for a series of evaluation questions asking students how they felt their KIOSC experiences had affected their perceptions or capability regarding science. These were presented in a similar manner to the post reflective surveys for out-of-school science learning experiences reported by Jensen and Sjaastad (2013) and Jensen and Bøe (2013). Comparison of change in students’ opinions with their pre-existing support and attitudes towards science facilitated further

investigation of research question four as well as contributing to answering research question five; What opportunities are there in an ongoing non-formal science education program for students to develop trajectories into post-compulsory STEM education? Research question five was also explored in more depth in the interviews but the survey components were included to aid in highlighting possible trends in student responses. Using questions from multiple surveys ensured that this survey contained well-defined, previously validated questions that had already been tested on thousands of students across the world. However in many cases adaptation of questions in terms of rewording, removing or adding particular statements was necessary in order to place questions in the Australian context and accommodate the focus of this study.