Data Analysis

4.5.1 Data management

All audio files were transcribed using the inbuilt NVivo 10 transcription function by the researcher. Transcription included emotional expressions (hmm, laughing etc) and incomplete or grammatically incorrect sentences in order to obtain as accurate

representation as possible of participants’ responses (see Appendix E for an example transcript).

Most data including surveys, audio files and transcripts were stored in NVivo 10 and classified according to school and participant type (student, teacher, staff). NVivo also provides linkage between files and internal memos which allowed each data type (audio recording, transcript and document) to be linked to notes explaining how that data was gathered and any relevant notes stemming from observation of the program. For instance, interview recordings were linked to memos that described how the interview went, the environment it was conducted in, visual observations of the room and any other relevant factors that might affect participant responses.

4.5.2 Interview Analysis

The coding function in NVivo was used to carry out thematic analysis on the transcribed interviews. Thematic analysis was used for this research as is a powerful and flexible tool that enabled in-depth exploration of the research (Braun & Clarke, 2006). When carrying out the analysis I followed Braun and Clarke’s (2006) procedure, the main points of which are:

1. Familiarising yourself with your data; 2. Generating initial codes from the data; 3. Searching for themes amongst codes; 4. Reviewing themes through coding new data;

5. Defining and naming themes, organising in a framework or map; 6. Producing the report, representing the themes.

The themes were further developed and subsequently structured according to the five research questions. This study uses Braun and Clarke’s (2006) definition of a theme as “capturing something important about the data in relation to the research question, and representing some level of patterned response or meaning within the data set” (p. 80). Where relevant, themes were further refined and compared between year level (7, 8) participant type (teacher, staff and student) and individual school. This was made possible

through the function of NVivo that allowed ‘attributes’ such as participant role, year level and school to be added to each interviewee response and their subsequent statements.

4.5.3 Survey Analysis

Due to the low number of completed surveys a statistical analysis was not conducted on the survey data. Instead Excel was used to sort, summarise and graph data to investigate student responses. Student responses were examined in terms of proportions (percentage of student group) out of 42 completed surveys. Responses to individual items were used directly but were also converted into a numerical format (Table 3). Response types 1-3 were used for variables looking at students’ backgrounds and response type 4 was specifically used for items asking students to reflect on how their experiences at KIOSC had changed their perceptions or behaviours. Several related items were compiled to allocate students a general score for particular components but this only occurred for items with the same response format.

Table 3. Format of items with type 1, 2, 3 and 4 responses for student agreement with statements and their conversion to numerical weighting.

Response Type 1: Strongly Disagree

Disagree Neutral Agree Strongly Agree

Numerical Conversion 0 1 2 3 4

Response Type 3: Never Once a year

Once a term

Monthly Weekly

Numerical Conversion 0 1 2 3 4

Response Type 2: None A few Some Most

Numerical Conversion 0 1 2 3

Response Type 4: Less likely to agree No change in agreement More likely to agree 57

Student scores across the compiled background variables were compared to determine if there were any trends or associations of interest. Different levels of these variables were also compared with the variables assessing change in students following their visits to KIOSC. This was done in order to ascertain if students of different backgrounds responded differently to their non-formal learning experience. Findings from the survey analysis were compared with students’ interview responses to determine if either type of response explained observed trends in the other.

Students’ score for variables combined from several survey items (e.g. Parent Science Support) were sorted into Very Low, Low, Neutral, Moderate and High levels. These levels were determined by the histogram analysis in Excel which uses Scott’s normal reference rule to calculate the histogram bin width. This formula was used as students’ scores tended to follow a normal distribution. An example of this categorization process is provided in Appendix G. As the number of combined items for each variable varied (Table 4) the scores that determined the levels for a variable were usually different.

Students’ scores for variables reflecting on their KIOSC experience (all Type 4 responses), were also combined from a series of related items (Table 5). The scores for response variables were divided into levels of negative change (-4 to -1), no change or neutral (=0), and positive change (1 to 4). Some response variables were calculated from multiple items and thus a further division between negative/positive scores (-2, -1 or 1, 2) and high negative/high positive scores (-4, -3 or 3, 4) was created for these items.

Items which asked students to compare science to other subjects or their ability in science to their classmates on a 5 point scale from 0 (worse/less preferred) to 4 (best/most preferred) were treated as individual scores and not compiled into a single variable. Additional variables which were not compiled included whether students’ immediate family members had a science related job, student gender, year level, and number of books in the family home. Survey items on family education and job level were not included due to incomplete responses.

Numerical Conversion -1 0 1

Table 4. Student scores for compiled variables relating to perceived support for their learning and pre- existing attitudes towards science. Students were asked to mark their agreement with individual statements which were then compiled into larger related categories as listed. Items marked as FLIPPED had their scores reversed.

Background

Variables Items used

Parental Study Support

(Response Type 1)

1. They expect me to do further education or training after high school, such as university or TAFE

2. They know how well I am doing in my classes 3. They always attend parents’ evenings at school Parental Science

Support

(Response Type 1)

1. My family talks to me about how science and mathematics will help me in my life

2. They think it is important for me to learn science 3. They think science is interesting

4. They would be happy if I decided to pursue a career in science Friend Science

Support

(Response Type 3)

How many of your friends: 1. Like science?

2. Think science is cool?

3. Get good grades in science?

4. FLIPPED Would think less of you if you did science activities? Friend Study

Support

(Response Type 3)

How many of your friends:

1. Care about their grades in school? 2. Encourage you to do well in school? 3. Would be described as smart or 'brainy'? Teacher Support

(Response Type 1) 1.2. My teacher makes learning science interesting and fun My teacher thinks I could be a good scientist one day Science

Participation (Response Type 4)

How many times a year do you?

1. Visit a science camp, club, received an award or done a university project

2. Go to a museum

3. Do science activities (e.g. science kits, nature walks, experiments) 4. Read a book or a magazine about science

5. Visit websites about science

6. Visit a science centre, science museum or zoo 7. Watch a TV program about science or nature

8. Talk with someone at home about what I've been learning in science class at school

9. Play games about science Science Aspiration

(Response Type 1) 1.2. I would like to study science more in the future I would like to have a job that uses science Science Affiliation

(Marked on a scale from 0 – 10)

1. At the moment on a scale of 0 to 10, would you describe yourself as a science person?

Table 5. Student scores for variables measuring change in student perception following their KIOSC visits. Students were asked to mark their agreement with individual statements (all response type 4) which were then compiled into the variables. Items marked as FLIPPED had their scores reversed.

Variables for change

following KIOSC visit Items Used

Change in science

affiliation 1. I think I could be a good scientist one day 2. People who do science are like me 3. I see myself as a science person

4. Others see me as a science person Change in perceived

science utility 1. Studying science is useful for getting a good job in the future 2. FLIPPED Science is not that necessary to get into

desirable courses at university or TAFE

3. Knowing science is useful in many different jobs Change in science

aspiration 1.2. I would like to study more science in the future I would like to have a job that uses science Change in science

interest or enjoyment 1.2. Learning science is relevant to my life I learn interesting things in science lessons 3. I look forward to my science lessons 4. FLIPPED Learning science is boring Change in preference

for learning science outside of school

1. I prefer learning about science outside of school

Change in science

class participation 1. I often take part in science class discussions and ask questions Change in confidence

in science ability 1. I get good marks in science 2. I can do well in science tests and assignments 3. FLIPPED Science is difficult for me

4. FLIPPED I am just not good at science Change in perceived

expectations for success

1. I will be able to master the skills and concepts in next year's science class

2. I could do a job that involves science Change in perceived

cost to learn science 1. If I study science in the future I will have enough time for friends and hobbies 2. FLIPPED People will make fun of me if I work hard in

science class