Analysis of VASI questionnaire data

The questionnaire data was manually analysed by coding the responses using a coding agenda adapted from a rubric used in the South African pilot study (Gaigher et al., 2014) provided in Table 3.5 below.

Table 3.5: Coding agenda for coding VASI questionnaire responses [Adapted from Gaigher et al. (2014)]

VASI item & NOSI Aspect

Naïve Mixed Informed

1a, b, and c: Scientific investigations may follow different methods. 1c: Only one Scientific method Or any two/more mistakes, e.g. 1b: yes, experimental and 1c: Similar or No Examples provided

No more than one of the following types of mistakes: 1b: Yes, it is an Experiment Or 1c: one general Method Or 1c: both examples are experimental Or 1c: both examples are non-experimental

All three answers must be appropriate

1a: Yes, the investigation is scientific as it aims to explain some aspect of the natural world 1b: No, it is not an experiment as there is no manipulation/control of variables/testing 1c: Yes, investigations can follow different methods: experimental/ practical/testing as opposed to nonintrusive/ non experimental/ research/ investigation/ observation/theoretical/ not-practical

Two suitable examples required: one

experimental and the other non-experimental 2. A scientific investigation should begin with a question not necessarily be testing a hypothesis Investigation should start with a hypothesis; also questions are not essential

A question is useful, but is regarded as part of a formal structure, investigation may be undertaken first and questions formulated later

A scientific question is the main reason why an

investigation is undertaken, a driving force to begin the investigation or inquiry.

VASI item & NOSI Aspect

Naïve Mixed Informed

3a. All scientists performing the same procedures may not get the same results

Similar procedures would always lead to the same results

Imperfect experimental

conditions may lead to different results

The human factor may cause different

interpretations of similar data, leading to different results 3b. Procedures followed in scientific investigations can influence results

Only one result is possible

regardless of the procedure

Different results would be primarily caused by the different

interpretations

Different procedures would yield different data sets which would lead to different results 4. Data are not

the same as scientific evidence There is no difference between data and evidence

Evidence differs from data; unclear/wrong/no Explanation

Evidence is generated from data, to support a claim/conclusion 5. Question drives the process of scientific investigations Team B did better, illogical or no Explanation

Team A did better, no explanation/argues that the tire has a larger effect than road Or, B did better and argues that the different roads have different effects on tires.

Team A did the best experiment because they addressed the investigative question 6. Conclusions should be consistent with data collected Option (a) is correct, with or without an explanation Alternatively, option (c) with no or illogical explanation. Option (c) is correct, i.e. ‘growth not related to sunlight’ with an explanation

Or, option (b) without explaining

Option (b) is correct, i.e. ‘plants grow taller with less sunlight’ because the data showed such a trend Speculations about the ‘unusual’ data are acceptable provided option (b) is chosen

7a & b. Explanations must be based on

One or no relevant ideas.

Only two relevant ideas.

Three relevant ideas: Two reasons: function of ideas

VASI item & NOSI Aspect

Naïve Mixed Informed

data and existing scientific

knowledge

larger hind legs/ comparison with existing models of dinosaurs/fitting of joints

One information type: existing knowledge of dinosaurs/skeletons/ joints

As indicated in Table 3.5, learners’ responses were either classified as naïve, mixed or informed understandings about scientific inquiry. Naïve understandings referred to responses which were not consistent with the NOSI aspect or were contradictory in nature, while mixed understandings referred to responses which were only partially consistent with a NOSI aspect or were correct but learners could not provide a satisfactory explanation for their reasoning. Informed understandings, on the other hand, referred to responses which were completely consistent with a NOSI aspect and learners, could provide satisfactory explanations for their reasoning (Lederman et al., 2014).

Table 3.6 shows an example of how one of the participant’s questionnaire were coded.

Table 3.6: Sample of scored VASI questionnaire

Number VASI questionnaire item Code Score

Q1

1. A person interested in birds looked at hundreds of different types of birds who eat different types of food. He noticed that birds who eat similar types of food, tended to have similar shaped beaks. For example, birds that eat hard-shelled nuts have short, strong beaks, and birds that eat insects have long, slim beaks. He wondered if the shape of a bid’s beak was related to the type of food the bird eats and he began to collect data to answer that question. He concluded that there is a

Number VASI questionnaire item Code Score

relationship between beak shape and the type of food birds eat.

a. Do you consider this person’s investigation to be scientific? Please explain why or why not.

Learner: “No because the person did not use any scientific enquiry. He did not mix chemicals or do anything that has to do with science.”

Naïve

1b. (b). Do you consider the person’s

investigation to be an experiment, please explain why or why not?

Learner: “Yes the person was really experimenting as he just went out to look at the birds and make some conclusions.”

Naïve

1c. c). Do you think scientific investigation can follow more than one method? If yes, please mention the two investigations that follow different methods and how those methods differ

Learner: “I do not think scientific investigations can follow more than one method. In other to collect data we need to search, we need to investigate to find the things we need. That process is an experiment.” Naïve 1 (overall combined score for question 1) Q2 2. Two students are asked if scientific

investigations must always begin with a scientific question. One of the students says “yes” while the other says “no”. Whom do you agree with and why?

Learner: “I agree with the student that said yes because it all starts with a question which will help us to investigate something, maybe help us in choosing the procedure too.”

Number VASI questionnaire item Code Score

Q3a 3(a) If several scientists ask the same question and follow the same procedures to collect data, will they necessarily come to the same conclusions? Explain why or why not.

Learner: “No, because every scientist has a different point of view. So this also applies to the investigations they may always come up with different conclusions.”

Informed 3

Q3b 3(b) If several scientists ask the same question and follow different procedures to collect data, will they necessarily come to the same

conclusions? Explain why or why not. Learner: “No, it all depends on how they started. They

may come up with different answers. So there are possibilities of that they can conclude the same but because of the different methods I don’t really think they would come to the same conclusions.”

Mixed 2

Q4 4. Please explain if “data” and “evidence” are different from one another.

Learner: “For me evidence and data are the same thing. This is because when scientists come with a theory they need to have evidence for it as well as data. Evidence must also be something you can see and data can include numbers, but since it is for the same purpose, I believe they are the same. Evidence must be something we can see, which proofs things.”

Naïve 1

Q5 5. Two teams of scientists were walking to their lab one day and they saw a car pulled over with a flat tire. They all wondered, “Are certain brands of tires more likely to get a flat?”

Team A went back to the lab and tested various tires’ performance on one type of road surface.

Team B went back to the lab and tested one tire brand on three types of road surfaces. Explain why one team’s procedure is better than the other one.

Number VASI questionnaire item Code Score

Learner: “Team A, because they are testing the various types of tyres as per the research question posed in the scenario.”

Informed 3

Q6 The data table on question six of the VASI shows the relationship between plant growth in a week and the number of minutes of light received each day. Given the said data, explain which one of the following conclusions you agree with and why. a) Plants grow taller with more sunlight. b) Plants grow taller with less sunlight. c) The growth of plants is unrelated to sunlight.

Please explain your choice of a, b, or c below:

Learner: “B, plants as we see in the table the more light the less the plant grows. When the light become too much there was no growth.”

Informed 3

Q7a 7. The fossilized bones of a dinosaur have been found by a group of scientists. Two different arrangements for the skeleton are developed as shown in the diagrammatic representation on VASI question seven a. Describe at least two reasons why you think most of the scientists agree that the animal in Figure 1 had the best sorting and positioning of the bones?

Learner: “The animal in figure one had strong lower leg bones to support its weight. It also has great balance due to stronger leg bones. This helps the animal stay upright and more likely to reach for food hence survival”

Informed

Q7b b. Thinking about your answer to the question above, what types of information do scientists use to explain their conclusions?

Learner: “It all starts from finding some fossil

evidence, making observations, using the data they collected and checking out the work that has been done by other scientist to guide their conclusions.”

Informed 3 overall combined score for question 7

Number VASI questionnaire item Code Score

Total VASI score 19/24

Table 3.6 illustrates how one participant’s questionnaire responses were coded and scored during the content analysis process for the all eight VASI items. A composite score and code were allocated for questions 1a, b and c and questions 7a and b, because the items jointly assessed a single aspect about scientific inquiry. Question 3a and 3b, on the other hand, assessed separate aspects of the NOSI, and hence were coded and scored independently.