Evaluation and Results of Phase 2

Evaluation in phase 2 was through basic repertory grid analysis and summative tests. Each are considered below.

5.2.2.1 Repertory Grid Interviews

The following section explains the nature of both types of repertory grid interviews that were conducted to inform how the pedagogy could be potentially modified from a student point of view.

Repertory grid interviews were carried out with a group of 4 students with a view to informing the refinement of the next iteration of the pedagogical tool.

The set of students who provided the information for each grid were drawn from the intervention cohort so as to be representative of a range of abilities. The findings are firstly presented in terms of ‘How I see myself as a scientist’ in relation to the scientists presented to them as elements in the repertory grid interviews (Galileo, Frankland and Fleming). Findings are then presented in terms of ‘How I see the pedagogical tool’ which was the version of the workbook that was

112

developed to support IBL in phase 2. The form of analysis used for the repertory grids is an ‘eyeball analysis’. Jankowicz (2004) views this as a preliminary or basic analysis with respect to other forms of analysis with repertory grids involving the relationships between the constructs and the elements. He describes constructs as indicating how the respondent thinks and the ratings as indicating what they think. Following an examination of the relationships, conclusions may be drawn.

How Students Perceive themselves as Scientists

A summary of the results of how the students see themselves as scientists is provided in the repertory grid (Figure 5.3).

Legend:

black = student chose this rating to indicate where they see a good scientist on the construct;

Blue _{= how the student sees themselves as a scientist now;}

Red

Note: = how the student sees themselves ideally as a scientist

Not all students responded to the rating exercise. This is also the case in Figure 5.4.

Figure 5.3: How I as a Student see Myself as a Scientist (N = 21)

The following sections detail how students perceive themselves as scientists in relation to Galileo, Fleming and Frankland in terms of constructs of contrasting poles including:

113

Persistent --- Gave up when told it did not work:

All students placed themselves in the middle of this polar range in that all gave themselves a rating of three. However, it is clear that they would like to move towards the ‘persistent pole’. This is in congruence with where they see scientists such as Frankland, Fleming and Galileo.

Thorough/Organised---Did not test things fully:

Once again students see themselves now as being mainly in the centre of the grid with one exception who sees herself at the ‘thorough / organised’ pole. Scientists are viewed as being at or near this pole, and this is also the pole towards which students ideally see themselves moving over time.

Loved what they did --- Did science for financial gain:

Currently, students see themselves as being quite close to the pole ‘loved what they did’ and ideally see themselves as being even closer to it. It is also at this pole where they see scientists.

Careless with respect to others --- Cared for others:

Students believe scientists to be almost evenly distributed along the whole construct. However, they see both themselves now and their ideal selves as being at the pole ‘cared for others’.

Confident ---Did not believe in what they did:

Students perceive scientists to be at the ‘confident’ end of the pole of this construct. At the moment they see themselves as being mainly near this pole but some are in the middle. The respondents almost entirely see themselves as ideally at the ‘confident’ pole of the construct.

‘Risk-takers --- Played it safe for themselves’:

Students see scientists as a group of people who are mainly risk-takers.

However, they see themselves as being in the middle of this construct or towards the pole ‘played it safe for themselves’. Ideally, they would like to be in the middle of the polar range of the construct.

114

The results suggest that students are displaying features of Kellyian permeability exhibited by their current position and desire to move closer to this pole. They are also exhibiting features of Kellyian aggression in relation to how they rate themselves in relation to being ‘thorough’ versus ‘not testing things fully’. Linked to the ratings along this construct is the view students hold of themselves as being ‘confident’.

How Students see the Pedagogical Tool

A summary of the results of how the students see the pedagogical tool is provided in the grid (Figure 5.4).

Legend:

Black = student chose this rating to indicate where they see the workbook on the construct; Red = how the student would ideally like to see the workbook

115

Asks questions --- Gives facts / instructions:

Students see the workbook that supported IBL as one which mainly asks questions as opposed to giving facts and instructions. Ideally, they would like to see it in the middle of the construct - as a mixture of both.

More difficult to understand the questions asked --- Easier to understand the questions and definitions:

Most students view the workbook as being at or near the pole of ‘More difficult to understand the questions asked’. Ideally, they would like to see the workbook as being towards the opposite pole, which is ‘Easier to understand the questions and definitions’.

Think for yourself and forget --- Just read and forget:

Currently, students perceive the workbook to be near the pole ‘think for yourself and forget’ and ideally they would like to see it move towards the other pole with most students favouring the final location to be in the middle of the construct.

Harder to study for exams as our answers written --- Easier to study for exams:

At the moment students believe the workbook to be mainly towards the pole ‘Harder to study for exams as our possibly inaccurate answers are written’, although some believe it to be closer to the opposite pole. However, the majority of students would ideally wish to see the workbook being at the opposite pole, ‘easier to study for exams’.

Forces you to think --- Saves you thinking:

The majority of students see the workbook as being a pedagogical tool that forces you to think. Their ratings are evenly spread across the construct regarding where they would like to see it ideally.

Not Structured --- Structured:

While some students view the workbook as being near the ‘structured’ pole, a greater number believe it to be at or near the ‘not structured’ pole. Responses indicate that they see the workbook ideally being at the ‘structured’ pole.

116

Gives you an opportunity to be a real scientist --- Does not give you an opportunity to be a real scientist:

Students perceive the pedagogical tool to be mainly at or near the pole that gives them an opportunity to be a real scientist. Some ideally see this as the end of the construct at which to remain but the rest (a minority) would like to see it move to the middle of the construct range.

Gives you an opportunity to develop your thoughts --- Don’t really develop your thoughts:

Respondents view the pedagogical tool to be mainly at or near the pole that gives them an opportunity to develop their thoughts. Some ideally see this as the end of the construct at which to stay located but the minority of others would like to see it move towards the middle of the range of the construct.

Gives you confidence to be a scientist --- Gives you confidence to be a student for exams:

Most respondents view the workbook as being located on or near the pole ‘Gives you confidence to be a scientist’. A sizeable minority believe it to ‘give you confidence to be a student for exams. Ideally, most students would like to see it move toward the middle of the construct.

In summary, the above results indicate that the pedagogical approach using IBL is one that students perceive as forcing them to think and develop their thoughts. This is seen as being due to the fact it asks questions of learners rather than just ‘gives facts’. It is therefore not surprising that students view the teaching and learning approach as giving them an opportunity to be a real scientist because it gives them confidence to do so. While it is gratifying that the affective dimension of learning is being acknowledged, a concern is helping them not forget after they have engaged in critical thinking.

5.2.2.2 Summative Exam Results

The results obtained by the intervention and control groups are given in Table 5.4 using the same exam as in phase 1, but with question 7 removed.

117

Table 5.2 Summary of Phase 2 Exam Results (N = 123)

Question No. 1 2* ₃* ₄* ₅* ₆* _N

% of Intervention

Correct 50.0 71.4 57.1 26.2 42.9 14.3 42

% of Control Correct 45.7 22.2 19.8 8.6 13.6 1.2 81

* statistically significant at the .05 level using a Chi-squared test

Table 5.2 indicates that the intervention cohort out-performed the control cohort in all questions in the exam. There was a significant difference between the percentage of students in the intervention who were successful in answering Questions 2 to 6, in comparison to the control group.

The results comparing the intervention group in phase 1 with the intervention group in phase 2 are given in Table 5.3.

Table 5.3 Summary of Phase 1 versus Phase 2 Exam Results (n= 94)

Question No. 1 2 3 4 5 6 7 N

Phase 1 Intervention

(% Correct) 71.2 67.3 42.3 28.9 36.5 7.7 21.5 52 Phase 2 Intervention

(% Correct) 50.0 71.4 57.1 26.2 42.9 14.3 42

The above table shows that the Phase 2 cohort did advance their knowledge in comparison to that of Phase 1 in all questions apart from those of Questions 1 and 4. Question 7 is not considered as it was not undertaken in the Phase 2 exam. An examination of Table 5.2 in conjunction with Table 5.3 would suggest that the areas where improvements need to be made are Questions 1 and 4. With regard to Question 4, approximately 20% of the entire cohort of students associated the number of types of atom present in an element or compound with the mass number or the atomic number of an element that they associated with the question. This information was not given in the question but was stated in the periodic table they received with their exam script. In relation to both Question 1 and Question 4, students were not invited to draw their answers and this may have been inhibiting for them given their capacity to convey their ideas. Regardless, it

118

was intended that the learning outcomes underpinning both questions should be the focus of the planning stage in Phase 3. General alternative conceptions in relation to the use of the incorrect inter-particulate distance, the inability to use symbols and the inability to distinguish between an atom and a molecule would also be included during the planning stage.