Validity of PracK Table 1

The validity of a test instrument is said to be the degree to which it measures what it purports to measure. A valid instrument must always be reliable but a reliable instrument is not necessarily valid. The terms often associated with validation of an instrument include construct validity, face validity, content validity and criterion related validity. Construct validity is the overarching construct, demonstrated by all the other types of validity measures (http://www.socialresearchmethods.net/kb/measval.php). This construct validity examines the relationship between the instrument being evaluated and the theory related to the construct being measured and the relationship between the instrument and other variables known to be related to the construct being measured (Kimberlin & Winterstein, 2008).

The PracK Table being evaluated here is not intended to measure teachers’ knowledge for practical work; it is merely a heuristic device for describing what teachers know. As such its validity cannot be assessed by the same criteria, as say, the validity of an instrument to measure a teacher’s content knowledge. What is important is that on the one hand all the categories and sub-categories included reflect some aspect of the teachers’ knowledge that influences their decisions around practical work and on the other, it includes all the categories and sub-categories likely to be observed. These aspects speak to the face and content validity of the PracK table. According to Haynes, Richard & Kubany (1995, p.239) content validity is “ the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose.” Neither of these measures can be subjected to statistical analyses and rely on the judgment of the researcher in this instance (Kimberlin & Winterstein, 2008).

The categories and sub-categories were derived based on extensive research on teacher knowledge, practical work and the relationship between the two. In addition, the preliminary study using the PPCK Table further informed the PracK Table. I was satisfied with the face validity of the instrument. To verify this judgment, the three participants involved in the determination of the reliability of the table were asked to make a judgment as well. They all agreed that on face value, the table appeared to capture those aspects of teacher knowledge important for practical work.

Assessing the content validity involved a more rigorous process of checking that most of the teacher statements and observations during the lesson could be coded using PracK Table 1, that most codes were used, and also that there was a good spread of codes across the table. The effects of unequal opportunities in education in South Africa prior to the first democratically elected government persist in the current education system, where schools can be categorized into three broad educational contexts. These are poor performing working class school, a well- resourced, high performing, middle class public school and well-resourced, high performing private school. Working class schools are poorly resourced, do not attract the best qualified teachers and perform poorly in the provincial and national systemic tests (Department of Basic Education, 2013). Schools serving middle class families are generally better off, and those serving upper middle class families, even better off in all three of these aspects of schooling. Clark and Linder (2006) showed how constraining the context of working class schools could be on teaching practice. Assuming that the context would also influence the nature of practical work in schools, the validity of the PracK Table had to be assessed in each of the three main educational contexts in South Africa.

With this in mind, and having already drawn on the evidence in a working class school for the development of the table, it was decided that a well qualified, experienced science teacher in each of the other socio-economic contexts would be interviewed and observed to assess the content validity of the table.

A comparison of the three teachers’ qualifications, teaching experience and the schools at which they teach is given in Table 4.1 below.

Teacher School

Grade

observed Qualifications Subject

specialisation No. of years teaching No. of years teaching at current school Luvo A – working class 9 B.Sc PGCE B.Ed (Hons) Chemistry Geography 13 7 Nadjwa B – middle class 8 B.Sc PGCE Microbiology Biochemistry 9 7 Michelle C- upper middle class 9 B.Sc B.Sc(Hons) HDE Microbiology Biochemistry 13 9

Table 4.1 Teachers’ biographical details

Just as described for Luvo in Section 4.1, pre- and post-lesson interviews were conducted with Nadjwa and Michelle and one of each of their practical lessons was observed. The structured questions across the two interviews were the same as in the interview with Luvo, except that the order in which they were asked differed, depending on the availability of the teacher. Again the intention here was not to provide an in-depth description of the other two teachers’ knowledge for practical work. Instead, a description of the school, the background of the teacher and the lesson observed, will be presented to sketch the context. This will be followed by a summary of i) the categories and sub-categories used and not used ii) the presentation of evidence of sub-categories not used for Luvo and iii) the presentation of incidences where the teachers’ knowledge could not be adequately captured by any of the existing codes.

School B is a well-resourced, high-performing, middle class, public school. The school has two science laboratories which are also used as classrooms and two other rooms which have a demonstration desk at the front. The school employs a full time laboratory technician who prepares practical equipment according to the teacher’s instruction. In addition, he does all the printing of worksheets and prepares resources and other material for the teachers in the science department. Learners at the school are engaged in a number practical activities in the junior grades but the number of practical activities decrease as they progress from Grade 6 to Grade 12. Many Grade 12 teachers are of the opinion that, given the demands placed on them to complete the syllabus and to prepare their learners for the high stakes matric examinations, they only have time to complete the prescribed practical activities. An internal science expo is run for the Grade 9’s in which they have to reseach a topic of their own choice and present their work to parents, teachers and fellow learners. Although encouraged to do so, very few learners take up the challenge of participating in the Cape Town Science Expo.

Nadjwa was teaching the section on the preparation, tests and properties of oxygen, carbon dioxide and hydrogen at Grade 8 level. She had given the learners notes on the topic and her intention was to demonstrate the preparation and tests for the three gases in the lesson I observed. She also intended comparing the solubility of the three gases.

The lesson was conducted in one of the school laboratories and all the equipment and resources had been prepared by the laboratory technician, and was set up on the demonstration desk at the front of the laboratory. Nadjwa, with the help of three learner volunteers, sucessfully demonstrated the preparation of the three gases, but the tests for the presence of carbon dioxide and hydrogen were unsuccessful. She did not attempt to compare the solubility of the three gases as planned because she was not able to collect enough carbon dioxide and hydrogen required for the investigation. Before the demonstration of each of the gases, Najwa showed a slide on the overhead projector showing the chemicals and the arrangement of the equipment required. Learners were also given a worksheet which they were required to complete as they went along and they had some time after the completion of the demonstration to complete the worksheet.

School C is a well-resourced private school serving an upper middle class community. There are a number of similarities between Schools B and C with regard to availability of resources and the types of practical activities learners in the schools are engaged in. School C has two dedicated science laboratories and two other rooms, referred to as mini-labs, which have a demonstration desk at the front and workstations along the three walls of the room. The school employs a full time laboratory technician who prepares practical equipment according to the teachers’ instructions. The technician, however, who was straight out of high school, had no experience of working in a laboratory, is still being trained by the teachers.

The number of practical activities in School C also decreases from Grade 8 to 12 as the time pressure takes it toll. As with School B, only the prescribed activities are done in matric. The school also used to run an internal science expo and some learners were encouraged to participate in the provincial competition. At the start of the Grade 8 year learners are made aware of the laboratory rules and are familiarised with the basic laboratory equipment. At Grade 8 and 9 level an attempt is made to include a practical activity for every topic taught. Michelle was teaching the respiratory and circulatory system to Grade 9’s. In the double period lesson observed, learners were asked to investigate the effect of exercise on heart rate. According to the teacher, the main objective of the lesson was for learners to learn the scientific method.

The lesson was conducted in a mini-lab in which 28 learners were seated in rows. At the start of the lesson the teacher stated the aim of the investigation and asked for possible ways to conduct this investigation. She then explained how to measure pulse rate and allowed them to practice doing this once. Learners were then asked to conduct their investigation in groups. They were encouraged to leave the classroom to reduce the noise level and were asked to report back to the classroom in 30 minutes. All the groups went outside- some running up and down the stairs, some running across the playground, and others doing star jumps in the quad. After half an hour, all the groups had returned to class. As learners returned they were given the worksheet and asked to complete the task on their own, but they were allowed to reference any book or notes they chose. This was done in almost complete silence. A few learners raised their hands and asked questions mainly to clarify what it was they had to do. At the end of the lesson all learners handed in their work. The teacher later reported that not all learners completed the task in the allotted time and they were then given the opportunity to complete it in the next period.

In Section 4.2.1 all the codes assigned for Luvo were shaded in Figure 4.2 above. These codes were transposed onto PracK Table 1 and all the codes evident for the three teachers were shaded in Figure 4.4 below, thus clearly showing the used and unused codes.

Figure 4.4 PracK Table 1 showing codes evident for the three teachers

Forty six of the 48 codes i.e. 95,8 % of sub-categories were used and there was a good

spread across all the main categories thus meeting criteria 1 and 3 (Section 4.3) - this

suggested strong content validity. A few explicit examples of evidence of each of the sub-

KBEP KCt KC GPK KIS KLU KA KCx

concept'acquisition'&' development 'scientific'concepts' related'to'the'practical' activity. curricular'

requirements manage'laboratory models'&'modelling sense'making'discussions learners instruments'and' technology demonstration discovery'learning skill'building 'nature'of'observation' vs'inference problem'solving

teach'laboratory'skills school'science scaffolds'development'_{of'skills conceptions'and'skills}elicit'existing' district authentic'science 'distinction'between' theoretical' explanations'and' empirical'/descriptive' accounts avoids'activities'or' representations'that' give'rise'to'alternative' conceptions combines'different' forms'of'instruction'for' effective'learning aware'of'time'scale'of' potential'activities'or' investigations .PracK.Table.1.7.Luvo,.Najwa.&.Michelle organise'phases'of' activity organise'and'manage' materials organise'and'manage' group'work curricular'saliency horizontal vertical province/nation community school classroom appropriate' background'reading scaffold'understanding' of'appropriate'use'of' skills scaffold'integration'of' ideas'with'ideas'in' other'domains scaffold'ideas'to' related'contexts develop'scientific' attitudes learn'scientific'method motivation'by' stimulating'interest'&' enjoyment recall'of'concepts recognise'flaws'in' problem'solving' approaches'and' conceptual'thinking recognise'limitations'in' thinking'and'skills summative formative historical'context'of' development'of'ideas 'how'key'ideas'and' skills'are'related evaluation'of'validity'&' reliability'of'scientific' information

categories which were unused for Luvo, but were apparent for Nadjwa or Michelle are presented below.

Michelle’s curricular saliency is clearly evident when she speaks about some of the other practicals she has done.

As I said for most topics we actually do a prac – so we also will do another one – where we will dissect the heart and the lung. So this specific prac lended itself to the learners practicing the scientific method which we find them very weak in. The dissection we find that if we assess that, that they actually do better at it come Grade 11. So that they do it later anyway so we rather not assess that now and there you are more assessing dissecting skills and drawing skills. We did that previously we did an onion cell prac. In that one we sort of assessed their drawing skills. [Michelle post-lesson interview, p.4] The practicals are not done merely as a matter of curricular compliance but are informed by her knowledge of the curriculum, knowledge of her school context and specific assessment purposes she has in mind.

As Nadjwa spoke about the other activities she intended doing after the demonstration of the preparation and tests of the gases, it became clear that she combines a number of different strategies in her teaching.

Ok – so – we try and have a test after each module so there is going to be a gases test on this module. But before that we are doing a little presentation activity. So we have 10 topics – you have things like …. of gases , respiration, baking, carbon dioxide – baking cakes kind of thing and they have to relate one of the three gases to their topics . So they should know that baking cake relates to carbon dioxide. And so we put them in groups of three and we allow them to do some research on it so they come back and each person in the group must present something about baking cake and how it relates to their topic. And then one of the criteria is to make a mind map of the topic and then presentable and copy down so that the rest of the kids to make the rest of them interested – cause what we found was that they would chat and not be interested so they need to copy down the mind map so the rest of them just listening they have to copy down the mind map as well. … And then they write their test soon after that. We always tell them they don’t really have to study the mind map but they just read through it and we ask them one question in the exams so why did the Hindenburg explode kind of thing or what is … so we don’t force them study the extra information but general information about the topic.

[Nadjwa pre-lesson interview, p.6]

Besides these activities Nadjwa also generally engages her learners in problem solving activities.

we give them two investigations – so we tell them you have to look at the effects of rainbows . Then you have to create your own rainbow. … So we don’t tell them anything. So they have to figure out how some do it in a dark room, others do it with water, other do it with the glass you know so they need to

figure out, research, carry it out and then come up with their reports. … How you carry that out – that is entirely up to you. So its amazing – so you get some kids come up with the same, some come up with ingenious things …

[Nadjwa post-lesson interview, p.8]

Here it is also significant to note that an understanding of the hazards associated with laboratory chemicals, procedures and equipment and the ways of avoiding or minimising these hazards is not only essential for teachers and laboratory assistants, but should also be explicitly taught. Evidence of such knowledge was not equally evident in all teaching contexts.

In Michelle’s school the learners entering the school laboratory for the first time are made aware of some of these hazards and it was reinforced when necessary.

but also when it is a specific prac when a specific rule is important like when you smell something you don’t actually smell it, you waft it and this prac is very important that you have your safety goggles because this is very corrosive or whatever … [Michelle post-lesson interview, p.8] In Nadjwa’s lesson, when one of the learners offered to assist the teacher, he immediately took off his school tie and put on his safety goggles. When other learners volunteered to assist, their classmates reminded them to wear safety goggles. This suggested that her learners also had this awareness of potential hazards and laboratory safety rules.

This awareness also influenced the choice of instructional strategies of both these teachers. Michelle says “Sometimes if its something we feel is too dangerous like with the reactions of

the group one metals we obviously demonstrate that one” [post-lesson interview]. One of the

reasons Nadjwa gave for doing the activity as a teacher demonstration rather than a hands- on activity is also safety.

It’s going to be a teacher demonstration ya… Basically because it includes hydrochloric acid and it also goes a bit quicker also. As I’ve said they had hands on experience already so, we choose the safer stuff. I think they did one on filtering – chalk and salt ….. [Nadjwa pre-lesson interview, p.6]

Knowledge about health and safety in the laboratory and associated with particular activities, procedures and equipment is not adequately captured in PracK Table 1 since it was not foregrounded in the preliminary study with Luvo. However, further investigation highlighted its importance and this has thus been included as a sub-category of ‘content knowledge’.

All other units across all data sources in this study could be coded using PracK Table 1 and thus satisfies criterion 2. The content validity is thus adequate as it captures the knowledge it intends.