Instruments

3.9.1 Pre-tests and post-tests

Knowledge gained from participating in an educational course is measured using pre-tests and post-tests. Before the training begins a pre-test, which is a set of questions to

determine the participants course content knowledge level, is administered. Participants are given a post-test after the completion of the course. The post-test can be the same set of questions used in the pre-test or may be a set of comparable questions. Whether the

Page 111 of 308

programme was successful in increasing the participant’s knowledge can then be

determined by comparing pre-test to post-test scores.

3.9.2 Using pre-test and post-tests

Whether or not participants have learned from the educational intervention can be determined from the test scores. However a well-designed pre-test / post-test allows teachers to understand which competencies and concepts were understood, which may need additional time and which pedagogical methods work well (Dimiter et al., 2003).

Pre-tests and post-tests may not be suitable for every educational intervention. If a course contains sufficient opportunity for the assessment of participants’ knowledge, for example

highly interactive courses where formative assessment could be used, then tests may not be warranted. It may not warrant the time requirement for creating, administering and analysing tests for very short courses (Black, 2004).

It is very important to consider the type of assessment to be used as the evaluation tool. In this research the pre-test / post-test method was used as it was fundamental to the research that there was an understanding of exactly what knowledge could be credited to the teaching. Also if only a post-test was administered the knowledge gains could not be assessed to any extent. In addition a limitation of the pre-test / post-test design of

knowledge gained is the recall of this knowledge in the future. Therefore a one month post-test was also used to assess entry into long term memory.

3.9.3 Developing a pre-test and post-test

Page 112 of 308

accurately if the device itself is faulty. The tests, pre and post, must be reliable and valid and composed of clear and well written questions.The following were taken into account when developing the tests.

Questions should be created that focus on the course objectives.

At least one question was developed to cover each course objective to ensure participants could demonstrate their knowledge of the most important course concepts. The questions were created so that extraneous details need not be memorised. The emphasis was placed on simple core concepts and facts.

Only questions to which clear answers were provided during the course were included. Participants were not tested on knowledge or concepts that were not sufficiently covered in the course.

A test was developed that should take the participants approximately 15 – 20 minutes to complete.

The course will take approximately two hours to complete. Therefore it was considered that a test lasting 15 – 20 minutes was appropriate.

3.9.4 Question creation

There are a variety of questions types that could be used in the tests (iTech, 2008). For example true / false; open-ended and multiple choice. True / false questions only ask the participant to choose between two possible options and are therefore very limiting. Open- ended questions may make the test more interesting for the participant; however they tend to require large amounts of time to complete and analyse. Multiple choice questions

Page 113 of 308

situations’. Therefore relative to the time constraints and the other factors discussed above

the test was comprised of short answer questions. Questions were very simply worded, unambiguous and focussed. Multi-phrase sentences were not used as these were considered as too complex for the age group. Simple sentences were used with as few words as possible. Vocabulary that could be interpreted in different ways was also avoided. The use of conjunctions such as ‘and’, ‘but’, ‘except’ and ‘or’ were also avoided

as these words imply a second concept which would be confusing to the participants. It was also considered important that the children should not be overwhelmed with words when attempting to answer the question. The test was reviewed by an expert in the GHS system for wording simplicity, usability and accuracy and the expected correct answers.

Rating and coding of test responses

Expert judgement was required as to the level of correctness and in order to rate the responses to the intervention test. The categories for coding were:

Correct: Is fully consistent with the intention or the meaning is identical with the GHS system. Answers not totally the same as the GHS meaning are included in this category but the information given would be sufficient as the basis for a safety action and / or precaution.

Incorrect: There is very poor relation to the intended GHS meaning or is completely wrong.

Crucial Misconceptions: Indicates a comprehension contrary to the intention of the GHS system. Such a crucial misconception may result in a dangerous behaviour or action.

Page 114 of 308

3.10 Sample

3.10.1 Sampling

Two primary schools agreed to participate in this study and all of the participants for this part of the study were pupils aged between 7 and 11 years old within these schools. All of these pupils, and teachers, were not known or related to the researcher. The schools were small Church of England schools where the head teacher gave the consent to take part in the study. Therefore these are samples of convenience. Due to the small size of the schools, this is both an advantage, as the intervention can be done within one session, and a disadvantage as the number of pupils in each age band are small. The schools were semi-rural village schools and consequently may not be representative.

3.10.2 School setting

All the data gathered from the school intervention teaching and testing was coded. The name code took the format Sx/y where S denotes school, x denotes school number and y denotes participant number. For example code S1/23 denotes school 1 and participant 23.