Part A of the Study

As Head of the Learning Support Faculty I conducted an analysis of student performance and noted that underachievement in tests and exams often resulted from misunderstanding the questions or answering inappropriately. I had shared with staff that this was dependent on students’ fundamental capacity to assemble information, process and organise ideas in narrative form.

i) Identification

The support staff observed that classroom practice showed that subject staff often misinterpreted difficulties as ‘misbehaviour’ although such students did not appear to understand concepts and respond in lessons. We agreed that we needed more knowledge of students’ competences in order to scaffold their learning.

ii) Planning

Narrative competence was being neither monitored nor addressed in the school and the support staff had observed that the time students spent talking, with educators and peers was minimal. The staff had shared their need for assessment materials, asking what I had previously used and recommended to develop students’ communicative competence. As an integral part of my annual practice, assessment or ‘screening’ tests were given to all 200 new students entering Year 7 to provide base-line information on their cognitive, linguistic and narrative competence and writing production abilities.Together the staff and I questioned the level of the children’s narrative competence in Year 7, thus beginning the first cycle of action research reported in this study as Part A: ‘An assessment of the

120 students’ communicative competence on entry to secondary school’. [This later led to Part B: ‘A comparison of interventions to improve it’.

iii) Implementation

The outcomes of the assessment of communicative competences test in Year 7 formed Part 1 of the action research cycle encompassing research into students’ levels of ability on entering secondary school.

Participants: The assessments involved the 200 Year 7 students, male and female, with an average age of 11 years 4 months.

Materials: They were assessed by: i) giving answers to a quiz of everyday ‘cultural’ knowledge; ii) providing an account of a familiar process; iii) re-telling a short story. Following testing, interviews with participants were conducted to gain qualitative data. Details are presented below.

Cultural knowledge test

This consisted of 10 quiz questions of cultural knowledge, at an age and social appropriate level, delivered orally by each form tutor to the class. One of the questions was designed to prompt a response to ‘what happened’ in a (then) recent global event that also required a short narrative typically composed of the subject, object and actions in a sequence, (see Appendix 1). A teaching assistant supported students requiring spelling support where needed when students recorded their answers, although this analysis focused on conceptual understanding.

Scoring: Each response was coded as either:

 Fully correct;

 Partially correct - where sufficient information is given to indicate some degree of knowledge;

 Incorrect - where insufficient or no information is given.

Providing an account of a familiar process

Participants were asked, on a one-to-one basis, to explain a familiar process by providing step by step instructions. In this case the process was ‘how to make a cup of tea’ since this was known to be something that girls and boys in local families did regularly. This research method relates to ‘elicited narrative’ (Berman and Slobin, 1994) and does not rely upon listening and recalling information but draws upon secure personal knowledge. The aim is to prompt participants to construct an appropriately sequenced, extended

121 narrative, i.e. one that contains several propositions. The assessment is not time-bound and participants are allowed to re-consider and correct their accounts. Responses were noted on the recording sheet.

Re-telling of a short story

Participants were required to listen to a short unknown story composed to contain 9 propositions of the type used in instructions, presentations and explanations. A summary of its content is that it is about a boy’s birthday and how his sister is preparing for it. The story re-telling process is not reliant on participants’ general knowledge, understanding or longer-term memory abilities. Instead it measures the research participants’ capacities to make sense of, re-structure and re-tell the story they have just heard in narrative form. The test draws upon capacities that are relied upon in schools. For example, when a teacher requires students to take in, make sense of and act upon a sequence of information, such as carrying out a series of tasks in science or design. A further example is in a history lesson when required to listen to an account of a series of past events, recall and research them. The story re-telling test draws upon both cognitive-linguistic processing and the construction of a narrative. Research indicates that a story re-telling allows participants freedom to respond and provide information at their own levels of thinking, language and narrative. Narrative story re-telling is also known to be a reliable indicator of students’ potential ability in reading (Bishop and Donlan, 2005).

Scoring: A rating of ‘partially correct’ was given for naming of propositions (items and ‘ideas’) in the procedure but without a full or correct sequence; it was also awarded for the order of those propositions given but where the list was incomplete. A rating of ‘fully correct’ was given for providing intact information (i.e. a full account). A rating of ‘incorrect’ was given for responses where both the propositions were incomplete and the sequence was incomplete or inaccurate.

Ethical Considerations

The tests and interventions in Part A and B of Study 1 took place within the parameters of the school’s curriculum, practice and action research. The participants were with existing members of staff, therefore no special safeguards were sought at that stage. At the point at which it was decided that the results of the Year 7 testing and results of the studies would be written up for out of school purposes the school staff, students, governors and parents were consulted and their permission obtained regarding publication beyond the school, whilst ensuring that the school and students were anonymous.

122 I emphasised the importance of explaining how information about both projects was to be analysed and used at the next whole staff meeting. Sensitivity was taken ensuring children who were volunteers in interviews did not feel pressurised into participating and that they were aware that they could withdraw at any time. Parents were informed of this also in the respect of giving consent. Care was taken to ensure more vulnerable participants were scaffolded. One staff member commented on the right for all children to have assistants acting as scribes where appropriate so all had equal opportunities, but this was part of school practice. Following the staff meeting I met with my own team who reflected on their perceptions of the staff meeting and clarified points. A sense of ownership and joint enterprise of two cycles of school improvement (Studies 1 and 2) resided in the intrinsic worth of the research projects and the fact that all activities were offered on a rotation basis.

Regarding informed consent, in my line management with the Principal I initiated Study 1 then Study 2 as integral to the whole school development plan. The proposal was placed on the Senior team agenda and ratified the following week at the governors’ meeting. The Year 7 cohort was taken to the university for the ‘Go’ project a part of the following Intervention Study (Cycle B). Parents gave informed consent for all children to be involved, which both enhanced all participants’ self-esteem and raised the school’s profile in the wider community.

For reasons of ethics and anonymity the school is referred to in this thesis as ‘City School’. No details or names are given. All were aware that the research project was designed to gather data on what students encountering difficulties would find supportive in order to improve interaction and learning through exploratory talk.

4.4.1.1 Analysis

The results are shown here. It was in response to these assessment findings that the second cycle of action research was planned.

4.4.1.1.1 Analysis of cultural knowledge quiz

The percentages of participants achieving ‘Fully correct’ answers to the 10 questions of test [a] Cultural Knowledge in the pre-test are ranked in the table below. They are shown in descending order, from the questions answered most successfully to those answered least successfully.

123

Table 4.1 - Percentage of correct answers by the Y7 school cohort to the Cultural Knowledge pre-test questions, presented in rank order (N=200)

Question Fully correct

responses

1. Where does milk come from? 83%

2. How many colours are there in the rainbow? 66%

3. Write a number less than 9083 66%

4. What is the capital of England? 61%

5. What do you call your Mum’s dad? 41%

6. How many countries are in the British Isles? 32% 7. Who is the Prime Minister of Britain? 29%

8. Which continent is India in? 27%

9. What are the points of the compass? 13% 10. What happened on September 11th, 2001? 1.5%

As the above scores show, the complexity of the questions is reflected in the results. It might be expected that the simplest questions, listed as 2 to 5, could be answered by all children eleven years at the end of Year 6, however the findings show that these questions were answered successfully by between no more than three quarters of students. 17 per cent did not answer ‘where milk comes from’, even allowing for ambiguity in the question and accepting some responses as in a ‘carton’ or the ‘supermarket’. Bearing in mind that this figure was obtained across a full cohort of 200 students it can be seen that the general knowledge level of the entire Year 7 cohort was very low.

The content of questions 6, 8 and 9 (the British Isles, major continents, points of the compass) might be considered as a fundamental base for students learning more about geography in their secondary school, indeed this material features in the primary National Curriculum. Success rates for these questions from 32 to just 13 per cent are very low in this context, as in 13% not knowing the compass points.

To put these scores for the school cohort into context, the table below compares the mean of the overall total scores of the cohort to those in the original group piloting the test devised by Sage (2000). Scores represent the total number of questions answered correctly out of 10.

Table 4.2 - Cultural Knowledge pre-test total scores of the Year 7 students compared with pilot group participants

Group Number in

sample

Mean score of

10 SD

Year 7 research group 200 5.4 12.7

124 While the mean total score obtained with the original pilot group was 8.2 answers correct out of 10, as shown above, the mean total for the research group in the school for the General Knowledge test was just 5.4 answers correct out 10. Furthermore, a wide standard deviation of 12.7 for the research group suggests a large cluster of very low scores which suppressed their overall mean. The standard deviation within the pilot group was far lower at 7.7. As might be expected there is a highly significant difference statistically between the scores of the research group and the pilot norm (p =  0.003).

4.4.1.1.2 Analysis of account of a familiar process: How do you make a cup of tea?

In this test the number of relevant components provided by participants in their oral narrative account of the familiar account is totalled (propositions, correct sequence, full narrative). To put the scores for this age-group into context, the original pilot group was a total score of 7.4 compared to 3.4 in the research group. The table below compares the mean of the overall total scores of the school cohort of 200 compared below with this norm.

Table 4.3 Research participants’ pre-test scores when providing a short account of a familiar process compared with pilot norms

Group Size of group Mean Score SD

Year 7 research group 200 3.4 2.9

Original pilot group 100 7.4 5.2

As the above results show there was also a highly significant difference in this test between the means of the two groups (p = 0.002), with the research group scoring less than half that of the pilot group. The research group revealed substantial difficulties in providing a full narrative for a sequence of events that they experience regularly in their daily lives. The research group showed a small standard deviation of 2.9 compared to a deviation of 5.2 among the pilot.

It is unlikely that the poor scores from the research group were due to participants not understanding or not recalling how to make a cup of tea. Bearing in mind the narrative difficulties demonstrated in the results shown above it may instead have been due to the participants’ lack of experience in structuring and expressing their thoughts in narrative form. Qualitative interpretations explain examples later. This is a competence further explored in the following test.

4.4.1.1.3 Analysis of story re-telling task

In calculating the test results the total number of relevant components provided by participants in their re-telling is totalled (propositions, correct order and syntax. To put the

125 scores for this age-group in context, in the original pilot (N=100) gave an average score in each section of 4.3; this contrasts with an average for the research group of 2.4, almost half. This measure of narrative ability is captured by the scores presented in the table below. The results show comparisons of the mean scores of the research group of 200 with the mean scores of the pilot group.

Table 4.4 Research participants’ pre-test scores when re-telling the story compared with pilot norms

Group Total mean no. of propositions expressed (SD) Mean of no. of propositions in correct order Mean of no. of propositions with correct syntax Mean of no. of propositions

plus intact (full) information

Year 7 research

group (N=200) 2.5 (7.6) 2.4 2.4 2.3

Original pilot group

(N=100) 4.5 (12.47) 4.2 4.2 4.1

As the results show, the overall ability of the Year 7 research group to listen and retell the short story was significantly lower than the pilot on this specific test. Scores of almost fifty per cent lower show that they had serious problems in taking in information and assembling the narrative coherently. Out of a total of 9 propositions the research group’s mean score was 2.5, with a standard deviation of 7.6. This compares with score of 4.5 and a greater standard deviation of 12.47.

The overall results show a highly significant statistical difference between the research group and the norm at a level of p = 0.0001. This suggests that very little information from the story was able to be retained and retrieved by participants in the group.

4.4.1.1.4 Qualitative analysis and overview of responses across tasks

Regarding the 3 initial tasks of eliciting narratives through (i) cultural understanding, (ii) description of an everyday event and (iii) story retelling, I have selected (i) cultural understanding, to consider the responses of 3 students’ in depth. The 3 students are Hitan, Tamsin and Carla. (see Appendix 3). Of particular interest are the responses to Question 10: ‘What happened on 11th September 2001?’ All 3 students give similar responses to this question in a single phrase. Hitam stated ‘the twin towers came down’, Tamsin responded ‘The towers got knocked down’ and Carla replied ‘The twin towers came down.’ All 3 answers are partially correct but do not provide a contextual or extended narrative.

Compared to the pilot test, over 98% of students gave partial answers, for example, ‘the twins got burnt’ or ‘Paris tower fell down’. Such answers may indicate a lack of knowledge

126 of cultural events regarding location, the what, who, how, where and when of events, for example the association of the Eiffel tower in Paris may have indicated that the student was searching for an association with a tower, (see Appendix 4). The reply ‘it’s not my birthday’ indicates perhaps a certain lack of awareness that school tests do not usually relate to individual, personal information but to more cultural matters.

To fully access their cultural knowledge the question could perhaps have been reframed to ask a series of answers in order to support the assembly of events in a coherent narrative. Similarly, perhaps the students could have been prompted. Some of the support assistants prompted the students after they had handed in their scripts and one student replied ‘Oh yeah.. I meant the towers that got knocked down - on the news. I just put osamabin’ The fact that one student responded about their birthday may have indicated this was more meaningful so it was relevant in a sense, however the disembedded nature of school learning implies that the question referred to shared, cultural knowledge. Just three students, totalling 1.5% of the cohort, managed to give a complete answer, (Appendix 1). One responded ‘Bin Laden bombed the twin towers in a terrorist attack and also threatened the Pentagon.’ However the test here was of the ability to construct a narrative and the results of this show that for many of the students this may have been a struggle.

A second observation relates to providing an account of a familiar process, designed to prompt a narrative that was less likely to depend upon either ‘external’ knowledge or even short term memory. Hitam, Tamsin and Carla all explained how to make a cup of tea (Appendix 3). Hitam explained his tea making but did not state the step of removing the tea bag, Carla did the same, unless they did not forget the step and it correctly indicates the way they both make tea. Tamsin gave an extended and elaborated account consisting of 12 components. Her vocabulary is graphic as she was able to provide visual details of how the spoon is used: ‘scoop a spoon of sugar in the cup’ and gave visual and technical information of ensuring ‘the spoon hits the bottom so the sugar at the bottom floats around.’ This suggests Tamsin has much concrete experience of making tea and is able to articulate her real life experiences meticulously. A major contrast in her performance is seen in the story retelling however. She gives just one phrase of the girl: ‘in the dining room writing an essay.’ This is accurate but consists of only one sentence and idea, missing out 90% of information; the remaining nine propositions.

In the story retelling task, Hitam’s story reflected 72% accuracy, in contrast to his short account of tea making, however it is noteworthy that he omitted key references to the story context, as in the time of year ‘May’. The main characters’ identities were also

127 omitted, ‘Luke and Helen’, as were the location and time of the main event, the ‘party disco at 7.45’. However, he conveyed the meaning of the story by substituting ‘loud’ for ‘deafening’ and ‘back garden’ for ‘back yard’ and ‘tops’ for ‘shirts’, but substantially