Quantitative results for reflective cycle two.

Tables 5.15 to 5.22 show the results for the questions from the survey. A total of 67 students from group A and 68 students from group B completed the survey. The students indicated the degree to which they agreed with the statements listed below. A Likert scale was used for Tables 5.15 to 5.21, and, as in cycle 1, a point value was assigned to each response

(strongly agree – 5 points, agree – 4 points, neutral – 3 points, disagree – 2 points, strongly disagree – 1 point). The eight questions are:

1. Studying collocations has been useful.

3. Studying collocations is more helpful than studying individual words.

4. For group A students only: Reading the definitions and example sentences has been useful.

5. For group A students only: Answering questions about the sentences has been useful.

6. For group B students only: Reading the definitions and example sentences has been useful.

7. For group B students only: Doing the 'Fill in the blank' questions has been useful. 8. Each week we studied 12 collocations. That was...

Table 5.15 shows the results for the first questionnaire item.

Table 5.15 Students’ responses about studying collocations Question

(n=135)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

1 Mean Std-Dev 5 4 3 2 1

Group A 3.70 0.92 11(16%) 33(49%) 17(25%) 4(6%) 2(3%) Group B 4.04 0.68 17(25%) 37(54%) 14(21%) 0(0%) 0(0%) Total 3.87 0.82 28(21%) 70(52%) 31(23%) 4(3%) 2(1%) Both groups of learners had an overall positive impression of studying collocations. However, the productive group had a higher average response compared to the receptive group (4.04 compared to 3.70 for the receptive group).

Table 5.16 Students’ responses about their ability to use collocations in conversations

Question (n=135)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

2 Mean Std-Dev 5 4 3 2 1

Group A 3.25 1.03 7(10%) 23(34%) 19(28%) 16(24%) 2(3%) Group B 3.16 0.80 4(6%) 16(24%) 35(51%) 13(19%) 0(0%) Total 3.21 0.92 11(8%) 39(29%) 54(40%) 29(21%) 2(1%) The receptive group and the productive group had a similar average response in regard to their ability to use collocations in conversations. The receptive group has a 3.25 average, while the productive group has a 3.16 average.

Table 5.17 Students’ responses comparing studying collocations to individual words

Question (n=135)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

3 Mean Std-Dev 5 4 3 2 1

Group A 3.52 1.01 12(18%) 23(34%) 21(31%) 10(15%) 1(1%) Group B 3.81 0.80 11(16%) 37(54%) 17(25%) 2(3%) 1(1%)

Total 3.67 0.91 23(17%) 60(44%) 38(28%) 12(9%) 2(1%)

The productive group had a higher average response when comparing studying collocations to studying individual words. The productive group had an average response of 3.81, while the receptive group had an average response of 3.52.

Table 5.18 Group A students’ responses about reading the definitions and example sentences

Question (n=69)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

4

Mean Std-Dev 5 4 3 2 1

3.77 0.79 11(16%) 35(51%) 19(28%) 4(6%) 0(0%) *69 responses (two group B students answered this question by mistake)

The receptive group had a positive impression about reading the definitions and example sentences in the collocation activities. The average response was 3.77.

Table 5.19 Group A students’ responses about the receptive task of answering questions about the sentences

Question (n=69)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

5

Mean Std-Dev 5 4 3 2 1

3.67 0.83 11(16%) 29(42%) 24(35%) 5(7%) 0(0%) *69 responses (two group B students answered this question by mistake)

The receptive group also responded positively to the receptive activity if answering questions during the collocation tasks. The average response was 3.67.

Table 5.20 Group B students’ responses about reading the definitions and example sentences

Question (n=62)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

6

Mean Std-Dev 5 4 3 2 1

3.95 0.71 12(19%) 37(60%) 11(18%) 2(3%) 0(0%) *62 responses (six group B students did not answer this question)

The productive group had a positive impression about reading the definitions and example sentences in the collocation activities. The average response was 3.95.

Question (n=62)

Likert Positive Responses

(%)

Neutral Negative Responses (%)

7

Mean Std-Dev 5 4 3 2 1

4.00 0.68 14(23%) 34(55%) 14(23%) 0 0

*62 responses (6 group B students did not answer this question)

The productive group had a positive impression about the cloze activity in the collocation activities. The average response was 4.00.

Table 5.22 Students’ responses in regard to the number of collocations covered each class

Question (n=135)

Negative responses Positive responses

Negative responses 8

Way too many

Too many Just about right Not enough Not nearly enough Group A 0(0%) 14(21%) 47(70%) 5(7%) 1(1%) Group B 1(1%) 5(7%) 59(87%) 1(1%) 2(3%) Total 1(1%) 19(14%) 106(79%) 6(4%) 3(2%)

Given the tasks and time required to complete them, 12 collocations was perceived to be an appropriate number per week in the students’ opinion. The productive group had a higher percentage of students who responded (87%) that 12 collocations was ‘just about right’ compared to the receptive group (70%).

The audio-recordings were assessed blind for fluency by three judges. The means and standard deviations are shown in Table 5.23. The blind judges used a seven-point scale to judge fluency: a score of 1 was extremely dysfluent and a score of 7 was extremely fluent. Prior to the assessment, the blind judges undertook a benchmark identification session to minimize the chance of inconsistent judgment. The voice recordings used during this session were not part of the later assessment process. A Fleiss’ Kappa test was conducted to measure inter-rater reliability. The observed agreement was measured at 0.375.

Table 5.23 Mean, Standard Deviation, and t-value for spoken fluency assessment Group A (receptive) Group B (productive)

Pre- intervention Post- intervention Difference Pre- intervention Post- intervention Difference M 2.81 3.09 0.28 2.84 3.56 0.72 SD 1.53 1.17 1.39 1.50 t 1.6048 2.9112

Both the receptive group and the productive group had higher scores on their post-

intervention spoken fluency assessment. The receptive group’s post-intervention score (a seven-point scale) was 0.28 higher than their pre-intervention score. The productive group’s

post-intervention score was 0.72 higher than their pre-intervention score. A two-tailed dependent t-test was performed on the results for the productive and receptive groups. There was not a significant effect for the receptive group, t(31) = 1.60,p<0.05. There was a significant effect, however, for the productive group, t(31) = 2.91,p<0.05, with the students receiving a higher score on the post-intervention spoken fluency assessment.

The quantitative results will be discussed in greater detail in section 5.3.9. These results will also be compared with the qualitative results from the open-ended questionnaire item. Overall, these quantitative responses indicate the collocation intervention was

worthwhile in the learners’ opinion, and there is evidence the productive group improved their spoken fluency. Several transcripts of the narrative retells are presented in the following section.

5.3.7 Transcriptions of the narrative retells used for the fluency assessment.