!
Figure 6.1 provides a summary of the measures of task performance used in this study.
!
Figure 6.1 Language production measures used in the current study
!
As described in the methodology chapter, a between-treatment comparison was achieved for each measure with the non-parametric Mann-Whitney U mean rank test using SPSS. The desired confidence interval was set at 0.95, resulting in an alpha of p<.05 for statistical significance. The results for complexity, accuracy and fluency are each
summarised separately at first, in that order, and then synthesised in the subsequent discussion. Effect sizes for these non-parametric tests are provided (see Grisson & Kim, 2012, for a full discussion). Complete data sets for both treatment groups can be found in Appendix 4.
!
!
area measure(s)
complexity (syntactic) clauses per AS-unit sub-clausal AS-unit % complexity (lexical) 1st 2k BNC/COCA %
accuracy (general) error-free AS-unit % average length of error free unit accuracy (error type grammatical errors per token lexical errors per token
6.2.1 Complexity
!
Table 6.1 below shows the the descriptive statistics compiled for the three measures of complexity.
!
Table 6.1 Descriptive statistics: complexity
!
The means and standard deviations were very similar for the measure of <2k BNC/ COCA but appeared divergent for the other two measures. These differences in means were checked for statistical significance and the results of those tests are summarised in Table 6.2 that follows.
!
Table 6.2 Mann-Whitney U results: complexity
!
Original Plays Adapted Plays
(n= 21 groups) (n= 21 groups)
measure mean standard
deviation mean deviationstandard
clauses per AS-unit 0.669 0.135 0.807 0.141
sub-clausal AS-unit % 0.345 0.131 0.26 0.12
<2k BNC/COCA 0.96 0.041 0.964 0.02
measure treatment mean
rank rank sum U Z
Asymp.
Sig ( p =) “effect size”
clauses per AS-unit original (n=21) 16.12 338.5
adapted (n=21) 26.88 564.5
107.5 -2.843 0.004* 0.2438
sub-clausal AS-unit % original (n=21) 25.4 533.5
adapted (n=21) 17.6 369.5
138.5 -2.065 0.039* 0.3141
1st 2k BNC/COCA % original (n=21) 22.5 472.5
adapted (n=21) 20.5 430.5
The descriptive statistics for the three selected measures of complexity showed greater syntactic complexity for the Adapted Plays, due to both a higher rate of clauses per AS- unit and a lower rate of sub-clausal AS-units. Contrastingly, there was no discernible difference in lexical complexity as both treatments were within a half percent of each other. These values were compared with the Mann-Whitney U test and the results of those tests indicated statistical significance for both measures of syntactic complexity, with the difference in clauses per AS-unit in particular achieving significance at the p< . 01 level. Expectedly, the slight difference in lexical complexity yielded no statistical
significance. These results indicate two observable trends in student language production: 1) student groups used more full clauses and multi-clause utterances in the Adapted Plays while, conversely, student groups used sub-clausal utterances more frequently in the Original Plays; and 2) these significant variations in full clausal and sub-clausal use were achieved with the same level of lexical complexity.
!
6.2.2 Accuracy
!
Table 6.3 on the facing page shows the the descriptive statistics compiled for the four measures of accuracy. Out of the four measures, grammatical errors per AS-unit showed the closest equivalence between treatments. Additionally, the results for lexical errors per AS-
unit and the average length of error-free AS-units were quite similar. Only the remaining
measure, error-free AS-units, showed divergence.
Between-treatment results were checked for statistical significance and the results of those tests are summarised in Table 6.4 on the following page.
!
!
!
!
!
!
!
!
Table 6.3 Descriptive statistics: accuracy
!
Table 6.4 Mann-Whitney U results: accuracy
!
Descriptive statistics for the four selected measures of accuracy showed a greater degree of accuracy in the Original Plays for general accuracy (error-free AS-units) and both
lexical and grammatical accuracy. The mean length that AS-units reached before an error
Original Plays Adapted Plays
(n= 21 groups) (n= 21 groups)
measure mean standard
deviation mean deviationstandard
error-free AS-units 78.333 11.629 70.619 12.913
avg. length of error-free AS-unit 3.135 0.677 3.327 0.558
grammatical errors per AS-unit 0.201 0.14 0.271 0.172
lexical errors per AS-unit 0.081 0.047 0.133 0.086
measure treatment mean
rank rank sum U Z Asymp. Sig ( p =)
“effect size”
error free AS-unit % original (n=21) 25.24 530
adapted (n=21) 17.76 373
142 -1.976 0.048* 0.322
avg. length of error-free
AS-unit original (n=21) 18.88 396.5
adapted (n=21) 24.12 506.5
165.5 -1.384 0.166 0.3753
grammatical errors per
AS-unit original (n=21) 18.79 394.5
adapted (n=21) 24.21 508.5
163.5 -1.434 0.152 0.3707
lexical errors per AS-
unit original (n=21) 18.05 379
adapted (n=21) 24.95 524
occurred was roughly equivalent, though the Adapted Plays had slightly higher values. These differences in means were compared using the Mann-Whitney U test, as shown in Table 6.4. The results of these tests indicated statistical significance for the percentage of error-free units, while the differences observed in the other three measures were not significant. Therefore, from the perspective of task performance, student groups in the Original Plays produced more accurate language overall. However, these student groups in the Original Plays did not produce error-free AS-units of a significantly greater size than the groups in the Adapted Plays, nor were their separate rates for grammatical or lexical accuracy significantly less than that of their Adapted Play counterparts.
!
6.2.3 Fluency
!
As explained in the methodology chapter, the nature of fluency in devised theatrical performance is a complicated matter, since the performance that a given audience observes features language that is prepared rather than spontaneous. Thus, fluency as a task performance measure in the current study differs from typical task condition effect studies (e.g. pauses, repetitions, false starts etc…). The selected measure of total number of tokens is discussed in this section, while the task-specific measure of overall theatrical quality of oral performance, a separate measure designed to account for the special nature of fluency in theatre tasks, is discussed separately in the next section (6.3). Table 6.5 below shows the descriptive statistics compiled for the measure of fluency.
!
Table 6.5 Descriptive statistics: fluency
!
Original Plays Adapted Plays
(n= 21 groups) (n= 21 groups)
measure mean standard
deviation mean deviationstandard
As the above table shows, there was a stark contrast between-treatments for the mean number of tokens in the final performances. This difference was checked for statistical significance and the result of that test is shown in Table 6.6 on the next page.
!
Table 6.6 Mann-Whitney U results: fluency
!
As the descriptive statistics for fluency indicated, there was a substantial difference in the length of the plays, with the Adapted Plays averaging almost thirty percent more words per play than the Original Plays. The Mann-Whitney U test conducted for these values showed that this difference in means was statistically significant at the p<.01 level.
!
6.2.4 Summary of results for complexity, accuracy and fluency
!
The results of the Mann-Whitney U tests showed statistically significant differences for
clauses per AS-unit, sub-clausal AS-unit %, error-free AS-unit % and tokens. The differences for
the remaining measures showed no significance. In sum, the Adapted Plays showed a treatment effect for syntactic complexity (clauses per AS-unit, sub-clausal AS-unit %) and fluency (tokens) while the Original Plays showed a treatment effect for accuracy (error-free
AS-unit %).
Skehan (1998, 2003) predicted that as task difficulty increased, there would be a trade- off between complexity and accuracy. Regarding this trade-off, a post-hoc correlation test was conducted on the statistically significant measures of fluency, syntactic complexity, and (general) accuracy using the non-parametric Spearman rank test in SPSS to compare
measure treatment mean
rank rank sum U Z
Asymp. Sig ( p =) “effect size” tokens original (n=21) 15.12 317.5 adapted (n=21) 27.88 585.5 86.5 -3.372 0.001* 0.1961
values across treatments. The results of this test, shown in Table 6.7 below, indicated a very significant inverse correlation between complexity and accuracy in the current study. These results provide further support for the LAC prediction of a trade-off between complexity and accuracy as task difficulty increases. Additionally, no significant
correlations were detected for either fluency and complexity or fluency and accuracy. This further suggests that any predicted trade-off for these theatre tasks excludes fluency as a possible contributing factor.
!
Table 6.7 Results of Spearman’s rho