Procedures used to test hypotheses

To test all hypotheses, pretraining, posttraining and follow-up scores were examined using repeated-measures ANOVA with helmert contrasts (Field, 2009; Tabachnick & Fidell, 2001). These contrasts examined the differences between the two groups (training and control) in the amount of change they demonstrated over time (interaction effects). Structured and casual conversation scores were analysed separately as the inclusion of conversation type as a third factor would have taken enough degrees of freedom away from the error term to reduce

power substantially given the small n. The two group by time interaction contrasts provided the most direct test of the research predictions. The first interaction tests if any change from pretraining to posttraining and follow-up is the same in both groups. The second interaction tests if any change from posttraining to follow-up is the same in both groups. An alpha level of 0.05 was used to

determine statistical significance. Main effects and effect sizes were obtained for all measures to provide further evidence as to the effectiveness of training. All analyses were computed using SPSS, Version 17.0 (2008).

2.8.2.1 Repeated measures ANOVA with helmert contrasts.

Repeated measures ANOVA is most suited for this study as the same people (i.e. paid caregivers and people with TBI) are repeatedly assessed at more than two points in time (i.e. pre, post and follow-up). It therefore avoids the problems associated with conducting multiple paired t tests such as inflation of type I errors. There are a number of assumptions that need to be satisfied when conducting repeated measures ANOVA. Firstly, scores need to be normally distributed (see section 2.8.1.1). Secondly, the rule of homogeneity of variances cannot be violated. Thirdly, participants should be independent of one another and finally, only people with scores present at all time points are included in the analysis (Howell, 1997; Marston, 2010)

The use of contrasts enables a researcher to locate where the differences between groups lie and whether the means of the groups follow a particular pattern. Essentially, a contrast is a comparison between two means (or groups of data). Contrasts can be divided into orthogonal and non-orthogonal. Whilst non- orthogonal contrasts are a follow-up analysis to ANOVA, orthogonal contrasts are an alternative way of conducting an ANOVA. In that sense, it takes the data further than repeated measures ANOVA and is a powerful contrast type when testing specific hypotheses or comparisons between means. Helmert contrasts compare the mean (of a group of data) and compare it to the mean effect of all subsequent categories which occurs for all groups of data except the last as there is no data to compare it against. In general, with k treatment means, sets of only (k -1) contrasts are possible. In the current study, there are three categories

of data (pre, post and follow-up) which will result in two rows of coefficients or two contrasts. The helmert contrasts will test whether any interaction between means is the same for both groups (i.e. control and training). The contrasts are constructed as follows:

1. We compare the mean of the pretraining scores with the average of the other two means (i.e. posttraining and follow-up).

2. We drop the first mean and compare the second mean (i.e. posttraining) with the third (i.e. follow-up). The second interaction tests if any change from posttraining to follow-up is the same in both groups.

The set of helmert contrasts can be represented by two rows of coefficients as shown in Table 2.4 (Field, 2009). A set of contrasts has the property that each is independent of the other. In other words, the first contrast does not affect the second, because the first mean is not involved in the second contrast. Taken together these helmert contrasts make up a set of orthogonal contrasts. To ensure their independence (or orthogonality) the sum of the products in any two rows is zero.

Table 2.4

Rows of Coefficients for Helmert Contrasts

Group Contrast 1 Contrast 2 Product

(Contrast 1 x 2)

Pre -2 0 0

Post +1 +1 1

Follow-up +1 -1 -1

Total 0 0 0

Use of orthogonal contrasts reduces the number of multiple comparisons conducted (from three comparisons of pre, posttraining and follow-up scores to two). Dealing with type I and II errors is difficult with such a small sample size. A Bonferroni adjustment (i.e. using a stricter p value based on the number of planned comparisons) would have protected against type I errors, however, it may have led to the increased likelihood of a type II error (acceptance of the null hypothesis when it is false). When there is an issue with multiple comparisons, Siegl (1990) suggests that a good compromise to the situation would be to include the results of interactions and the individual estimates (i.e. means and standard deviations) to assist the reader to understand the results being discussed. A researcher should also refrain from examining non-significant results (which was not done in this study) and examine the significant results for individual differences as this is likely to increase the chance of a type I error (Siegl, 1990).

2.8.2.2 Main effects.

In two-way experiments main effects can be reported which examine the effect of one variable and ignore the effect of the other. Main effects can be a useful approach to understanding your results when you do not have interaction effects. For this study, main effects will be evaluated for group and time

(independent variables) for each dependent variable. In other words, were there improvements in interaction on particular dependent variables irrespective of group and did differences exist between the two groups irrespective of time for particular dependent variables?

2.8.2.3 Effect sizes.

Effect sizes were also calculated which quantify the size of the difference between two groups and refers to the magnitude of treatment and practical significance of the findings. Effect size attempts to explain the total variance within the data. One measure of effect size is partial eta squared which provides the proportion of variance in a dependent variable that is not explained by other variables in the study. Each of these measures provides a value from 0 to 1. A partial eta-squared of less than 0.01 represents a small relationship, less than 0.06 a medium relationship and greater than 0.14 a large relationship between variables (Sink & Stroh, 2006). However, caution must be taken when

interpreting the results of the current study as a small sample size and unusual scores in the sample can easily distort estimates (Strube, 1988). For example, estimates of eta squared can be inflated with small sample sizes.