LMCV is the extent to which the researcher can be sure that programme effects in this case, were caused by the programme mechanisms, all other potential explanatory mechanisms having been accounted for, as far as possible. As Cook and Shadish (1994, p. 561) suggest, it is better to rule out threats to LMCӦ ‘by design’. There are epistemological difficulties in accounting for alternative interpretations for programme effects, particularly because the contexts in which critical realist social scientists study phenomena are so complexly ordered. In reality, there may be many further threats, or ‘mechanisms’, that either remain unidentified,
89 or are impossible to account for. Campbell (1957) suggested that the researcher need only be concerned with plausible threats, as they relate to the study population, context, and time.
Hence, careful consideration was given to LMCV in designing this quasi-experiment. As some of the previous research in this area demonstrates that these threats were often not considered, with reported effects that may just be due to chance, the strength of the design here took on a new significance. Cook and Campbell (1979) identified several major threats to internal validity, or LMCV, as applied here. However, the following six were identified as having a potential impact upon LMCV within this study:
1. History: When an observed effect could be explained by an unrelated event which takes place between the pre-test and the post-tests;
2. Maturation: An observed effect which could be explained by the respondents ‘growing older, wiser, stronger and more experienced’ (Cook and Campbell, 1979, p. 52) between the pre and post-tests;
3. Testing: Cook and Campbell (1979) asserted that respondents may become familiar with the testing instrument, in this case, McLafferty’s Attitude Scoring Tool, enhancing their ‘performance’ in completing it, falsely making it appear as though their attitudes had improved either immediately following the programme, or over the course of the study; 4. Selection: Students in both the programme and CGs self-selected to be
included in the study, which can bring with it a number of biases, which must be acknowledged, and
5. Mortality: Participants in a study withdraw from it, prior to completion. This can cause the composition of the groups to be altered, and no longer comparable.
6. Compensatory Rivalry: Participants in the CG try harder to achieve better scores than the PG, to compensate for not having access to the programme (Cook and Campbell, 1979).
After having decided upon the use of an interrupted time-series design, acknowledging its strength within social science experiments (Wagner, et al., 2002), the next significant design decision was to include the CG, from a different
90 University, whose participants did not know about the PG, or attend the educational programme. The inclusion of a group for the purposes of comparison helps to rule out ‘history’ and ‘maturation’ threats; observed movement in the time-series data between the pre and post-tests would probably be observed for both groups, were a change in attitudes down to an historic event, experienced by both groups. Inclusion of a CG also accounts for ‘maturation’; again, considerable movement in the time- series data for either or both groups would be expected, were it due to the ‘maturation’ of participants (Cook and Campbell, 1979).
A number of issues need to be considered with regard to ‘testing’ threats to LMCӦ. It was a particular concern in this study, because the attitudes of both groups were measured over a long period of time, on repeated occasions. Repeated measures of attitude in the pre-programme phase were useful in building a picture of the students’ attitudes over time, and in ruling out ‘history’ and ‘maturation’ threats to LMCV, as already discussed; again this was not accounted for in previous research. But this could have pre-sensitised students to the subject being measured, and made them familiar with the tool being used to measure it (Cook and Shadish, 1994). Although sampling issues are discussed in detail later on in this chapter, it is worthy of brief mention here, as the technique was also instrumental in controlling for ‘testing’ threats. Pragmatism dictated that the programme and CGs could not be randomly selected in the truest sense. The groups from which the samples were selected were 'naturally occurring' in that the students were arranged into groups upon commencing their chosen nurse education programme. Respondents in each sample self-selected for inclusion in the study. The attitudes of a sub-sample of these were measured at each data point (n = 10), and were identified using a random number generator. So, although the study groups were not randomly selected, the sub-sample from these main groups at each data point was. This meant that individuals did not necessarily have to complete McLafferty’s Attitude
91 Scoring Tool (MAST) at every data point, reducing their chances of becoming familiar with, and pre-sensitised to it.
ӧith regard to ‘mortality’ and ‘selection’ threats, as already explained, the attitudes of a sub-sample from each group were measured at each data point. This technique had utility in that it ensured sufficient data were generated, without having to repeatedly test every potential participant at each data point (Jaeger, 1997). Hence, it was important in ensuring that no particular respondent was over-burdened by testing, whilst allowing a picture of the attitudes of both groups toward working with ill older people to be gained. Part of the success of this quasi-experiment lay in ensuring that whoever consented to participating in the study, stayed in the study. So, although sub-sampling reduced the potential sample size at each data point, it was considered to be a worthwhile design decision in ruling out ‘mortality’ as a threat to LMCV.
Bias related to the selection of the groups needs to be considered. As Parahoo (2006, p. 269) notes, students on a course are part of a ‘captive population’. This is discussed more fully in section 3:8, on ethical considerations.
It is in these ways that this study attempted to overcome the perceived methodological weaknesses of previous research in this area, by design. Demographic information was also collected from each participant, at each data point, in order to inform subsequent analysis, by demonstrating the comparability of the PG and CG, throughout.