Modifications Addressed by Content Validity

Overall, the feedback received was very positive with many of the reviewers commenting on the depth and breadth of data that would be generated by this survey. As the foundation of the survey is based on Hackman and Oldham‘s (1975a) original JDS, the approach to developing insights into the CJD, CPS and the AO is strongly supported in the literature and thus wasn‘t raised as a concern by the review group. Some issues that were identified include:

 One whole question from section one of Hackman and Oldham‘s original JDS had been omitted. This was an important omission to discover as it was one

153

of the criteria that fed into the calculation of the Motivating Potential Score (MPS).

 One of the explanatory notes on the Likert scale for section 3 was typed

incorrectly creating polarised responses to those that would have been intended by the student completing the survey.

 Some suggestions were made on the use of vocabulary to make

understanding a little easier and the correction of some minor typographical errors were observed.

 Two suggestions were made, in the interest of brevity, to omit some sections

of the JDS that generated information that may not be directly relevant to the aims and objectives of this research. These suggestions were to firstly omit the section which explored ‗feedback from the work itself‘ as this was intended by the authors (Hackman and Oldham) to develop insights into the need for work redesign rather than work as education. In addition, another reviewer observed that questions related to job security may not be a goal or motivator to short term student internships.

These were all logical suggestions and valuable to the development of a pilot test which would follow. The comments relating to the omission of specific questions were weighed against the cost (in terms of extending the instrument) verses benefit of leaving them in. It was decided that although these may extend the time taken to undertake the survey by a minute or two and the results gained from these sections may not be fully utilised in this study, it is the intention of the author to undertake

154

further research in this area in the near future and thus these may be considerations that may factor into a different research design. As a result, they were not omitted from the survey.

4.3.3. Reliability

Having undertaken the initial tests on face and content validity that relate to the JDS data collection tool and determined that the revised measure would provide an accurate reflection of reality (Brotherton, 2008); it then became important to test the measure for reliability. Common definitions of reliability suggest that its purpose is to seek to outline the consistency associated with the data collection tool when it is used over a period of time, with the same subject groups, under the same conditions (Joppe, 2000).

According to Brotherton (2008), determining the reliability of a measure can be accomplished in two ways. The first is to adopt a measure that has been successfully used in other studies where the author(s) have demonstrated its value and reliability by developing appropriate results. Over the years, as increased interest in the design and re-design of work has become more important, researchers from a number of disciplines have sought to test and re-test the reliability of Hackman and Oldham‘s (1975a) JCM and the accompanying JDS.

Although some of these studies suggest minor modifications and improvements to the model, the positive indications proposed through empirical tests in education (Lawrence, 2004; Van Dick et al., 2001), penal facilities (McDowall-Chittenden, 2002), retail students (Feldman and Weitz, 1990; Paulins, 2006), hospital workers (Lee-Ross, 1999), hotel workers (Lee-Ross, 1993, 1998a, 1998b, 2002, 2005) and

155

business studies students (D‘Abate et al., 2009; Rothman 2003, 2007) all indicate its reliability as a measurement tool.

According to Brotherton (2008), the second determination of reliability can be achieved through a process of statistical analysis. In addition to the observational evaluations outlined above, a number of other authors have sought to test their research instruments (to determine reliability) by using a plethora of more sophisticated quantitative methods. The recommendation in many research methods texts (Brotherton, 2008; Field, 2009; Gonzalez, 2009; Saunders et al., 2007; Shavelson, 1996; Veal, 2006) suggests that internal reliability tests should be undertaken on the primary research data collection instrument at the pilot study stage, before using it in a much broader capacity in the full study. By testing the reliability of smaller data sets, it is possible to determine if consistent results are produced and thus conclusions can be made that the measures are, indeed, reliable (Brotherton, 2008).

Although the JDS has previously been empirically tested in other studies, the modifications made to the survey (as a result of the conceptual framework, research design and content validity), led to the exploration of additional reliability tests in order to enhance the rigor within this study. The textbooks, journal articles and web resources dedicated to the area of quantitative analysis and statistical testing typically offer the reader insights into a number of methods used to estimate reliability. After review, it was determined that the internal consistency reliability test is the most appropriate way to proceed based on the design of this study. Further examination of these methods (split-half adjusted, Kuder-Richardson formula 20 (K-R20) and Cronbach‘s alpha coefficient) was undertaken to determine the most

156

appropriate reliability test for this project. Due to its prior use by both Hackman and Oldham (1975a) and other authors (Lee-Ross, 1993; 1995; 1996; 1998a; 1998b;

1999; 2002) Cronbach‘s alpha coefficient was selected and is discussed in detail in Section 4.10.2.1, Page 167 of this methodology. Once a decision was taken to subject the JDS to a second stage reliability test (using Cronbach‘s alpha coefficients), the modifications were made to the data collection instrument, based on the outcomes of the content and face validity tests. However, the statistical reliability tests can only be run after the pilot study has generated a quantitative data set to analyse.

Having outlined the rationale for the use of a modified JDS in the first phase of this study, it is now important to present insights into the makeup and characteristics of the population that will be used to conduct the research. In addition, some background to the university that will be used as a case study is also offered.