Test Analysis Procedures

On the advice of Veal (2006), the final stage of the pilot testing considerations is to test the data analysis procedures. This recommendation provides an opportunity to use the completed questionnaires, generated through the pilot study, and test the

167

usefulness of the data analysis techniques considered for this study. By taking the completed surveys (n=31) and testing their validity in a trial run situation, opportunities exist to identify any potential issues that may arise at advanced stages of the study, thus saving time and resources.

The statistical testing procedures undertaken on the pilot results include a comparison of measures such as correlations of CJD to CPS and AO and some linear regression models in SPSS. This allowed an opportunity to see if all relevant information required to run these calculations were present in the survey. In addition, reliability tests were examined through the use of Cronbach‘s alpha coefficient (Section 4.10.2.1, Page 171). The outcomes of these identified some issues with low alpha coefficients relating to task significance and dealing with others meaning that students weren‘t answering some questions in a consistent way. After discussions with some student participants from the pilot sample, some of the wording was changed to clarify the objective of the questions relating to these CJD in order to obtain a higher reliability coefficient in the main survey. This resulted in the use of bold font on parts of the questions to ensure respondents read the intention of the question correctly and their answers were uniform across the range of questions relating to measuring the job characteristics of their internships. Other statistical tests run involved multiple linear regression models to observe the impacts independent variables (Core Job Dimensions) had on dependent variables (Critical Psychological States and Affective Outcomes).

The pilot test was conducted in a classroom with two separate groups of students (n=31) where the researcher and field agents could offer some context to the study before distributing the surveys. This was a deliberate strategy so verbal feedback

168

could be given immediately and issues identified. In addition to a couple of minor concerns raised over use of words, vocabulary, and reliability of responses, the two main issues identified from the pilot test were:

 Better instructional guidance was required for those students who had already

completed an internship ensuring their responses were orientated to the one experience rather than a combination of those already undertaken.

 Further guidance would be required to those students whose internship

rotated around multiple departments so they could be instructed to offer a general perception of their overall experience rather than have it isolated to just one role within their employment.

These both seemed logical and important changes to assist the student answer the questions with a greater clarity and uniformity. Once these had been addressed, the second round of pilot testing was undertaken on another small group of students in a classroom setting (n=12). These students contained 3 individuals from the original test cohort along with 9 subjects who were undertaking the survey for the first time.

Although unplanned, this worked out very well as it allowed those who were taking this for the second time, an opportunity to comment on the improvements made.

The second pilot test process was completed smoothly and resulted in only one minor suggestion to the modified JDS. This recommendation centred on providing a brief description for some of the management competencies (outlined on page 18 of the survey – see Appendix 6) to ensure students, who may be unfamiliar with the terms, responded appropriately. The change was made and it was concluded that no further pilot testing was required. To that end, preparations were made to

169

commence the process of proctoring the survey to members of GVSU‘s HTM student body over a period of approximately 9 months.

4.8. Distributing the Modified Job Diagnostic Surveys

Due to the length of the modified JDS and in order to maintain a consistent approach to administering the questionnaires, Field Agents will be used to distribute these personally to student groups where context to the research can be shared before respondents submit their answers. Undertaking the pilot studies in this way had been useful and the results generated had been without incident. This strategy also allows for clarification should questions arise from the student body as they complete their answers. In addition, as the researcher held the role of Department Head within GVSU‘s HTM programme, further considerations were given to potential bias that may result in the responses if the questionnaires were distributed personally. It was feared that students may not be as candid in their answers (even though it could be completed anonymously) and would mask issues by answering in a more favourable way. In total, 42 course sections of the HTM programme at Grand Valley State University were visited during the fall, 2009 and winter, 2010 semesters.

4.9. Data Analysis Techniques

Research design is a key consideration in the development of a project of this nature. It connects each of the component parts together by demonstrating how these will work in unison to address the thesis‘ aims and objectives posed. Having previously considered research paradigms (Section 4.1, Page 135), data collection methods (Section 4.2, Page 146) and sampling (Section 4.6, Page 159), the final consideration to an effective research design is to review (and justify) the measures and treatments for which the data collected will be analysed. This process