A second expert panel participated in this phase to rank in importance, on a scale of 1-10, the best practice statements and composite indicators obtained at the end of Phase 2, and to give their reasons for their rankings. Following this survey, six participants participated in individual, semi-structured interviews, to elaborate on their reasons for selecting the best practice and composite indicators in the second survey. The best practice and composite indicators for quality management in Masters courses in Educational Administration in Private Higher Education Institutions in Thailand were established.
Step 1: Selection of Panel 2
Thirty-four private higher education institution administrators and instructors in graduate schools of Educational Administration and related fields at private higher education institutions in Thailand were invited by letter to participate in this single round survey. This group comprised Panel 2. The survey was designed to test the practicality of the composite indicators and their variables that were constructed by the Delphi method as part of Phase 2. Step 2: Survey of Panel 2
In this step, Panel 2 was surveyed by mail, and asked in the fourth pencil- and-paper questionnaire, to rank in importance, on a scale of 1-10, the composite indicators and their variables and to give the reasons for rankings. Space was also provided after and beside each item for the convenience of the respondents to fill in their comments after their ratings and for them to give reasons for their rankings.
The rank-ordered listing of utility of items obtained from the third round questionnaire responses were used in Phase 3. The third round questionnaire ended the Delphi survey for this research, because consensus among the group of panel experts had been reached: a majority of respondents had adjusted their responses within the criteria, and three mailings are sufficient in order to arrive at consensus (Brooks 1979, quoted in Custer et al., 1999, 2). Three rounds proved sufficient to attain ‘stability in the responses’ (Turoff & Linstone, 2002, 223).
The refined data obtained from the fourth questionnaire involved statistical treatment and were computed using Microsoft SPSS v.11 software. Themedian and quartiles (the fourth, third, and first quartiles), the mean, and the aggregate score for each item of the fourth questionnaire responses were determined and the items ranked in order of importance, from highest to lowest, according to their median scores. Those items with the same median score were ranked according to their mean score, as before. Details of the analysis of all data obtained in this step are contained in Appendix A, Tables A1-A7.
Step 3: Semi-structured interviews
In this step, six participants from the second survey, who had been selected at random, were invited to participate in individual, semi-structured interviews in order to give their recommendations, and to elaborate on and reconsider their reasons for selecting the composite indicators and their variables. Prior to the interviews, they were given a statistical analysis of responses to the fourth questionnaire; a summary of these responses may be found in Appendix A, Tables A8-A10. The six participants in Phase 3, Step 3 of the study, were represented by a president, an assistant president, and directors and deans of six Thai universities and colleges.
The respondents’ median and mean scores ratings of items on the fourth questionnaire – relating to the utility and usability aspects each item –
TABLE 3.3 MEAN SCORES FOR UTILITY AND USABILITY CATEGORIES
Mean Group Score Utility/Usability
8.21 – 10.00 Highly desirable/Highly feasible 6.41 – 8.20 Desirable/Feasible
4.61 – 6.40
Neither desirable nor
undesirable/May or may not be feasible
2.81 – 4.60 Undesirable/Probably unfeasible 1.00 – 2.80 Highly undesirable/Definitely
unfeasible
treated nominal scales as interval data. The items were sorted on the basis of their desirability, from highest to lowest mean score, and treated in the same way as for the third questionnaire. The composite indicators and their variables that had a utility and/or usability group mean score above8.20 were selected for generating the set of best practice statements.
The composite indicators for quality management in Masters courses in Educational Management in Private Higher Education Institutions in Thailand were established as those items for which the utility and/or usability aspect group mean scores were above 6.40. These group score criteria were also based on an application of Dalkey et al. (1969) and Jillson (1974, in Turoff & Linstone, 2002). The utility and usability aspects group mean scores, together with their categorisation are listed in Table 3.3.
Summary
In this chapter, the researcher has provided a discussion of the research methodologies used in this study. The researcher used questionnaires and interviews to conduct the three phases of the research. The researcher has provided information about the sample population of each of the phases. The development of the instruments for each phase has been described, as have
FIGURE 3.2 PHASES AND STEPS
Phase 1: Research Review
Phase 2: Delphi Survey
Step 2: Delphi survey of First Expert panel (A three-round paper-and-pencil Delphi survey)
Step 1: Selection of First Expert panel (N=19)
The best composite indicators tested for practicality
Step 1: Analysis of research related to composite indicators and their variables
Step 3: Semi-structured interviews (N=6) Step 2: Questionnaire developed for Delphi survey
Step 1: Selection of Second Panel (N=34)
Phase 3: Survey and Semi-structured Interviews
Best practice composite indicators established
Step 2: Survey of Second Expert panel
the qualitative and quantitative surveys employed in analysing the data. Figure 3.2 illustrates the interrelations of the three phases of the study: one, a Research Review; two, a Delphi Survey; three, a second survey followed by a set of semi-structured Interviews in order to clarify the first, second and third questionnaires; four, paper-and-pencil questionnaire for establishing the best practice and composite indicators for quality management in Masters Degree Program in Educational Administration in private higher education institutions in Thailand.
In the next chapter, the findings of the first of these three phases will be analysed and discussed.