Making the Prospective Study Operational

Once the presentation of scientific evidence from the existing body of knowledge justified the prospective study, the investigator made the research project operational by transforming its original conception through mental impressions on how the study ought to be done into executable research actions, activities and interventions that accomplished the envisaged purpose of the study and its scientific objectives. Various statements that are included in Chapters 1 to 6, confirmed that the study would require two research approaches: one quantitative and the other qualitative. It was believed that the dual approach to data analysis would enrich the study and also elicit the kind of research outcomes that would yield rich and innovative information. 6.5.1 Developing the research strategy

From a psychotechnical perspective the empirical design of the study was a two-way between-groups design that compares independent groups (Kirkpatrick & Feeney, 2007, pp. 49-57). The design was thus appropriate for statistical analysis of variance (also refer to Section 6.6.3 of Chapter 6).

6.5.2. Selecting the measuring Instruments

An existing measuring instrument, augmented by two psychometric instruments that were specifically developed, constructed and evaluated in the early stages of the prospective study, yielded 20 dependent variables. The Multifactor Leadership Questionnaire assessed twelve preferred leadership styles. The first of two new measuring devices evaluated six executive tasks (refer to Section 7.3.4 of Chapter 7) while the second assessed two executive behaviours (refer to Section 7.4.4 of Chapter 7). The availability of new measuring instruments warranted calculation of reliability estimates. Two statistical formulas for the estimation of reliability are

106 provided in Section 6.5.2 of Chapter 6. The resultant coefficients of reliability are included in Chapter 7. Measurements for the current study was derived from participants’ responses to questions in the Executive Leadership Questionnaire and captured in a data set for the purpose of thorough statistical analysis in order to realize the scientific and research objectives of the prospective study (in this regard, refer to Sections 3.4 to 3.6 of Chapter 3, and Subsections 6.4.1. to 6.4.3 of Chapter 6). The Executive Leadership Questionnaire appears in the dissertation as Appendix A.

6.5.3 Demarcating the target population

The first stage of sampling was a computer-based initiative aimed at identifying, contacting and notifying executives of the prospective study, and also included an invitation to participate in the prospective study. Respondents were asked to provide information on the type of employment contract that they had entered into with their employer and the tenure of their agreement. The desired research outcome was the drawing of two independent samples from the two sectors, each with sizeable numbers of subjects, stratified in terms of their employment contracts as potential participants with either fixed short-term or indefinite (or even permanent) long-term employment contracts. The initial phase yielded four distinct subgroups, two from each sector, one of which included executives that had negotiated fixed short-term while the second subgroup included executives that had negotiated indefinite long- term employment contracts. In stage 2 of the sampling process, four random and thus independent subsamples, equal in size, would be taken from the ranks of the four subgroups derived at in phase 1 and enrolled for participation in the forthcoming study

6.5.4 Calculating the initial sample size

Research of quality requires optimum sample sizes. The current researcher applied Howell’s one-sample formula (2004, pp. 341-346) to determine the sample size:

Formula 1

This formula, upfront, required three statistical parameters: an α level, a statistic of the power of a test, called a δ value, and a γ ratio, or the difference between to

107 arithmetic means divided by a shared standard deviation [(μ1 – μ2)/s]. The α value is

a probability level that is set for tests of significance of normally distributed data. The power statistic lowers or raises the likelihood of correctly rejecting a false hypothesis statement (H0). A high power statistic reduces the chance of accepting an incorrect

H0 while a lower power value will increase the likelihood of accepting an incorrect H0.

The γ value permits an examiner to specify a statistic’s effectiveness for detecting small but significant differences in consequent statistical calculations.

In calculating an optimal sample size, the researcher set the three parameters at 0.05, 0.80 and 0.250 [(10.50 – 10.00)/2]. In research practice, the chosen power value is converted to an equivalent δ value. Table E.5 in Appendix E of Howell‘s (2004, p. 522) textbook brackets a power value of 0.80 with a δ value of 2.80. The γ ratio is a function of the n the forthcoming study: γ was set at or 0.250. Insertion of the latter two values in the above formula (2.80/0.250)2 generated a numerical value of 125.44 as the required sample size.

The sample size of 125 ought to optimize the purposes of the pending study. In practice the total sample of 125 participants would be stratified in two employment sector subsamples of 62 participants each, whereafter each of the two sectors would be further stratified into two subgroups each, one group made up of 31 executives who had entered into fixed short-term employment contracts and the remaining subgroup of 31 executives who had negotiated permanent of indefinite long-term employment contracts.

6.5.5 First-phase sampling

The three major independent variables were categorical variables and thus suitable for manipulation in quasi-experimental designs. The nature of the study mandated two-staged sampling. Executives employed in the public and private sectors were defined as the two target populations. The first stage of sampling was a computer- driven initiative that identified, contacted and notified executives of the prospective study, and included an invitation to participate in the prospective study. Respondents were asked to provide information on the type of employment contract that they had entered into and the tenure of their agreement. The desired research outcome was drawing of two independent samples with sizeable numbers of subjects from the two

108 sectors, stratified in terms of either fixed short-term or indefinite (or even permanent) long-term employment contracts. The initial phase was designed to yield four distinct subgroups, two from each sector, one of which included executives that had negotiated fixed short-term while the second subgroup included executives that had negotiated indefinite long-term employment contracts. In stage 2 of the sampling process, four random and thus independent subsamples, of equal size, would be taken from the ranks of the four subgroups derived at in phase 1 and recruited or enrolled for participation in the forthcoming study. Stratification of the two target populations into four subgroups mandated a 2 x 2 factorial format for the quasi- experimental design. All incumbents who were initially approached for participation but whose employment contracts either had been terminated or were not renewed or had vacated their posts for whatever reason were not considered for inclusion in the prospective study.

6.5.6 Locating, contacting and enrolling participants

The processes of contacting executives, formally inviting them to participate in the coming project, recruiting them and sending the Executive Leadership Questionnaire to them was done through linked electronic transmission and telephonic interviewing. Completion of the questionnaire, on average, took 23 to 30 minutes. The measuring instrument was scripted into a Computer and Telephone Interview (CATI) system by means of a programme called NIPO. The NIPO ODIN Interview System was based at a market-research institute in the Netherlands.

The NIPO software consisted of two programmes: the NIPO ODIN Developer served the development of questionnaires, including syntax checking, testing, coding of and verbatim answers, while the NIPO ODIN Interview System activated and conducted the actual interviewing. Further functions of the programme include coding of open- ended responses and exporting of data to DIANA, SAS, SPSS, and QPS systems, as well as fixing and adjusting column numbers, and renumbering questions on data files.

A project leader or client would construct a concept questionnaire in a word processing format, such as Word or WordPerfect. The client, not needing to know anything of the ODIN programme, is able to read and understand this questionnaire.

109 A number of repeats results in a final questionnaire that is agreed upon by the client and the project leader where after scriptwriters convert the measuring instrument with the Nipo Odin Developer and test the questionnaire until it is formally approved by the project leader. Other scriptwriters create a file that captures respondent’s telephone numbers and if required, stratification or configuration files that aim to consolidate the process of gathering data. Once the files are loaded into the NIPO ODIN interview system, the interviewing can then be activated while interviewers can start with their fieldwork. The most effective way to commence fieldwork is to launch a pilot session that tests conjoint handling of the measuring device by interviewers that interact with a few respondents.

The CATI system, itself, is a telephone surveying technique in which the interviewer follows a script provided by a software application, such as NIPO. The conduct of computer-assisted telephone interviews requires interactive voice response (IVR) solutions. Phone surveys can be initiated by outbound phone calls or can be done in response to callers. The phone systems Smart Message Dialer and IVR software can call a potential interviewee and transmit a customary greeting. A caller is given the option to be interviewed by phone or can talk with a representative, if needed, or leave a voice message, request additional information or simply decline to do the interview. The phone interview application also can be programmed to interact with touchphone responses or can record each question response for later analysis. If the client requires it, the supervisor can listen to 10% of the interviews. He or she does so in compliance with the SAMRA code of conduct. Monitoring of calls is done remotely. Supervisors can monitor calls of interviewers by looking at a screen. The phone system permits them to log on to the system. A telephone management system stores the interview data. Ipsos South Africa has 104 workstations in the call centre

6.5.7 Assessing the initial sample response

The first-phase of the sampling attracted 128 participants: 96 (75.0%) from the private sector and 32 (25.0%) from the public sector. Among the 96 private sector executives, 68 (70.8%) had entered into long-term employment contracts compared to 28 (29.2%) of their counterparts with short-term employment contracts. Within the

110 ranks of the 32 public sector executives, eight (25.0%) had long-term employment contracts while 24 (75.0%) had negotiated short-term contracts.

6.5.8 Second-phase sampling

The researcher originally intended to draw four random subsamples of equal size from the four targeted subpopulations during stage 2 with the intention to recruit and enrol them for participation in the pending study. The unexpected poor response by public sector executives with long-term employment contracts forced the researcher to convert the forthcoming research project into a single-sample quasi-experimental study.

6.5.9 Redefining the target population

The study supervisor agreed with the aforementioned proposed conversion of the target population. All executives who had initially responded, regardless of their sector of employment, were aggregated in a single target population. Participants from the revised target population were recruited and enrolled for participation in the actual investigation. The study population was stratified in three tiers: at the primary level in terms of executives who had negotiated either fixed short-term or long-term or permanent employment contracts, at the secondary level, in terms of gender, and at the tertiary level, in terms of age cohort group.

6.5.10 Revisiting the scientific objectives

The enforced pooling of the envisaged four samples into a single study sample did influence proximal outcomes 2, 4, 6, 9 and 10 in the sense that the public-private sector comparison of the impact of executives’ length of term of employment on their executive effectiveness was no longer possible. Likewise, research objectives 2, 4, 5 and 6 no longer had to differentiate between and accommodate executives from the public and private sectors. The emphasis, henceforth, was on the pooled single sample of executives.

111 The construction and psychometric evaluation of two new measuring devices are described in Sections 7.3.4 and 7.4.4 of Chapter 7. The first instrument assessed six executive tasks while the second device assessed two executive behaviours (refer to Sections 7.4 and 7.5).