The extent to which results can or may be generalised to the sampling population is a function of the number of observations in the chosen sample and the representativeness of the sample, while the power of inferential statistics tests also depends on sample size (Elmes, Kantowitz & Roediger, 1999; De Goede & Theron, 2010).
Sample sizes of 200 observations or more are appears to be satisfactory for most SEM applications (Kelloway, 1998; MacCallum et al., 1996 as cited in Smuts, 2010).
Three issues are relevant when deciding on the appropriate sample size for a study that intends using SEM. The first consideration is the ratio of number of observations to the number of parameters to be estimated. A situation in which more freed model parameters have to be estimated than there are observations in the sample would not be regarded as acceptable. Elaborate measurement models contain more variables, have more freed parameters that have to be estimated and consequently require larger sample sizes. Bentler and Chou (cited in Kelloway, 1998, p. 20) recommend that the ratio of sample size to number of parameter estimated should fall between 5:1 and 10:1. On the other hand Jackson (as cited in Park, 2003) advices 10: 1 or even better 20:1. The proposed measurement model (Equation 1) and the proposed procedure for operationalising the latent variables would in terms of the Bentler and Chou (cited in Kelloway, 1998) guideline require a sample of 1500 - 3000 research participants to provide a convincing test of the structural model (300 freed parameters).
The statistical power associated with the test of the hypothesis of close fit (H0: RMSEA ≤ .05) against the alternative hypothesis of mediocre fit (Ha: RMSEA > .05) is a second consideration to take into account when deciding on the appropriate sample size (Smuts, 2010). Statistical power in the SEM context refers to the probability of rejecting the null hypothesis of close fit (H0: RMSEA ≤ .05) when in fact it should be rejected (i.e., the model fit actually is mediocre, (Ha: RMSEA > .05). Too high statistical power would mean that any attempt to obtain formal empirical proof
for the validity of the model would be futile. Even a small deviation from close fit would result in a rejection of the close fit null hypothesis. Conversely, however, too low power would mean that even if the model fails to fit closely the close fit null hypothesis would still not be rejected. Not rejecting the close fit under conditions of low power does not provide very convincing evidence on the validity of the model.
Power tables were compiled by MacCallum, Browne and Sugawara (1996). These tables can be used to derive sample size estimates for the test of close fit, given the effect sizes assumed above, a significance level (α) of 0,05, a power level of 0,80 and degrees of freedom (ν) of (½[(p+q][p+q+1]-t)=6806.2-300=3403-300= 3103. The MacCallum et al. (1996) table, however, only makes provision for degrees of freedom up to 100. Syntax developed by Preacher & Coffman (2006) and available at http://www.quantpsy.org/rmsea/rmsea.htm was utilised to determine the required sample size for the test of close fit. For this purpose a significance level of .05 was specified, statistical power of .80, 3103 degrees of freedom, RMSEA was set to .05 under H0 and RMSEA was set to .08 under Ha. The Preacher and Coffman (2006) software indicated that a sample of 19 is required to ensure statistical power of .80 when testing the close fit null hypothesis.
The third consideration to take into account when deciding on the appropriate sample size is practical and logistical considerations like cost, availability of suitable respondents and the willingness of the employer to commit large numbers of employees to the research. The latter consideration prevents any attempt to draw a sample of 1500 respondents for the purpose of this study.
The only way of escaping from the impasse created by the incompatibility of the foregoing considerations is to utilise a different approach to the operationalisation of the latent coaching competencies. Rather than using the individual items to represent the latent variables item parcels will be formed. This will reduce the number of parameters that have to be estimated from 300 to 204 and the degrees of freedom to 391. In terms of the Bentler and Chou (cited in Kelloway, 1998) guideline a sample of 1020 is now required. The Preacher and Coffman (2006) software now
indicates that a sample of 57 is required to ensure statistical power of .80 when testing the close fit null hypothesis.
Taking all the above considerations into account, it was suggested that a sample of 200 – 250 units of observation should be selected by means of a non-probability convenience sampling procedure.
An alcoholic beverages producer was finally identified as a participating organisation by the researcher after several approaches to various companies based on personal contacts did not yield desired results. Detailed information about the research proposal and procedures were in the initial contact (see Appendix C). A letter of agreement to participate was provided by the company (see Appendix D). After the approval from the ethics committee of the University of Stellenbosch the researcher commenced with data collection. Managers at this company (alcoholic beverages producer) as well as their subordinates, peers and superiors were invited to participate in the research via e-mail to complete the self rater and other-rater version of the survey questionnaire (see Appendix E). The URL linked to the web survey using an online survey tool created by the University of Stellenbosch was sent. The Survey consisted of eight pages including the completion events page.
Participants were instructed to read the consent information and to give their consent before beginning (see Appendix A & B).
Based on a trial conducted earlier on, the estimated time for completing the survey was 15-20 minutes. Two weeks after sending the initial invitations reminders were sent out (see Appendix F).
The survey data was received via the internet without any personal identifiers of the respondents. The received data was stored anonymously in a password protected file in the researcher personal computer.
From the 2398 selected respondents only 492 entered and completed the survey (Response rate of 20.5%). After cleaning data the sample was reduced to 39811. Demographic information of the respondents included gender, age, education, years of service, peromnes job level and relationship to the candidate in the case of the other rater version of the survey.Tables 3.2 and 3.3 shows the sample composition by demographic characteristics.
Table 3.2
Demographic Information: Gender, Education, peromnes Job levels and Relationship to the candidate.
Table 3.3
Demographic Information:Age and Tenure
11 Initially after cleaning data the sample size was 399, however after treating for missing value it came to 398 as reported above.
Demographic Category Frequency %
Gender Male 233 58.4