RESEARCH METHODOLOGY
3.10 THE PRE-STUDY (PILOT) AND THE MAIN STUDY .1 Introduction
The primary empirical data collection components or artefacts of this research are comprised of two online survey questionnaire instruments, the pilot/pre-study and the main study, responded to by emerging market risk managers as outlined in previous sections. As indicated by Hinkin (1998), in scale development it is necessary to use independent samples. The sample of the pilot study is comprised of emerging market telecommunications industry risk managers, and the second, main study, of members of the Institute of Risk Management of South Africa (IRMSA), i.e. emerging market risk managers. The sample and methodology of executing the instruments are discussed in further detail below.
The literature had indicated that for studies incorporating a cross-cultural element like this one, a sample as homogeneous as possible should be selected (GLOBE, 2004). Therefore, the emerging market risk managers being studied hailed from specifically the telecommunications industry and IRMSA. The second sample presented a broader base upon initial validation of scale and constructs, to import greater statistical power for the various empirical tests being conducted. As proposed within the literature (e.g. Morgado et al., 2017), the pre-study and main study processes and their requisite samples were also consciously promulgated as intermediate steps with regards to providing expert knowledge in developing the ERM values scale and constructs. The respondents were very familiar with the conceptual and theoretical elements of enterprise risk management values. The samples are elaborated on in further detail in Chapter 4 (results), where the demographic data and results are presented.
Questions around ERM values in the survey questionnaire instruments took the form of values statement items (manifest variables) designed to develop factors, clusters or dimensions (latent variables) that would act as dependent variables for further research purposes around the business and management sciences domain of ERM. Similar to the culture values dimensions being empirically tested, these were evaluated on a scale to give an empirical, quantifiable result for statistical analysis. All the questions (items) were based on interval scales allowing the respondent to express a range of responses to the questions.
According to Sjöberg (2002), quantitative rating scales are the preferred method for surveying values statements such as risk perception. Sjöberg (2002) further concluded that category or Kline scales with a limited number of response categories, such as 5 to 7, appear to be preferable, giving respondents the opportunity to express a suitable array of responses on scales. Hinkin (1998) referred to studies that had demonstrated that coefficient alpha reliability with Likert scales was shown to increase up to the use of five points, but then levels off. Selecting this type of scale for this study enabled statistically-significant differences or variations in scores, that could be attributed to ERM or culture values, to become apparent in the survey data analysis.
Data were collected by means of the Stellenbosch University SUrveys application, which is an online software product provided by Checkbox (Version 4.7). Examples of the actual survey questionnaire instruments utilised are provided in Appendices C, D and E.
According to MacKenzie et al. (2011), once the items and the measurement model have been finalised, data need to be collected in order to test the psychometric properties of the scale and to evaluate technical issues, such as convergent, discriminant and nomological validity. For this purpose, a pilot study was first conducted on the ERM Values Scale items.
3.10.2 The pilot/pre-study
According to MacKenzie et al. (2011), it is very important that the pre-test sample represents the population for which the measures are designed. For the pilot study, the sample was comprised of managers familiar with enterprise risk management in telecommunications companies deploying an ERM framework. Unfortunately, the number of respondents was smaller than desired, with 34 valid responses in total from 65 invitations sent out to respondents. MacKenzie et al. (2011) purported that the literature recommends sample sizes of minimum 100 respondents as a “rule of thumb” to conduct exploratory factor analysis (EFA). However, in an article dedicated to a review of and recommendations regarding sample sizes in factor analysis, MacCallum, Widaman, Zhang and Hong (1999) concluded that smaller sample sizes can be acceptable when communalities are high and factors strongly determined. Hinkin (1998) referred to samples sizes of only 20, suggesting that small samples may be appropriate for analysis, where factor loadings are high.
According to MacCallum et al. (1999) small sample size validity has been positively tested in many studies by a process of using sub-samples within a larger sample and checking the results from various slices of sub-samples. For the main study, the sample size was certainly large enough to pass any guidelines from the literature on both the overall recommended respondent number, i.e. 200 for CFA (Hinkin, 1998), as well as the guideline of respondents per item, i.e. more than ten (>10) (MacKenzie et al., 2011).
3.10.3 The main study
The main survey questionnaire instrument captured data on the ERM values scale, the Hofstede and GLOBE culture values scales as well as demographics of the sample respondents. For the purposes of the main study, permission was granted by IRMSA to gain access to the database of IRMSA associates (risk management practitioners as discussed in the previous chapter). The response rate was positive and over 300 valid responses were collected, providing an acceptable basis for CFA testing. This sample was very appropriate for the reasons previously presented, i.e. that respondents are experts in ERM and have knowledge of the ERM content and construct domain, further contributing to validity of the scales and constructs. The demographics of the sample are summarised and discussed in the related results sections of Chapter 4, where the full results are presented and discussed.
3.11 CONCLUSION
In summary, it was established that it is of great interest to the global business management community, from both a practitioner and academic perspective, to understand the ERM values of (risk) managers in emerging markets. Organisations exist to create value and ERM is central to value creation. In order to understand an organisation’s risk culture, and specifically how this culture relates to ERM practices, it is critical to be able to measure and understand the values of individual managers towards ERM key success factors. Currently, no theory, nor measurement scale, nor constructs of ERMVs exist, which can be used to predict or explain differences in ERM behaviour or practice due to values of managers. It is the aim of this research to take a significant step towards filling this gap in knowledge.
Following the ‘classical’ scale and construct development process as detailed in the literature, this chapter provided a full description of the research methodology implemented to address the research problem and questions – namely the development of an ERM values scale and resulting constructs to empirically measure ERM values. Recent comprehensive reviews of the construct development literature in the business and management sciences have demonstrated that much of the ‘classical’ process remains similar to the methodology detailed in Churchill’s (1979) seminal work in marketing management. However, there have certainly been advancements in terms of study design improvements, identifying limitations, elaborating and reporting on each of the steps of the process. This is particularly relevant to technical issues around factor analysis and cross-validation of constructs.
This study has attempted to follow a “best practice” process compiled and adapted from the literature in terms of designing the stages of scale and construct development and testing, as well as providing comprehensive reporting on each of the steps. In the next chapter, the specific tests conducted are elaborated on and the corresponding results from the two studies and samples are presented and discussed.