Thesis writing and reflection cycle 1: Research instrument development

As described in chapter 4, a reflection section was inserted at periodic intervals to reflect on the thesis writing process and on the personal learnings during the thesis writing process. This section describes the first thesis writing reflection cycle after the initial development of the research instrument. The reflection was done on the outcome of the testing of the reliability and validity undertaken with SPSS.

5.7.1 Personal Reflective Pause: Research instrument reliability and validity results One of the most significant learnings related to the development of the research instrument was that the practitioner needed to be about aware how much work was required to undertake all the steps for validating a research instrument. Another important learning was that the practitioner needed to gain knowledge about qualitative data evaluation and quantitative data evaluation. Moreover, the practitioner’s statistical knowledge was strengthened, the practitioner learned how to evaluate data samples via SPSS and how to transcribe interview data. This experience showed that the practitioner sometimes jumped to tasks without knowing how much work it would require. This experience would help (and has helped) the practitioner to judge an idea and its consequences on the thesis work much better, so that the practitioner can quickly decide in the future, whether or not to spend the time, and whether the idea is of sufficient value to take the practitioner’s research forward. Another surprising issue was that during the development of the research instrument, the assumption was always that there were five areas to be investigated. However, the collected research data suggested the use of only three areas. This prompted intensive reflection, as to whether something was wrong because the practitioner had not yet reached at such mature working levels with these kind of statistical tools and methods. Consultation with the doctoral supervisor resulted in suggestions about some very good readings. After several discussions, there was good evidence to stick with the five areas; it could be that the data sample did not mirror the whole bandwidth as required. Therefore, the idea of undertaking a statistical measure with random generated data was conceived. The questions to be answered here were: How can random data be generated? How should the data be evaluated? The whole process is described and conceptualised in the next Kolb reflection section.

5.7.2 Description: Concrete experience

Since the final evaluation of the shared leadership research instrument was over, and all the validity and reliability tests were executed with SPSS, the expectation was that the statistical methods would confirm the five shared leadership items that were defined during the literature review. However, the CFA suggested the use of only three factors. During the interviews, one shared leadership area that was not confirmed during the interviews already had to be revised.

5.7.3 Reflective observations

On noticing that the statistical reliability and validity testing did not deliver the expected result, the practitioner felt that sufficient care had not been taken to define the ‘right’ items during the definition of the research instrument. How should the practitioner proceed? Should the five items be merged into three items? Could it be that the gathered sample size did not reflect the whole population as required? How could the practitioner show that the defined areas are still valid, although maybe not for the gathered research sample? Would the practitioner need to define a new area and new items? However, this would have implied going through the validity and reliability cycle once again; further, there would still be the risk that the new items might deliver the same results by not confirming the predefined areas. 5.7.4 Conceptualisation

Thinking further on the issue, it became clear that further statistical tests needed to be undertaken, but with a different set of data because for the available data set, it could not be validated, whether the sample reflects the group that the practitioner intended to research. Prior literature that criticises the CFA and suggests alternative methods to determine the amount of factors to be retrieved was reviewed. During this review, the idea of using random data sample sets of the same matrix sizes as those of the research instrument was conceived. The CFA with this random data for 100 data sets showed that the factors retrieved are greater than three (Zwick and Velicer, 1986).

5.7.5 Action plan

To undertake the CFA with a random data set, how to create a random data set and which tools were to be used to create the random data needed to be investigated. The Matlab tool seemed to be a useful tool for creating such a random dataset (Chipperfield and Fleming, 1995). Once the random data was created, the CFA and the reliability test needed to be repeated 100 times using SPSS to retrieve the eigenvalues and the Cronbach’s Alpha values. With the 100 eigenvalues and Cronbach’s Alphas that were retrieved, further statistical

analysis could be performed to justify why more than three factors should be chosen (contrary to what the CFA with the real dataset had suggested).