I NSTRUMENTS

The instruments tested in this preliminary study include, 1. Project Environment Questionnaire

o Demographics of the project manager

 Name

 Current Role

136

 Gender

 Age

 Country of residence

o Number of years project Experience

o Perceived organisational project management maturity o Type of projects being managed

o Complexity of the projects being managed 2. Individual Project Management Competency 3. Coping Process Assessment

o Primary Appraisal

o Coping Strategy Selection

a. Project Environment Questionnaire The questions include

- Age

a. under 30 b. 30-40 c. 41-50 d. over 50 - Gender

1. Male 2. Female

- In which country do you work?

- How many years project experience do you have?

- How many year work experience do you have?

- What is the perceived level of project management maturity of your organisation

1. Initial Level - ad hoc and chaotic; relies on the competence of individuals not the organisation's

2. Repeatable Level - there is a project management system and plans are based on previous experience

137

3. Defined Level - common, organisation wide understanding of project management activities, roles and responsibilities

4. Managed Level - stable and measured processes against organisational goals; variations are identified and addressed

5. Optimising Level - the entire organisation is focused on continuous improvement

b. Individual Project Management Competence Instrument

The GAPPS based individual project management competence instrument was included in a study being conducted by Crawford and Aitken as a follow up to the earlier research conducted by Crawford (2000b) using the Australian National Competency Standards for Project Management. In the pilot study an alternative process and scale were trialled. Participants were asked to describe a single project they had managed within the past three (3) years. Participants rated the project against the CIFTER factors and were told whether their project was below Global Level 1, a Global Level 1 project or a Global Level 2 project. They were then asked to rate each of the performance criteria in the GAPPS standard using the following scale:

1 - I have never done this or have only contributed to this being done as part of a team under supervision.

2 - I have done this occasionally on minimally complex (G1) projects.

3 - I have done this often on minimally complex (G1) projects.

4 - I have done this occasionally on complex (G2) projects.

5 - I have done this on many complex (G2) projects.

The results of the trial proved the scale to be unusable for statistical analysis as it was not a graded scale. This scale was abandoned and the original used by Crawford, with minor changes, was selected. This scale is documented in section 5.4.2.

Complexity was measured using the CIFTER from the GAPPS project manager standard. The seven (7) items were rated on a four (4) point scale using the descriptors in the table below.

138 Table 17 - CIFTER Factors

Project Management Complexity Factor

Descriptor and Points

1. Stability of the overall project context Very high (1) 2. Number of distinct disciplines,

methods, or approaches involved in performing the project

3. Magnitude of legal, social, or environmental implications from

4. Overall expected financial impact (positive or negative) on the project’s stakeholders

5. Strategic importance of the project to the organisation or organisations

6. Stakeholder cohesion regarding the characteristics of the product of the project

7. Number and variety of interfaces between the project and other organisational entities

The data collected from the CIFTER was analysed and the Cronbach alpha for the seven (7) item instrument is .655 acceptable by some standards but below the .7 recommended by Nunnally (1978). Exclusion of item 6, Stakeholder cohesion regarding the characteristics of the product of the project, increased the Cronbach alpha score to .701 (N=72).

A factor analysis was conducted to further explore the factor structure of the CIFTER.

The CIFTER is designed to measure a single item, complexity. Factor analysis is used to confirm. Inspection of the correlation matrix identified a number of coefficients of 0.3 and above. The Kaiser-Meyer-Oklin value was 0.719, exceeding the

139

recommended value of 0.6 and the Barlett’s Test of Sphericity reached statistical significance, supporting the factorability of the correlation matrix.

Principal component analysis revealed two (2) factors with eigenvalues greater than 1 explaining, 37.142 and 21.054% of the variance respectively. The two (2) factor model explained 58.197% of the overall variance. However, from further inspection of the Scree Plot a clear break was evident after the first factor. A second factor analysis was performed on the basis of a one-factor model. The single factor solution revealed a simple structure with all items except item 6 loading onto the one factor.

The full analysis is detailed in the Appendix below Section 12.1.7.

c. Coping Instruments

Appraisal of Controllability was measured using a single item taken from the study Folkman et al. (1986) conducted examining the relationship between cognitive appraisal (primary & secondary), coping processes and short term outcomes with stressful situations The question asked subjects to rate how they usually felt when considering stressful situations where

1. ‘you can change or do something about the event’

2. ‘you have to accept the event’

3. ’ you needed to know more before you can act’

4. ‘you have to hold yourself back from doing what you want to do’

Coping Strategies were assessed using the dispositional format of the Brief COPE measuring how subjects usually try to deal with stressful situations. The Brief COPE is a 28-item assessment that measures 14 coping scales including, Self Distraction, Active Coping, Denial, Substance Use, Emotional Support, Instrumental Support, Behavioural Disengagement, Venting, Positive Reframing, Planning, Humour, Acceptance, Religion and Self Blame. Subjects were asked to rate to what extent they usually used each of the coping strategies listed on a 4-point scale which is

1. ‘I don’t usually do this at all’;

2. ‘I usually do this a little bit’;

3. ‘I usually do this a medium amount’ and 4. ‘I usually do this a lot’.

140

The Cronbach Alpha score is acceptable for this sample (.6631). Internal reliability was explored for each scale. For the scales, Emotional Support, Instrumental Support, Behavioural Disengagement, Venting, Positive, Reframing, Planning, Humour, Religion and Self Blame high internal consistency scores were found ranging from .6827 to .9166. Cronbach Alpha’s for Active Coping, Denial and Substance Use were below .6 but remained above .5, the minimum acceptable level (Carver, 1997).

However the scales Self Distraction and Acceptance returned unacceptably low scores of .3360 and .4615. These scores are similar to those found by other researchers (Carver, 1997, Fillion et al., 2002).

5.5.3 Summary

The demographic questions were refined following the analysis of the testing results.

The combination of questions regarding project and work experience did not provide any useful data and were re-worded for the final study to focus more narrowly on the project experience only. This was split into two questions focusing on number of years project management experience and specifically number of years project manager experience. The question regarding project management maturity was re-worded to reduce the number of words (and time to read and respond) without materially altering the question.

The scale used in the individual competency assessment during the pilot was in fact a nominal scale which resulted in a categorical set of data which was unusable for correlation and regression analysis. The scale was revised for the final data capture to create an ordinal scale and in doing so the number of projects candidates were asked to describe and rate using the scale increased from one (1) to two (2). The CIFTER was determined to be suitable for inclusion within the final study even though the analysis of internal reliability and factor structure provided some doubt as to the strength and accuracy of item 6.

The results of the control rating proved to be unusable, as with the individual competency scale, the control rating was also not a scale. It also became clear that the question of controllability in isolation did not address in enough detail the multiple stages of appraisal contained within the Transactional Theory of Coping (Lazarus and Folkman, 1984b). The hypotheses in this study deal specifically with both primary

141

and secondary appraisal and the influence of control at both stages. The final study altered the scale used with this question to a 5 point Likert scale and added a second question specifically related to the primary appraisal of threat/challenge. For secondary appraisal an entirely new instrument, Generalised Self Efficacy, was added to explore the nature of control at this juncture in the coping process.

The instrument testing proved the Brief COPE to be applicable and relatively robust with a project manager population and suitable for inclusion in the final study. The low internal reliability scores for two of the scales is considered to be more likely due to the small sample size and the use of the instrument in a dispositional format rather than a fundamental flaw. The instrument was converted to the situational format for the full study.