5.1 Data sources and analysis
5.1.1 Structured interviews
The interview questions were either seeking categorical data or ordinal data. A question concerning the size of the company, for example, sought information to enable classifying the organization on a scale of small, medium or large. In this case, the question was accompanied
Page 116
by an ordinal list and the appropriate organization size was selected from the list during the interview.Questions that sought categorical data were accompanied by a check list of the expected common responses and space for recording responses that were not already in the list. Where the responses used differing terms to describe the same type of phenomenon, the actual term used by the interview subject was recorded. In later encoding, the range of terms was examined and groupings made so that essential differences were preserved but simple terminology differences were not accorded undue significance.
When encoded into SPSS, the dataset required 100 variables, some of which had been added to represent aggregations of other variables. For example, there were a number of possible methods project managers used to determine if their projects were “on track”. In addition to recording which methods a project manager said they used, an additional variable recorded the total number of methods. Similarly a variable was added for “Organizational distance” since it is a combination of the variables that were responses to several other questions.
5.1.2 Content analysis
Analysis of the transcripts was left until long after the interviews were completed. Analysing all interviews in a condensed period helped get a consistent analysis by reducing the tendency to modify criteria over time.
Content analysis most commonly counts the frequency with which something is mentioned as indicating its importance (Lacity and Janson, 1994). This analysis seeks to identify the different mechanisms used by project managers to monitor, control or coordinate their projects. Of interest was the number of the project management mechanisms of the different types rather than any indication of importance accorded them by the project manager. The various
mechanisms were classified according to frameworks established from the literature (Table 9). The data were encoded into an SPSS data file using 82 variables, separate from the data set used for structured interview data. The two data sets were checked to ensure that both the project manager code and the organization code matched. A mismatch would have indicated a missing interview or a data entry error.
Of the 82 variables, 4 variables characterised the organization (organization size, process capability, project size, project novelty), 3 variables established an assessment of organizational distance (cultural difference, structural distance, time zone difference and administrative distance), 7 variables recorded the different types of automatic mechanisms (CSCW/Workflow, configuration management, defect/issue tracking, automated testing, release management,
Page 117
development life cycle and project web page) plus another variable to total the group. Then the mechanisms use to monitor, control and coordinate were each represented by forty-five variables (Appendix D), plus a variable to record a total for the group. Recording the raw and aggregated data in this way allowed several different analyses to be explored to ensure that the findings were sound.5.1.3 Qualitative analysis
Each interview is a small, albeit narrow, case study (Yin, 2003) conducted to explore the research question that the choice of project management mechanisms will depend on the organizational distance between the project manager and elements of the project team. Each interview is examined for themes that support the research question, or that support possible rival propositions.
Qualitative analysis is better able to reveal the reasons for actions, why a decision was made or a project management mechanism was chosen, and to give depth and meaning to a complex situation (Leedy and Ormrod, 2001). While this research primarily investigates which project management mechanisms are used and the relationship between organizational distance and the use of project management mechanism, augmenting the quantitative analysis with qualitative analysis may give a deeper understanding of the results.
5.2 Statistical power
The statistical tool used in this research to examine relationships is most often Pearson’s correlation. While a correlation cannot prove causation it can show that there is a relation between two variables. Cohen (1977) argues that the significance level of a correlation should be augmented by considering the effect size, which is one of the main determinants of statistical power. Conventional definitions of the effect size present in a correlation offered by Cohen are: Small r = 0.10
Medium r = 0.30 Large r = 0.50
Page 118
5.3 Sample Characteristics
5.3.1 Organization size
Organization size can be measured in different ways such as number of employees (ABS, 2002) or the total income of the organization (ATO, 2004) among others, with the choice of measure seeming to depend on the purpose of the classification. For this research the number of personnel in the organization, actual or estimated, will be adopted as the measure of organization size. This estimate includes the whole organization, not just the software
development part of it. Table 24 gives the distribution of organization size. The size divisions were chosen because they reflect approximately where organizations tend to change structure (Mintzberg, 1979), from direct supervision through simple, single layer management through to multi layer management.
Alternative measures of organization size such as gross turnover or assets were not used because it was thought that some organizations may have been reluctant to divulge such information. Normally people are less reluctant to supply the number of employees.
Table 24: Organization size in the sample.
Frequency Percent Small (< 10 staff) 12 37.5 Medium (11 - 30) 4 12.5 Large (31 - 120) 3 9.4 Multinational (<1000 or multinational) 13 40.6 Total 32 100.0