7 Fit Front-End Development: a Framework for Creating Project Success
7.4 Discussion on the Framework
In sections 7.1 to 7.3 it was shown how a quantitative analysis, accompanied by a qualitative interpretation step, can be used to fit the front-end development of a project to the specific project situation. The general structure of the developed framework was described, and the way to implement it was illustrated with two examples. Although the value of applying the framework was not proven, the practical applicability was shown.
The limitations of and implementation requirements for the framework as it is presented in this thesis are discussed in the remainder of this chapter.
7.4.1 The Assumption of a Functioning Organization
This framework does not provide a way to assemble a front-end development phase from scratch. It assumes an organization that is already functioning, but wants to incrementally improve its own performance. Using Shell as an example of such an organization, the functionality of the
framework has been illustrated. However, for companies with a less elaborate project
management approach, the assumption of a functioning organization might be a limitation to the applicability of this framework.
7.4.2 The Availability of the Right Project Performance Data
The data analysis step requires the presence of data on past project performance. Even in the largest organizations, these data might not be readily available, especially if the success criteria set for a specific project are not common for the organization owning the project. Looking at the Shell example used in this thesis, collecting and analyzing sufficient data was a complex task, but the results of theses steps were good enough to suggest improvements for the existing FED practices.
To optimize the use of this framework, it cannot be implemented as a stand-alone solution for project management process improvement. It requires implementing a review system that captures the quality of FED inputs (both deliverables and activities) as well as project outcomes on all possibly relevant project success criteria. The following data should be gathered*:
• Data showing the performance on FED inputs (e.g. quality of FED deliverables, quality of
execution of activities such as DVPs, values of FED quality indicators) as well as the resource intensity with which they were performed in the different FED phases.
• Data showing the project outcome on the variables mentioned in the project success
criteria description belonging to the project of which the FED phase is to be built.
• Data about the project characteristics (see 7.1, step 2).
7.4.3 The Availability of Sufficient Project Performance Data
In step 2, when trying to reduce the set of all projects to a smaller subgroup, care should be taken not to reduce the number of projects to a too small number to draw meaningful conclusions on the correlations. Although no statistical theories have been consulted regarding what a “too small number” of projects is, applying the same constraints as should be applied to a chi-square cross tab test will not be a too strict** requirement. For the case of an input indicator which can assume
three values, this would mean that for each of these input values at least five projects need to be closed-out and documented that were scored with that specific input value.
Obviously, the more selection criteria are applied to determine which projects to use as members of the subgroup, the fewer members remain. This makes it harder to satisfy the analysis criteria mentioned above. Making more projects available for analysis will require a systematic,
permanent data collection effort of which the results can only be reaped after a couple of years.
* For this purpose, audit data are more useful than assurance data, since an audit focuses on comparability and
measurability rather than on improving the process in an existing project. The data entry forms from IPA, PDRI and Van Pelt (2008) could act as a starting point for setting up the data collection system.
** In a chi-square cross tab test, the (in-)dependence of variables is tested. Using a finite number of possible values on the input (i) and output (o) scale (ordinal/nominal scales are allowed), and looking at the resulting i x o matrix containing the frequencies with which the cells in this matrix occur, the following requirements should be met: (1) all cell-frequencies must exceed 1, (2) only a maximum of 20% of cell-frequencies can be between 1 and 5. In case these requirements cannot be met, columns/rows in the matrix can be grouped until the requirements are fulfilled.
Enlarging the number of projects available by comparing with projects owned by other players in the industry is an option for the short term (practically speaking, for the oil and gas industry this would mean asking IPA to conduct analyses specifically related to optimizing the success criteria set for the project under consideration).
However, if the selection criteria for the subgroup of projects are chosen with care in a pragmatic way, also with smaller databases meaningful analyses can be performed. Pragmatic here means using experience in deciding which specification criteria are truly important in selecting. For estimating for example, the difference between a Greenfield and Brownfield project is commonly known to be large (unstructured interview with CP1, 2008), so if cost predictability is the main success criterion, only considering the applicable one of these two groups would be one of the first steps to take.
Another guideline in the selection of the subgroup is to select based upon the complexity elements that are scored “high” for the specific project. Elements of high complexity determine the most important differences with other projects and therefore require extra attention.
These steps might appear complicated at first sight. However, it might be valuable to take these steps, since preliminary analyses show that differences between problematic input areas do exist for different types of projects (e.g. looking at location). Furthermore, in chapter 5 it became clear that different types of projects get different FED activity intensities prescribed. The data used in this research did not support analyzing the effect of these different intensities, but doing this in the future seems promising.
7.4.4 The Quantitative Approach
The framework for improvement of the FED presented in this chapter is very quantitatively focused. The search for points for improvement is done using data analysis. The disadvantage of this approach is that all sources of information that cannot be quantified or are not quantified are left out of consideration, seriously limiting the view.
The people using this framework should use the results as a guide for determining which FED activities to conduct or not, and for increasing their understanding of front-end development. If correlations are found, they should be further explored in order to increase understanding of the project management process and its implications. However, reducing focus only to issues found through applying the framework would not show a good understanding of the project
management profession. For example, new practices cannot be identified by using the framework, since it only uses evaluations of current practices.
The framework therefore should only be implemented by organizations being well aware of the limitations of this framework. These companies should have the capacity and will to deploy other project management process improvement efforts simultaneously.
7.4.5 Ways to Fit
In the research it was found that there are basically two ways to fit an activity to the project: performing the activity or not, and if performing is deemed beneficial, varying the intensity level with which it will be applied. In this framework making this distinction was not facilitated. It was only indicated how important sufficient performance is, not what sufficient performance is. For future improvements of the framework, it is recommended to develop a way to take the level of intensity with which an activity is performed into account in the data analysis step. This way fitting can be done less ambiguously: rather than speaking of “investing extra effort compared to the company average”, the optimal level of effort invested in a certain activity can be indicated.