Phase One: Evaluation of PMs’ Performance in solving SSSP Challenges

In this phase, the study subjects are asked to answer a set of four self-completion questions. Due to time limitations and availability, the subject can chose when to complete these questions and send them back by email for analysis. This set encompasses four SSSP scenarios corresponding to the classes and complexity levels discussed and solved in Chapter 3. The subjects in this part are asked to provide their best allocation and estimated project time span according to the resulting allocation schedule, and the time that they consumed to solve each question. As the scenarios’

data provided to subjects are the same as those used for the SSSP approaches evaluation, this will allow us to demonstrate whether the subjects perform the allocation in a similar way to the approaches, and to infer the factors that might play a key role in contributing to good quality results from the subjects.

Findings and Results

One of the subjects (D) did not provide answers to this phase’s questions, and asked to proceed to the next one. The reason given by the subject for this matter is the absence of real factors that these scenarios did not include such as the description and details about the intended software to be developed as well as the roles that are required for a single team to perform the project activities. However, the subject claims that it can be seen within the large size companies’ projects, similar to the one presented in scenario four, where tasks, dependencies, skills, and productivity are all that a PM would consider while performing the allocation. For details on the scenarios provided to the subjects the reader can refer to Section 3.4.2.

In addition, another subject (M) has only provided answers for the scenarios but could not proceed to the next phase. The problem was the time availability and implication of the different time zone, as the subject is constantly traveling. The solution provided by each subject in this study of

estimated project time span and the time consumed to solve each scenario are presented in the following Table 41.

Table 41: Study Subjects Responses

Table 41 presents the subjects’ responses who participated in this phase for PM performance study. As their identity remains anonymous, letters are used to differentiate between the subjects’

identity in all the associated tables and statements. This table moreover includes each subject’s answers of estimated project time, the time consumption, and allocation method (s)he adopted to solve each scenario. The allocation methods that each subject adopts to solve the scenarios differ

Participant Name DT NCS E M

Scena rio 1

Time Performance/Minutes NA76015202022 Min Solution -Days NA8881111.5 111.5 111.5 111.5 Allocation MethodNADynamic teams Dynamic teams Rigid teams with Percentage individuals, no simultaneous works

Two resources to each taskDynamic teams

Scena rio 2

Time Performance/Minutes NA106040506044 Min Solution -Days NA8782412.5 412442111.5 Allocation MethodNADynamic teams Dynamic teams Rigid teams with Percentage individuals, no simultaneous workRigid teamsDynamic teams

Scena rio 3

Time Performance/Minutes NA157540303028 Min Solution -Days NA104104132132132117.8 Allocation MethodNADynamic teams Dynamic teams Rigid teams with Percentage Rigid teamsRigid teamsDynamic teams

Scena rio 4

Time Performance/Minutes NA178060757058 Min Solution -Days NA2482101188478.5 442479 Allocation MethodNADynamic teams Dynamic teams Rigid teams with Percentage Rigid teamsRigid teamsDynamic teams

from one to another, and some have even used different methods to each scenario. When the allocation method is “dynamic”, this means that the subject has allowed a team to change its members from one task to another. “Rigid team” on the other hand, is when the team has its members from the start of the project working together till the end without any changes to its members. “Percentage” moreover, is when a resource is working simultaneously on multiple tasks, so each task has a percentage of his/her working time dedicated for completing this task. Another type that has been used by one of the subjects is individual allocation that considers allocating only one resource to each task. Moreover, another allocation was also made by allocating two resources to each task.

The overall inferences from the results shown in Table 41 can help to understanding the subjects’

behaviour corresponding to each scenario. From Table 41, it can be seen that as the scenarios’

level increases from one to four, the subjects in general spend more time to solve that scenario than the one before, leaving scenario four taking the highest time to solve. It is also noticeable that both T and N subjects were able to provide good quality answers as their project time estimate is too close to the optimized (optimal) one. It is interesting to consider why those two subjects were able to provide such good answers. This situation shows why the work for this thesis has supplemented phase one by the second phase of interviewing subjects to gain more explanation.

In addition, it can be seen that the subjects for scenario three have close results to the optimal one as by subject C, S, and E with 132 days, and to some can be even more identical to the optimal one such as subject T and N with 104 days. This reflects the simplicity of the scenario’s attributes and how the subjects are familiar with this situation so they have responded well to this scenario. It can be seen too that the dominant allocation method adopted across the subjects’ answers is the dynamic team method.

It is noteworthy that some subjects, were not only trying to minimize project time, but they were also trying to balance the allocation of twelve resources over the whole project, even with the dependencies between these WPs. This can be seen over the solutions of subject C, S, and E. For instance, subject S have created a list, titled “age allocation”, that provides the percentage of the work load, having the WP’s effort divided by the overall project effort, over the number of resources. Subject S used this list to know how many resource (s)he should assign to each WP. By using this way of allocation while balancing the more skilled resource to the most fitted WP, it could end with the result of subject C having the least skilled and productive for a WP that has a very high estimated effort amongst the others.

Analysis of PMs and SSSP approaches’ solutions

Comparing these results with the ones obtained by the nine SSSP approaches, one can see how in some cases some of the PMs have performed the resource allocation and project scheduling similar to the approaches, and in others the PMs have performed badly. For instance, if we look at the best SSSP approach like DiPenta01 for level one and the best PM’s result by subject N for the same level of (scenario 1), it is obvious that the subject was able to provide a good quality answer similar to the one of DiPenta01. However, the subject has consumed of one-hour time to find this answer. For the same level, in addition, five subjects have provided bad solutions, such as the answer of subject M with 115.5 days of estimated project time. The answer of this subject was based on having all the tasks starting at the same time where a single resource assigned to each, and for cases of a large task size such as task 2, 3, 4, and 5, two resources are assigned.

Therefore, the estimated project time defined by the subject is the maximum time length among all the tasks with 115.5 days of task 2. It is noticeable that the approach in Kang01 has provided exactly the same estimate as those PMs.

For level two, we can see that subject E has performed badly too providing 442 days of estimated project time. The subject has created the resources plan with a static view of resource allocation regardless the precedence relation between the project tasks that can allow for a dynamic allocation to be used. For instance, the subject’s plan has misused the resources who are assigned to proceed task(s) by being idle till the precede ones are finished. The subject has assigned three resources for task 2, two resources for task 3, two resources for task 5, and a single resource for each of the other tasks. This can show clearly how subject E had a static view of the resource planning, where resources are distributed to tasks without any care of schedule. It is noteworthy that none of the SSSP approaches described in Chapter 4 has provided similar to this bad estimate.

The worst estimate provided for this level of complexity is by Minku01 of 109.17 days, which is clearly show how a SSSP approach can help with resource planning and project time estimation, as PMs with years of experience are struggling to provide similar estimate.

For level three, the PMs were able to provide much better estimate than the SSSP approaches, as this level requires only a direct matching of resources’ skillset to tasks. The SSSP approaches, on the other hand, have provided a very fast estimate to project time. However, these estimates are very poor if we compare them to the PMs and optimal ones. So, for this level we can say that the PMs can outperform the approaches. Nonetheless, the worst project time estimate can be seen by three subjects’ (C, S, and E) solutions of 132 days. This poor estimate can be explained by the solution that subject S has provided. This subject has created his/her schedule by the assumption

of making the tasks work in parallel and assigning the competent resources who are possessing the required skill(s) to each task. As the project has four skillset categories, the tasks are divided by these categories, where each particular skillset is required by two tasks. On the other hand, only three resources possess the required skill(s) for each type. The subject’s decision was then on assigning two competent resources to the task that is larger in size among the set, and the third resource to the smaller one. Noteworthy that this has made the subject leaves task 4, which is smaller in size compared to the one that requires the same skillset but larger than many of the other tasks, assigned to a single competent resource. This has accordingly led to the longest time estimate among the parallelised tasks of 132 days.

For level four, both SSSP approaches and PMs have provided similar estimates, where the performance can be the key subject that shows the difference between the approaches and PMs.

In this case, it is obvious that the SSSP approaches are able to provide much faster estimate than the PMs. However, if we look at the best estimate among the PMs and approaches, we will find that one of the PMs (Subject N) has provided much better estimates than all the other approaches and PMs with 210 days of project time. On the other hand, a very bad estimate among the PMs’

can be seen by subject C, with 1188 days of project time. This estimate is based on distributing the resources with a percentage for participation to project tasks. The estimate by this subject had in general two resources assigned to each task, except task 5 with one resource. In addition, the resource who is possessing the required skill(s) was assigned to task 3 with 50% participation and the other who don’t possess these skill(s) was assigned to this task with 100% participation.

Moreover, the resources assigned to task 2 are possessing the required skills, however, they have been assigned with 50% participation to each. For task 4, one of the resources assigned is possessing the required skill(s) and the other is not, and both were assigned with 50%

participation. For task 6, the same theme of resource allocation to task 3 and 4 is used with two resources that one is possessing the required skill(s) and the other is not, but both are participating in this task with 100%. For task 5, a single resource is assigned to this task however, with 50%

participation. The time estimate for tasks 3,2,4,6, and 5 are 300, 223, 240, 45, and 380 days respectively. While these tasks forms the critical path of the project schedule, these estimates have all together formed the project time estimate of 1188 days.

This survey has shown how hard the SSSP problem is for a PM to consider its all parameter while focusing on the optimal project time target. The main theme that these PMs have used is balancing the amount of resources assigned to tasks without consideration of the idle time that could this assignment cause on the overall project time.

To express how an average PM would probably perform in solving the scenarios is to average the subjects’ solutions. The following Table 42 summarizes the average of the subjects’ responses compared to the optimized (optimal) solution provided for each scenario in Section 3.4.2. Note that the average of the responses is calculated based on six subjects as the seventh one has no response recorded for this phase.

Table 42: Evaluation of Study Subjects’ Performance

PM

challenges Attributes AVG of Subjects solutions represented in minutes. The second attribute of “min solution” is the average of estimated time span of the corresponding project scenario, represented in terms of days. It can be seen from Table 42 that scenario one and three on average are simpler for a PM to solve than when dependencies and/or skills and productivity, represented by scenario two and four, have to be taken into consideration. From Table 42, it can be seen that the subjects were able to provide answers for scenario four, which is a simple project in size, within 60 minutes on average, however, their average of estimated time span is the double of the optimal one with estimated 507.58 days to complete the project. That shows how hard the SSSP problem is. A question has arisen as to why for those two scenarios the subjects were able to provide good answers. To explore more about the subjects’ knowledge and background, and their demographic information the following Section 6.4.2 discusses these aspects.

6.4.2 Phase Two: Follow-up Interview for Qualitative Study

This phase is performed by interviewing the subjects with at least one hour time slot for each. The interview is carried out according to the subject’s meeting preferences either through face-to-face or internet-conferencing meeting. Three subjects were unable to have face-to-face meeting due to their location that is far from University of East Anglia (UEA).

The interviews combined follow-up questions within the interview-structure, and opened new room for discussion. The interview questions can be found in Section 5 of the Appendix. The results of these questions are discussed according to the questions’ categories, which are divided into seven. These categories are:

 The organizational size level that the subject represents.

 The subject’s project management experience.

 The project attributes that the subject thinks are important.

 The allocation method that represents what the subject practices.

 The considerations that the subject thinks a PM has to think about while forming a team.

 How the subject do his/her project scheduling. and

 The objectives that the subject thinks it represent the management goal(s).

For more details on these categories and their detailed questions the reader can refer to Section 5 of the Appendix. Note that subject (M) did not complete with us in this phase, and we were unable to explore any of his/her demographic information in this phase. In addition, subject T, N, C, S, and E are all from the same organization, but from different geographic branches’ locations. The outcomes from interviewing the six subjects for the organizational level and project management experience categories are presented in the following Table 43.

Table 43: Responses of Study Subjects for Organization Level and Experience Interview Categories

From Table 43, it can be seen that the main participants were from large size organizations.

However, their years of experience vary from one to another. The least experienced in management can be seen in the table as subject D, whereas subjects T and N are the most experienced amongst them all with four years difference between them. It can be seen too that the subjects who represents large organization with large project and teams’ size are combining agile and waterfall models in their projects. This confirms the observation reported in [130] that the

Subjects Study AspectsDT NCS E Organization size level Medium size Large financial organization

Large financial organization Large financial organization Large financial organization Large financial organization

Expe rie nce

Years of PM experience 62630171017 Development methodologyAgile Agile and waterfall Waterfall Agile and waterfall Agile and waterfall Agile and waterfall Project Sizes Medium size team Large and multiple teams Large and multiple teams Large and multiple teams Large and multiple teams Large and multiple teams

large organizations are in favouring of using hybrid methods, which combines different development methodologies as waterfall with other(s), over the agile approach.

The responses of subjects for the aspects of project attributes and resource allocation are depicted in the following Table 44.

Table 44: Responses of Study Subjects for Project and Resource Allocation Attributes Interview Categories

Subjects

teams Rigid teams Rigid teams Rigid teams What do you

interpretation. However, each has represented productivity according to his/her practice. For example, subject N has described productivity as the analogy of a resource compared to his/her colleagues with respect to learning speed and synergy with the team members. Others have almost the same concept as they represent productivity by the speed of developing story points, and in relation to other team members, as with subjects C, S, and E.

In addition, it can be seen from the Table 44 that half of the subjects claim the existence only of scenario two, which only consider dependency between project tasks. It is worth mentioning that they all understand that the resources they have in their organization or company are sharing similarity in terms of skills and productivities, as the HR department applies standards and quality check. However, the other half of the subjects support the existence of scenario four as a reality of

complexity level they face within their projects, and they do believe that the resources differ in their skills and productivity. What is also important to mention, that all the subjects do not use productivity as a factor while they staffing and scheduling their projects.

The method adopted by each subject differs from one to another for their project resource allocation. Subject D leaves the resource to decide their tasks. For subject T, (s)he allocates the resources to teams after creating different permutations so (s)he can decide which one is better based on the team velocity. Subject N allocates his/her resources to teams according to their skills, but when the expertise is required for another team working on another task, they can change teams’ members. A consensus can be seen with subjects C, S, and E to allocate resources to a single team where the personality factors plays the core role to create a coherent team, and to avoid any

The method adopted by each subject differs from one to another for their project resource allocation. Subject D leaves the resource to decide their tasks. For subject T, (s)he allocates the resources to teams after creating different permutations so (s)he can decide which one is better based on the team velocity. Subject N allocates his/her resources to teams according to their skills, but when the expertise is required for another team working on another task, they can change teams’ members. A consensus can be seen with subjects C, S, and E to allocate resources to a single team where the personality factors plays the core role to create a coherent team, and to avoid any