RESULT INTERPRETATION AND PROCESS AMENDMENT

This section presents the feedback received from the experts for each of the development stages along with the corresponding comments/opinion given by the experts for improving the process. The experts gave their

84

opinion/comments for improving few stages which they were not completely satisfied.

The results are discussed for each of the development stages, taking into account the feedback received from individual experts. For each stage, the individual experts’ feedback/concern is reported along with the reason for concern. Further, a response is provided for each concern raised by the experts.

Finally, the proposed OSSD process is amended depending on the feedback/

comments and is presented accordingly. These amendments include - an alteration to the development stage as a whole, alteration with regard to the activities carried out or the frequency with which the development stage/

activities of a development stage are preformed, wherever necessary.

7.4.1 Feature Selection 7.4.1.1 Expert 1:

Feedback/Concern: According to the first expert, the ‘feature selection’ stage was not realistic, especially because the proposed process suggests that the entire community should participate in feature selecting before it is developed.

Reason for Concern: Going through the feedback of expert 1, it could be made out that that the expert had compared the proposed process with the OSS that the expert had been working with. His major concern was that the features were developed only when funding was approved.

Response to the Concern: The issue of funding approval is not true for most of the OSS products, as many of them are developed for addressing the immediate need and hence, the OS developer does not wait for funding. The OS community mainly invests time and effort in developing a feature. Hence, the expert’s concern would not be significant, especially if the OS feature is of high importance.

7.4.1.2 Expert 2:

Feedback/Concern: According to the second expert, there should not be an over dependence on the core team.

85

Reason for Concern: According to this expert, the proposed way of feature selection might be appropriate for small and very crucial features but might not suit a very large feature. In fact, this might make the core team to spend lot of time in feature selection.

Response to the Concern: Although he points out the core team cannot be given too much authority, he also mentions that only the core team is capable and hence, should be responsible for selecting and finalising the feature for development. This was a self-contradicting statement.

7.4.1.3 Expert 3:

The third expert did not suggest any changes to this stage.

7.4.1.4 Amendment in Proposed OSSD Process:

The feature selection process was explicitly divided into two phases, based on the feedback of the second expert. In the first phase, the OSS community would take a lead in selecting the feature for development (for instance, voting and suggestions). In the second phase, the core team would take the final call.

This would provide equal opportunity to both the core team and the community in selecting the feature for development and would not make anyone in the OSS community to feel less important.

7.4.2 Requirement Specification Stage 7.4.2.1 Expert 1 and Expert 3:

The first and third expert did not suggest any changes to this stage.

7.4.2.2 Expert 2:

Feedback/Concern: Freezing the feature requirement specification is too strict for an OSS environment.

Reason for Concern: Freezing the feature requirement might result in the feature not being improved/ elaborated. Further, in an OSS community, the

86

document should be kept open even during its development to the entire community.

Response to the Concern: The proposed OSSD process suggests freezing the feature requirement specification only for the immediately occurring feature release. Also, the core team can freeze the requirement only if the entire community or at least the working team agrees on the proposed feature specifications. In addition, it is important to understand that the proposed OSSD process visualises the OSS development as a continuously evolving process. Hence, freezing the requirement would assist the community to focus on developing the feature. Notably, if someone requires the feature to be elaborated later, the OSS community would allow them to raise a request for new development.

With regard to keeping the requirement document open at all stages to the entire community, it is not a major issue and could be decided by the respective OS community. However, an important point to be noted is that publishing an incomplete, unclear document might create confusion among the entire community. This was the main reason why the proposed OSSD process suggested making the document open to the community once the initial version is developed. The community could then provide suggestions/

recommendations for improving the same.

7.4.3 Design Specification Stage 7.4.3.1 Expert 1:

Feedback/Concern: The first expert again had a comment with regard to arranging the finances for the design activities.

Response to the Concern: As previously mentioned; not all the OSS developing community, aim to have finances arranged for developing an OSS feature.

87 7.4.3.2 Expert 2 and 3:

Feedback/Concern: Allowing the entire community to validate the design specification may be too tedious. Hence, only the core team should be given full authority to validate the design process.

Response to the Concern: The proposed OSSD process suggested a similar approach, with the additional prospect for the OS community to give their suggestions with regard to improving the design. This was because; there could be occasions where the unseen problems could be identified by third parties.

7.4.3.3 Amendment in Proposed OSSD Process: Having the design specification validated by the entire OSS community is not mandatory but would be highly recommended. The decision could be taken appropriately by the core team depending upon the feature under development.

7.4.4 Implementation Stage

All three experts stressed and acknowledged the importance of the unit-testing strategy adopted in this stage, in the proposed OSSD process.

7.4.4.1 Expert 1 and Expert 3:

The first and third expert did not suggest any changes to this stage.

7.4.4.2 Expert 2:

Feedback/Concern: The proposed OSSD process should emphasis on publishing the inter-mediatory milestones that are usually achieved during the software implementation stage.

Response to the Concern: This is an important point to be considered, as the proposed process only suggested periodic update about development stage.

7.4.4.3 Amendment in Proposed OSSD Process:

The proposed OSSD process is modified such that, as and when the community achieves a milestone (even if it is an intermediary one); it should

88

be published to the entire community. This would have an added advantage that the development progress would be periodically revealed to all stake holders.

7.4.5 Software Testing Stage 7.4.5.1 Expert 1:

The first expert did not raise any major concern and did not suggest any changes to this stage.

7.4.5.2 Expert 2 and 3:

Feedback/Concern: Providing explicit criteria for testing before the OS community start the actual testing might limit the overall testing scenario.

Further, the community as a whole should use a bug tracking system.

Reason for Concern: The tester might not look beyond the test case that is already provided. The bug tracking system would ensure that all the bugs are fixed.

Response to the Concern: Providing an explicit criterion for testing would enable the new comers is the OS community to kick start their testing expedition.

7.4.5.3 Amendment in Proposed OSSD Process:

The proposed OSSD process would stress the OSS community to incorporate a bug-tracking system. Further, the process suggests performing testing activities during various stages of development especially when multiple components interact with each other.

7.4.6 Integration and Release Stage

One of the experts overlooked the necessity of verifying the software unit produced with the requirement and design before release. On the other hand,

89

the other two experts stressed the importance of verifying it against the requirement and design.

Hence, due to the majority of experts (two out of three) are in favour of verification, the verification stage was maintained as originally proposed.

7.4.7 Summarizing the Process Amendment

All three OSS experts were fairly content with the proposed OSSD process and also provided their feedbacks/comments for each individual development stage. Out of six development stages, significant concerns were raised for three stages. They were Feature Selection Stage, Design Specification Stage and Software Testing Stage. Further, there was a concern with regard to the

Stages Amendment(s) to the Process Activities

Feature selection stage (Two-phase

stage with iterations)

Feature selection is divided into two phases.

(i) In the first phase, community selects feature for development.

(ii) In the second phase, core team takes the final call on all the selected features before adding it to the roadmap.

Design specification stage (single-phase stage

with iterations)

Recommends that the entire community take part in validation of the design documents but is optional.

Implementation stage (single-phase

stage with iterations)

(i) Publish all the inter-mediatory milestones to the entire community.

(ii) Publish updates periodically regarding the development.

Software testing stage (single-phase

stage)

(i) Stresses on the usage of bug-tracking systems.

(ii) Recommends performing testing/review activities at different stages of development rather than putting it as a last one.

Table 7.2 Amendments to process activities

90

The amendment for Feature Selection stage includes dividing this stage into two sub-stages. In the first sub-stage, the OSS community would select the feature for development. In the second sub-stage, the core team would decide on the features based on the community’s choice. For the Design Specification Stage, the proposed process recommends that the entire community take part in validation of the design documents. However it is not mandatory and could be decided by the core team based on the feature under development. The amendment to the Implementation Stage includes publishing the inter-mediatory milestones along with the periodic updates. Finally for the Software Testing Stage, the proposed process stresses on using bug-tracking systems.

This would help in tracking all the important bugs so that it can be fixed before the release. Further, the proposed OSSD process suggests performing testing activities during various stages of development, especially when multiple components interact with each other.

Notably, a concern was raised by one of the expert on the Requirement Specification Stage and Integration and Release Stage. However the reason given by the expert was either inconsequential and/or biased. Therefore, no changes were recommended to these two development stages.

7.5 Summary

In this chapter, various validation approaches appropriate for this research work were discussed. Further, an explanation was provided on the selected

‘expert review’ approach followed by a detailed explanation on the questionnaire-based technique that was selected to realize the ‘expert review’

approach. The feedback from the experts were then interpreted along with their major concerns and the reasons for those concerns, Importantly, all the major concerns were addressed and the proposed generalized OSSD process was improved as when and where required. The next chapter would conclude the thesis and particularly, revisit the research questions. Notably, it would point out the advantages and limitations of this work, along with the potential future research direction.

91