Obtaining Results for Task 1: Identifying the Usefulness of Each Document

Based on the responses received for task1 through the questionnaire (Appendix B), where each respondent was presented the list of 23 documents (identified from round 1of data collection) to be assessed for usefulness of each document, the following heuristics were used to identify the degree of usefulness of each document for each role.

Heuristic 1: If a particular document receives 75% or more votes for a certain role,

then, that document is considered to be a very useful document for that particular role.

Heuristic 2: If a particular document receives less than 75% but 50% or more votes for a certain role, that document is considered to be a useful document for that particular role

Heuristic 3: If a particular document receives less than 50% votes for a certain role,

that document is considered to be a not useful document for that particular role (hence not considered in the analysis report).

The votes received for each document, against each role, were counted and divided by the total number of respondents to determine the proportions (the percentage figures) to apply the above heuristics. Table 4.4 shows a portion of researcher’s data collection sheet, containing the votes. Four documents and three roles are recorded as a sample.

Table 4.4: Sample of Total Counts Received for Four Documents for Each Role

Document Customer Product Owner Team Lead

Maintained Product Backlog 8 16 15

Specifications 10 10 10

Standards 8 10 10

Stakeholder requirements 13 13 11

In determining the proportions, although the panel size was 19, the effective number of respondents had to be considered as 18, because the responses given by one respondent was thoroughly incomplete; these incomplete responses were not counted in composing the spreadsheet containing the votes. As an example, the proportion of votes received for the document ‘Maintained Product Backlog’ as being useful for customers was 8/18, which is 4.4%, hence not deemed a useful document for customers, based on the heuristics used by the researcher.

Table 4.5, which is a screen dump from the researcher’s spreadsheet, depicts the usefulness of each document for each role. The reader should note that the blank cells (dash) in Table 4.5 denote documents that are ‘not useful’ to a particular role.

Table 4.5: The Degree of Usefulness of Each Document for Each Role

Product Owner Scrum Master Å---Development Team ---Æ

The next step is to combine the results in Table 4.5 on the basis of three key role categories—Product Owner, Scrum Master, and Development Team—in Scrum (see Table 4.1). Both Product Owner and Scrum Master Categories contain two roles (hence two cell entries in Table 4.5) while the category Development Team contains eight roles (hence eight cell entries in Table 4.5). The two cell entries (each cell correspond to a particular role) on the degree of usefulness for each category, Product Owner and Scrum Master (Table 4.3), were combined into one entry using the “AND” operation where two inputs get resolved into one output (Batten, 1973). The operation works similar to "ADD" operation, which takes two values and produces one result (Janikow & Michalewicz, 1991). In this technique, precedence is given to the value which has high value (e.g. Very Useful over Useful). Table 4.6 depicts the criteria used for combining two cell entries in Table 4.5 into one output (entry). As regards the category Development team (Table 4.1), since there are eight cell entries corresponding to eight roles, the criteria (Table 4.6) were used in pairs to combine into a single resultant output (i.e. 8 cell entries Æ 4 cell entries Æ 2 cell entries Æ a single cell entry).

Table 4.6: Cell Entry Combining Criteria

Cell 1 Entry Cell 2 Entry Combined Output

Useful Useful Useful

Useful Very Useful Very Useful

Useful Not Useful Useful

Table 4.7 depicts the results after combining the information in Table 4.5 using the combination criteria described above. Accordingly, Table 4.7 depicts the degree of usefulness of each document to each key role category in Scrum.

Table 4.7: The Degree of Usefulness of each Document for the Key Scrum Roles

Document

Degree of Usefulness for each Role Scrum Master Product

Owner

Development Team

1. Maintained product backlog ** **

2. International Specifications ** ** *

3. Standards & Regulations ** ** *

4. Stakeholder/Customer ** ** **

5. Business Flows ** ** *

6. Road Map Items (RMI) * ** *

7. Features ** ** *

8. Test Verification Report * * **

9. Sprint demonstration recordings * * *

10. Retrospective notes ** **

11. Training materials

12. Risk Analysis reports ** **

13. Technical Design * * **

14. DB designs **

15. Developer documentation **

16. Traceability Matrices * * *

17. Unit Test and test reports **

18. Source codes files **

19. Bug tickets * * **

20. Test plans * * **

21. Bug Verification report * **

22. User manuals **

23. Companywide quality procedures * * **

Notes: * Indicates a Useful document based on the votes received

** Indicates a Very useful document based on the votes received No asterisk (*) indicates being not Useful, based on the votes received

4.2.2.2 Obtaining Results for Task 2: Identifying the Importance of Each Document

In task 2 of round 2, through the questionnaire (Appendix B), each respondent was presented the list of 23 documents identified from round 1of data collection (with option for the respondent to add more documents to the list) and was asked to rate each document for each role using a 5 point rating system: 1 being extremely important, 2 being important, 3 being neither important nor unimportant, 4 being not important, and 5 being extremely unimportant; a numbering (ranking) system is used to judge the importance of documents because unlike usefulness, importance is a value judgement

(see section 4.2 for the definitions of usefulness and importance). As a sample, Table 4.8 shows the votes received for 1s (extremely important) through to 5s (extremely unimportant) for four documents (there are 23 documents altogether) and three roles (there are 13 roles altogether). Based on these votes (values for 1s through to 5s) the weighted average score was calculated for each document for each role. These weighted averages were then and rounded-off to the nearest integers to determine the degree of importance of each document for each role. As an example, the weighted average value (rounded-off to the nearest integer) for the document ‘Product Backlog’ for the role ‘Customer’ is 3, which translates to ‘neither important nor unimportant’, based on the scale used. The following sample calculation.

Weighted ..Average_{P r oductBacklog,Customer} [(1* 5) + (2 * 4) + (3 * 7) + (4 *1) + (5 * 2 )]/19 48/19 2.52 3 (Rounded)

Table 4.8: Sample Scores Received for Each Document

Document Customer Product Owner

Maintained Product backlog 1=5; 2=4 ;3=7 ;4=1,5=2 1=17; 2=0 ;3=0 ;4=0,5=2 Specifications 1=4; 2=6 ;3=6 ;4=1,5=2 1=6; 2=11 ;3=1 ;4=0,5=1 Standards 1=5; 2=8 ;3=2 ;4=1,5=3 1=8; 2=7 ;3=2 ;4=0,5=2 Stakeholder requirements 1=14; 2=2 ;3=1 ;4=5,5=2 1=17; 2=0 ;3=1 ;4=0,5=1 Business flows 1=8; 2=6 ;3=1 ;4=2,5=2 1=10; 2=7 ;3=0 ;4=0,5=2

In this study, the focus is on identifying important documents (rounded-off weighted average score of 4) and very important documents (rounded-off weighted average score of 4). Therefore, any rounded off weighted average score of 3, 2 or 1 rating of importance for a particular document for as particular role was treated as an unimportant document.

Table 4.9, which is a screen dump from the researcher’s spreadsheet, depicts the degree of importance of each document for each role. The reader should note that the blank cells (dash) in Table 4.9 denote documents that are unimportant to a particular role.

Table 4.9: The Degree of Importance of Each Document for Each Role

Product Owner Scrum Master Å---Development Team ---Æ

Table 4.10 depicts the degree of importance of each document to each role category (i.e. Scrum Master, Product Owner, and Development Team) after combining the information in Table 4.9 using the combination criteria that was used to combine similar information on the degree of usefulness of documents. Accordingly, Table 4.10 depicts the degree of importance of each document to each key role category.

Table 4.10: Degree of Importance of each Document for Each Key Scrum Roles

Document

Degree of Importance for Each Role Scrum Master Product

Owner

Development Team

1. Maintained product backlog * * *

2. International Specifications * * *

3. Standards & Regulations * * *

4. Stakeholder/Customer requirements * * *

5. Business Flows * * *

6. Road Map Items (RMI) * * *

7. Features ** ** *

8. Test Verification Report * * **

9. Sprint demonstration recordings * * **

10. Retrospective notes * * *

11. Training materials * *

12. Risk Analysis reports * *

13. Technical Design **

14. DB designs **

15. Developer documentation **

16. Traceability Matrices * * **

17. Unit Test and test reports **

18. Source codes files * * **

19. Bug tickets * * **

20. Test plans * * **

21. Bug Verification report **

22. User manuals * * **

23. Companywide quality procedures * * **

Notes: * Indicates Important document based on the votes received

** Indicates Very Important based on the votes received

No asterisk (*) indicates being Not Important, based on the votes received