As stated before, in each project, link shall be made between product features, require-ments and test cases. Thus, it should be easy to understand, what each test must validate and purpose why each requirement exists (it should be linked to a product feature).
Also, with proper traceability information, it should be easy to spot not tested require-ments or product features (or those with few tests).
However, traceability information is not well displayed to managers/users in exist-ing tools, which causes unspotted errors in projects and/or can decrease the quality of final product. It needs to be improved!
As a solution, this thesis proposes a dashboard with overall Status of Testing. It is described in this section.
5.2.1 Dashboard Principles
The dashboard is based primarily on traceability information required by ASDP.
These traceability links can be followed in two directions. First, it starts by high level definition of product features and drills down into details – low level requirements, associated test cases which validate the functionality, several executions of each test case called test runs, eventually reaching defects found during testing. Thanks to these links, it can be shown for each product feature:
• how well are related requirements covered by test cases, 64
5. DASHBOARDDEFINITION
• how many related tests passed, failed or have not been executed yet,
• how many test runs are scheduled for execution,
• how many defects are related to this product feature and whether they are re-solved already.
Next, it follows the links in the up-ward direction, propagate the results of each level and draw conclusions from it. It can be for example said that a failed test case should be executed again, when all its associated defects have been resolved. Additionally, for each product feature it can provide conclusions like:
• related requirements are not well covered by test cases,
• related test cases are not yet scheduled for execution,
• related test runs are scheduled, but still pending to be executed,
• related tests have failed and thus need to be re-executed after defects has been resolved,
• there are open defects, which need to be resolved,
• and possibly others (in the future).
Mockup in Figure5.2depicts basic example how such a dashboard would display traceability information.
Figure 5.2: Mockup of Traceability Dashboard – basic idea
Drill-down
When the manager identifies a possible problem displayed on the dashboard, he needs to be able to drill down into details to locate the problematic documents and ideally fix them. Figure5.3depicts an idea how such a drill down mechanism could work.
Also, filtering capability seems to be useful for providing answers to particular questions the manager may need to answer.
65
5. DASHBOARDDEFINITION
Figure 5.3: Mockup of Traceability Dashboard – Drill down capabilities
5.2.2 Alternative View
As stated Chapter4 chapter, there are many projects without any product features de-fined. The dashboard should provide a solution also for them. Luckily, the dashboard is pretty flexible so a fallback solution can be done easily. Instead of displaying product features in the left-most column, they can be skipped and instead folders with require-ments displayed. The rest would be the same.
Additionally, some projects have use cases as the high level description of their project. There are then linked to individual, low level requirements. Because both prod-uct features and use cases (and most others) are stored in the same way in Contour, they can be easily swapped product features for use cases as well. Figure5.4presents how this dashboard may look if based on use cases.
As can be seen, it is almost the same view.
5.2.3 Unassociated Requirements and Test Cases
We decided to re-use the folder structure from Contour, because the users are already familiar with it. Therefore we should display the items in the same structure. Next to each folder, there is the total number of items in it, which are not connected to a product feature (or a requirement respectively). Such a view is depicted in Figure5.5.
The user should be able to expand/collapse individual folders. Leaf nodes shall be hyperlinks into Contour, so that the user can just click on the item and add the link. Ideally, there might be an option to link the requirement/test case directly from the dashboard, so that the user does not need to switch to Contour. Also, it would be great to link multiple documents at once.
However, if we would like the dashboard to create new links, we would need write access to Contour. This would be more complicated and therefore we have not tried it 66
5. DASHBOARDDEFINITION
Figure 5.4: Mockup Traceability Dashboard – Based on Use Cases in the prototype.
5.3 Summary
This chapter discussed several metrics and indicators which should help the managers to answer questions they have. It also proposed mockups of a dashboard, which would present these metrics clearly, so that managers can combine the information from all of those at once.
67
5. DASHBOARDDEFINITION
Figure 5.5: Unassociated documents – Requirements and Test Cases
68
Chapter 6
Implementation
Previous chapter defined several important metrics and a dashboard to display them to the users. In order to verify usefulness of this dashboard, working prototype needs to be implemented and evaluated by users.
This chapter describes prototype implementation of the dashboard and provides some examples of its usage.