A total of five Scrum tools were evaluated that feature a task board. Criteria for evaluating those tools were defined in section 4.2. Table 17 summarizes the outcomes of these criteria for all five tools. The table uses different kinds of ratings:
• A
✔
or a✗
to indicate whether a certain aspect is available or not.• N/A if an aspect is not applicable.
• ++ (very good), + (good, o (neutral), - (bad) & -- (very bad) to indicate the level of a certain aspect.
• A number to indicate the amount of an aspect.
A green colored cell means that a tool is better than the remaining four tools in terms of that very aspect. If two tools share the same mark, the cells is not colored at all.
Criterion JIRA ScrumWorks Pro
TFS / Urban
Turtle Rally VersionOne Usability
Dashboard
✔
✔
✔
✔
✔
Number of Dashboard
information sources 4 1 2 8 5
Tasks show the user
/ avatar / description
✗/✔/✔
✔/✗/✔
✔/✗/✔
✔/✔/✔
✔/✗/✔
Drag & Drop
✔
✔
✔
✔
✔
Keyboard Shortcuts
✔
✗
✗
✗
✗
Context-based help
✗
✗
✗
✗
✗
Creating tasks
✔
✔
✔
✗
✔
Criterion JIRA ScrumWorks Pro
TFS / Urban
Turtle Rally VersionOne Resources
Minimum resolution XGA XGA < XGA XGA SXGA
Browser
requirements + + ++ - o
Quality
Phone support - + N/A o o
Response time + + + + +
Integration
Number of products
in vendors portfolio 13 4 3 1 1
API
✔
✔
✔
✔
✔
Connectors for third
party products
✔
✔
✔
✔
✔
Number of available
plugins 400+ N/A N/A 89 N/A
Functionality
Full screen mode
✗
✗
✗
✔
✔
Columns customizable
✔
✔
✗
✗
✗
Font size - -- - + -- Story estimation (Storypoints/Time)✔/✔
✗/✔
✔/✗
✔/✗
✗/✔
Task estimation (Storypoints/Time)✗/✔
✗/✔
✗/✔
✗/✔
✗/✔
Motivation Comments✔
✔
✔
✔
✔
Task dots✗
✗
✗
✗
✗
Automatic page refresh✗
✔
✗
✗
✗
Acoustical / visual trigger✗/✗
✗/✗
✗/✗
✗/✗
✗/✗
Costs Hosted ++ + N/A - o Download ++ + o N/A -Counting the amount of green cells produces the following outcome:
JIRA ScrumWorks Pro
TFS / Urban
Turtle Rally VersionOne
Green cells 6 2 2 3 0
Table 18: Scrum Tools Results
However, this outcome does not mean that JIRA is the best choice for all companies that are interested in using a Scrum software tool. Instead, each tool has certain aspects in which it outperforms the others:
• JIRA is the cheapest option and offers a lot of functions, but is not yet as mature as the other tools.
• ScrumWorks Pro uses an outdated approach by splitting its tool in two client applications, but leads quality-wise.
• Microsoft Team Foundation Server with Urban Turtle is frugal when it comes to resources and offers a great integration (however Microsoft-only), but scores quite low when it comes to usability.
• Rally has the greatest maturity and a lot of well-thought features. It is the only tool that uses a reasonable font size for its task board. However, it is also the most expensive solution for hosted tools.
• VersionOne also offers a lot in terms of functionality, but is not as user-friendly as Rally. The information dashboard is a unique feature but comes at a cost, as VersionOne is the most expensive solution for downloadable tools.
Although all five tools are capable of representing a Scrum task board electronically, this tool evaluation backs the paper vs. software evaluation in section 3.3, namely that all software-based task boards score low when it comes to accessibility and motivation. Not all tool vendors have grasped these aspects: only two out of five vendors have implemented a full screen mode and only one vendor uses a big font size (although this font size is still too small to support good readability). In combination with big screens that lack any way of inputting data, software-based task boards cannot compete with their paper-based pendants as they are reduced to information panels that are ignored after a short time. Acoustic or visual triggers could create task board awareness; unfortunately none of the five vendors has implemented anything at all that comes even remotely close. On top of that, even important basic functionality like automatically refreshing the screen when changes are made is not implemented in any of the tools. Instead, only ScrumWorks Pro at least refreshes the task board every 30 seconds.
However, the evaluated tools also back up the higher score of software-based task boards when it comes to archiving. All five tools automatically archive sprint data, three of the tools allow checking back on previous sprint data when viewing the task board, and the remaining two have a dedicated function to do so. The evaluation has also shown that tool vendors make extensive use of connecting their product with software from other vendors to support company workflows. All vendors have made sure that the products within their portfolio are able to exchange data with each other; each tool features an API as well as various connectors, some of the tools also feature plugins or apps that allow external developers to create additional functionality to improve these tools. By doing so, tool vendors try to compensate missing features that are currently not available within software-based task boards.
Wrap Up
4.5
The purpose of this chapter was to reach the second part of the first half of the research goal by giving an answer to research question 4.
In section 4.1, criteria for selecting Scrum software tools that form the basis for this evaluation were defined, since a great amount of Scrum tools exist – too much to evaluate them all within the scope of this master thesis project.
Section 4.2 reused some criteria from the software versus paper analysis in chapter 3, next to defining new criteria that were needed to perform a dedicated software versus software evaluation.
The actual evaluation of Scrum tools that are able to represent a task board electronically was performed in section 4.3.
Finally, section 4.4 concluded the results and showed that the evaluation backed up the analysis findings in chapter 3 next to the fact that the representation of a software-based task board is in some aspects limited and cannot outperform its paper-based pendant.
By answering research question 4, the first half of the research goal, namely the task board analysis has been completed. The following chapter will solve the second half of the research goal by defining company guidelines.