Modify staging

data model to include transaction revenue. Validate that transaction revenue is properly calculated Develop profit margin table in staging Modify profit margin table to include net profit margin lookups Determine the logic calculating gross profit Develop ETL logic to calculate transaction gross profit Add other necessary fields to transaction table for net profit calc. Creat BI report sh owing gross profit by transaction Validate that gross revenue is properly calculated Associate transaction fact table with date dimension Ensure that net and gross profit measures appear correctly in transaction fact

Develop a new user report that shows customer profit by store Validate that customer net and gross profit are correctly calculated Verify calculation of net and gross profit As a profitability analyst I need the ability to examine net

profit per customer per transaction for any customer so that

I can identify the customers who might become more profitable. 13

ETL V lid t th t

As a profitability analyst I need the ability to examine gross

profit per customer per transaction for any customer so that

I can identify the most profitable

customers. 8

ptg6843605 TENETS OF AGILITY 53

even a temporary rise in trajectory. As long as the team understands and properly manages the impact of these anomalies, there should not be cause for alarm. Remember, the primary goal is the completion of working fea- tures. Task completion is simply in support of that goal.

Many Agile teams use electronic tools such as Microsoft Excel to manage their burn-down charts. At the time of this writing there are a number of fairly elaborate Excel solutions and tools for tracking burn-down, both free and commercial, available on the Internet. Likewise, most of the commer- cial and open-source Agile project management tools available provide a burn-down tracking feature. Whether you elect to use an electronic burn- down or a manual one like the example in Figure 2.11, this tool is most effective when it is updated daily and is made visible to the entire team.

Agile Analytics Practice: Team Self-Monitoring

Teams monitor their own velocity, burn-down, and card walls on a daily basis. These visible controls are ever present in the team’s workspace. Outsiders can see them but should not use them as performance metrics.

20 15 10 5 0 Number of T asks Remaining Iteration Days Mon Tu e We d Thu Fri Mon Tu e We d Thu Fri

ptg6843605

Monitor Story Completion, Not Task Time

A brief observation about the tasks in Figure 2.10: Even though they are not particularly readable, you may have noticed that they do not include time or effort estimates. It isn’t clear whether some tasks are more or less time- consuming than others. Although there is a temptation to estimate and track task completion time, doing so has the undesirable effect of taking the team’s focus off of what is important: completed features.

Instead, the team should strive to define tasks that are small enough to be completed in less than one workday. Some will take less than an hour and others will consume nearly a full day of effort. Many teams establish the practice of defining tasks that are expected to take less than half a day to complete. Doing so enables the development team to self-manage their tasks as a simple to-do list. As with any to-do list, new tasks can be added to the list, and tasks that become irrelevant can be removed without a burdensome change procedure.

Also, you may have noticed that owners are not assigned to the tasks in Fig- ure 2.10. As previously mentioned, effective Agile teams operate as a collec- tive. By maintaining a collective set of tasks, any member of the team who is available can complete any task provided he or she has the necessary skills to do so. In practice it may be implied that the ETL developers will handle ETL tasks, data modelers will handle modeling tasks, and so forth. How- ever, Agile teams work best when they behave like generalists rather than specialists. It is not uncommon on a healthy team to see a DBA stepping up to write a bit of ETL code, or an ETL developer stepping up to write some PL/SQL if needed. This promotes better cross-disciplinary understanding of the entire solution, and team members have the opportunity to develop new skills. During each daily stand-up meeting the team cooperatively decides what should be done, and team members volunteer for the tasks that they are best suited to complete. The project manager facilitates this process.

One Agile Analytics team of which I was a member established the practice of assigning an estimated effort of 1.0 hour to every task. It was our goal to define tasks smaller than half a day. Some took less than one hour, and some took more. We did not establish this practice until well into a multi- year, multirelease project. In the beginning our team estimated task effort in person-hours, and, like many teams, we tracked actual time versus esti- mated time. Like most effort estimates, ours were routinely incorrect. After shifting to the simpler practice of not estimating task effort and assigning 1.0 to every task, we discovered the following benefits:

ptg6843605 TENETS OF AGILITY 55

 Our iteration-planning time became much shorter because we didn’t have to estimate effort for every task.

 We were liberated from the previous impact of adding, removing, and modifying tasks once the iteration was under way. Prior to our simpler approach task changes messed up the burn-down chart and caused a bit of grief for the project manager.

 Our entire team shifted its focus more toward story completion and away from task completion.

 Tasks assumed thei r proper role as to-do items rather than measures of productivity.

Not long ago I was asked by a client to conduct an assessment of their Agile Analytics efforts. Although they were doing many things very well, they were experiencing a particular problem during iteration planning. Roughly 90 percent of their iteration-planning time was spent in task definition and estimating. The project manager was intent on aligning task time estimates with the personnel time allocated to the project. So, if Russell was allocated 25 hours per week to the project, all of the tasks assigned to Russell had to add up to 25. If Russell was overtasked, he had to lower estimates or elimi- nate tasks; if he was under 25, he had to account for how he would spend his available extra time. Surprisingly little attention was given to understanding the user stories, story acceptance criteria, or related business drivers.

I observed several interesting effects that this task-centric planning caused. First, the team members were frustrated because they didn’t see the value in this type of planning. Second, the team tended to sandbag its task time esti- mates so that they added up to the “right” number. Third, it promoted indi- vidualism over team cooperation because individuals were assigned “their tasks” to fill their personal time capacity. Finally, it was not a collaborative team planning effort; it was a grilling of the team by the project manager to get the information he needed to plug into his planning tool. In the end the team left the meeting exhausted and frustrated—and that was on a Monday morning at the beginning of a new iteration. Not a good way to start!

However you choose to tailor your iteration planning and daily tracking practices, make sure that your team’s primary focus is on the completion of working features in the DW/BI system. Keep in mind the first principle of Agile Analytics: “Our highest priority is to satisfy the BI user community through early and continuous delivery of working user features.”

ptg6843605 Agile Analytics Practice: You Get What You Measure

Agile Analytics teams and Agile leaders measure what is important. That is principally the delivery of high-quality, high-value, working BI features. Progress and performance metrics should be built around value delivery and quality rather than on such things as task completion and velocity.

W

-U

Creating a weekly live television variety show requires a lightweight approach in which time is not wasted on things that don’t directly contrib- ute to the show. It requires lots of face-to-face teamwork among writers, actors, stagehands, and other people. It requires creative ideas and visions, followed quickly by the exploration of those ideas to see if they are good enough to keep. It requires adapting to the feedback of test audiences and to external events. Creating a live TV show requires airing at the scheduled time whether or not the creators need more time. And by the way, the show had better consistently get high network ratings to stay alive. This type of pressure quickly forces creative teams to abandon practices that don’t help and to emphasize and fine-tune the ones that do.

This chapter on Agile Project Management lays the foundation and frame- work for a set of lightweight Agile Analytics practices that are introduced throughout the remainder of this book. Agile Analytics is formed of a blend of practices from XP, Scrum, Crystal, FDD, and APM. These prac- tices have been adapted to the unique challenges of data warehousing and business intelligence. They require discipline and rigor but should never get in the way of the goal of producing high-value, working BI features every two weeks. Good Agile Analytics teams tailor these practices to meet their needs. They abandon practices that don’t help, and they emphasize and fine-tune the ones that do.

In this chapter we have examined the core differences between phased/ sequential processes and iterative/incremental/evolutionary processes. We have explored the key differences between traditional Plan A Do cycles and the APM cycle of Envision A Explore. These key differences primarily reflect a mind-set shift in our approach to project planning and manage- ment. This mind-set shift is the foundation of Agile Analytics. Built on this foundation is a set of planning practices and execution practices that rep- resent a significant change in the way we build data warehousing systems.

ptg6843605 WRAP-UP 57

This approach embraces changing requirements. It seeks user feedback early and often. And it focuses on the early delivery of business-valued features.

This overview of Agile Project Management is not a substitute for a deeper study of APM as presented in Highsmith’s book. APM introduces a well- defined collection of phases and practices for effective project visioning, speculation, exploration, adapting, and finishing. Agility requires a very different style from the project manager than traditional projects. The proj- ect manager becomes a team facilitator and enabler rather than a task com- pletion manager. This chapter has highlighted the tenets of agility—those behaviors and attitudes that make the difference between a team that has Agile DNA and a team that does not.

In my experience working with numerous data warehousing groups in their Agile Analytics adoption, the most common problems I have observed involve project management practices. Although development practices, collaboration practices, and other practices are important, they do not seem to be as critical to Agile success as project management. Teams that are suc- cessful in adopting good APM practices have established the basis for rapid Agile maturation in other practice areas.

ptg6843605

59

Chapter 3