It can be applied at any point in time in the [development] life-cycle 8 These seven characteristics of the technical debt metric enable effec-

tive governance of the [development] process.

9. The above characteristics of the technical debt metric enable effec- tive governance of the . . . product portfolio.

Highsmith says this about technical debt: “It’s expensive to fix, but much more expensive to ignore. Technical debt reduces future earnings, but even more critically, it destroys predictability which in turn impacts market capi- talization in the near term, not in the future” (Highsmith 2010b).

When a team allows technical debt to accumulate unabated, the system will eventually reach a point of stagnation. Defects are prevalent and increas- ing. It becomes too costly and risky to add new features. And fixing bugs becomes a full-time effort. When a system is referred to as a “legacy sys- tem,” you can be sure that it is mired in technical debt. Nobody wants to touch those.

Agile Analytics Practice: Prioritize Debt

Agile teams intentionally identify, track, prioritize, monitor, and pay down their technical debt. Technical debt stories should be prioritized alongside user stories to balance new feature development against debt reduction.

ptg6843605

REFACTORING 157

Effective Agile Analytics teams seek to identify, monitor, and pay down technical debt. Whenever developers make choices that incur technical debt, or when they discover preexisting technical debt, they log it into the team’s debt-tracking system. The debt-tracking system, like the defect-tracking system, need not be complicated or high-tech. A simple chart on the wall using index cards to capture “technical debt stories” is an effective way for a colocated team to manage its technical debt.

Unlike for the valuation and prioritization of user stories, it is the techni- cal team that prioritizes technical debt stories and estimates the value of eliminating each one. It is important for the product owner to ensure that the priority of new user stories does not always trump the priority of tech- nical debt stories. Technical team leaders must advocate for opportunities to schedule debt stories into development iterations. Some Agile project teams agree to allocate 15 to 20 percent of team capacity in each iteration to debt reduction, leaving 80 to 85 percent for user story development. Other Agile teams occasionally designate explicit debt reduction iterations that eschew user story development in favor of debt reduction work. Still other Agile teams find ways to eliminate technical debt in the course of user story development. Whatever the approach, it is essential to continuously moni- tor and pay down technical debt.

R

Refactoring is an important technical discipline that serves two very impor- tant purposes. First, it is a technique for safely evolving the design or models without breaking previously working features and components. Second, it is a technique for eliminating technical debt without breaking previously working features and components. Note the common elements of these purposes—without breaking previously working features and components. An ever-present consideration of iterative development is ensuring that the work we are doing in this iteration does not have an adverse impact on the work we’ve done in prior iterations. The next chapter introduces the impor- tance of test automation in DW/BI development. Together with testing, refactoring is an essential engineering practice that enables Agile teams to be effective.

The practice of refactoring was first introduced to the software community by Martin Fowler (1999). He defines refactoring as “the process of changing a software system in such a way that it does not alter the external behav- ior of the code yet improves its internal structure.” This practice has had a

ptg6843605

profound impact on the quality of software application development. It has enabled programmers to improve a design after it has been written. Within a few years after Fowler introduced refactoring, nearly every single interac- tive development environment (IDE) had support built directly into the tool to help programmers refactor their code.

Scott Ambler and Pramod Sadalage introduced refactoring to the database community in 2006 with a very thorough catalog of database refactorings and transformations (Ambler and Sadalage 2006). They describe a database refactoring as “. . . a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics (in a practical manner).” A database refactoring may affect structural elements of the schema such as tables and views, or functional elements such as stored procedures and triggers. Not only must database refactoring preserve the core database behavior and semantics; it must also preserve all external sys- tems that are coupled to the database schema such as business applications and data extract processes. The database refactoring principles (and most of the refactorings) introduced by Ambler and Sadalage are directly appli- cable to evolutionary data warehouse development. Therefore, this section of the chapter will provide an overview of the refactoring discipline rather than comprehensive coverage of the topic. Every data warehouse developer should have a copy of Refactoring Databases in his or her personal library.

Refactoring is not the same as ad hoc restructuring of code or data models. While restructuring is a good practice, it is important to understand that refactoring means something very specific. In fact, refactoring is the key to effective evolutionary database design. Refactoring relies on regression test- ing to ensure that your changes have not broken anything. Moreover, the regression test suite must be automated to enable you to repeatedly execute your tests quickly and easily in the course of refactoring. The focus of the next chapter is on test automation and test-driven data warehouse develop- ment (including regression testing), so we’ll defer discussion of those details for now.

Ambler and Sadalage distinguish the six categories of refactoring presented in Table 6.1. Within each category there are several specific refactorings and transformations.

ptg6843605

REFACTORING 159