Rapid Evolution

Reused software evolves unusually quickly because it must meet the needs of a demanding and diverse community. Software Conguration Manage- ment [FHO92], with distribution capability, is the base technology nec- essary to address the problem. The various versions of the software must be easily stored, accurately recoverable, and automatically distributable. This requires a well-dened, controllable process. In our case, this process is known asAdvsoft. This section contains an example of the use of the Advsoft process applied to a tool called Yeast.

Figure 1.1 is an illustration of the goals of a conguration management process. The general notion is that there is an existing base of software components from which the components of a specic system are chosen, assembled, tested, and distributed to a user.

This Software Base is partitioned in a variety of dimensions. For ex- ample, the actual software components may be stored in a variety of geographical locations, and components are used to create the desired featuresof the resultant application system. A specic application system

6 Belanger and Krishnamurthy

is selected from available versions (such as revisions), views, and perhaps components from other systems. Finally, selection is often based on a set of Modication Requests from a change control system. Following this selection, a system is assembled, tested, packaged, shipped to a customer site, and installed. At that point, it is ready for customer use. This ability to create a system selected from a variety of dimensions is crucial to the multiple use of software objects. We have made considerable progress in this area for example [FKSV94].

As an example of the above concept, consider the delivery of a specic version of a tool, Yeast, to a customer. It might look like this:

For Customer - North Carolina Lab

Select the August 1993 Version of Yeast

with Modification Requests for "NC" added, with Fault Tolerance & Security Features, using "libast" libraries.

The preceding syntax is ctitious (since these functions are currently not provided in a single tool), but the problem and capability are not. This de- livery specication could result in selection and packaging of all changes to Yeastnecessary to create the requested system from a computer in Mur- ray Hill, New Jersey, and shipping only the changes to \North Carolina Lab" to update their Yeast conguration. Once the changes are deliv- ered to North Carolina, the software is assembled, tested, and installed there. Note that though our customers use diverse software/hardware platforms, software requests as in the above example are free of plat- form specications. This is because we view portability as fundamental to software reuse and strive to develop technologies and disciplines that ensure portable code. We also ship source code. Changes necessitated by a port to a new platform are considered equivalent to bug xes. Chapter 2 discusses our approach to writing and maintaining portable software.

Our experience has been that software that is reused is subject to constant change. This is a corollary to the Second Law of Program Evo- lution [LB85]: \As an evolving program is continuously changed, its com- plexity, reecting deteriorating structure, increases unless work is done to maintain or reduce it." Software that is reused widely will be subject to many requests for enhancement. If, as in our case, software creation is

Software Reuse: A Decade-Long Experiment 7

done in small aggressive teams (sometimes consisting of only one indi- vidual), the rate of change of reusable software can be substantial. This process is managed using the following principles:

A single person owns responsibility for the process of distribution of

software.

A predictable process for freezing basic software components is in place

(this has proven hard in practice).

The responsibility of ensuring that software ready for distribution has

been tested is a distributed process. The responsibility is currently that of the software's creator.

Conguration, distribution, and process are replicated at user sites.

The entire process can be reused.

Conguration and distribution (and to a limited extent|testing) are

part of a single process. The goal is to combine and automate the processes of: congure, assemble, test, package, ship, and install (see Figure 1.1).

Few requirements can be enforced on the target (user) machine. The

current assumption is that it is a UNIX-like system with at least a 7th Edition UNIX Shell and a C compilation system. Even these require- ments are being relaxed.

Some of these principles have been easier to accomplish than others. There is a single responsible manager. Conguration and distribution are part of a single process, and few requirements are placed on either the target machine or on the capability of the distribution mechanism (for example, distribution is over wide areas and existing network facili- ties). Freezing the basic components in an environment in which changes are frequent has proven dicult. It could be done with a formal change management process, but we, as a research organization, have chosen to encourage change and work hard when freeze dates occur. Our choice is biased in the direction of frequent improvement, as opposed to control. Downstream distributors use formal change management. Their releases are less frequent but better controlled and tested. In Section 1.3.1, we trace the Advsoft process, as applied to a specic tool.

The technology in use for conguration and distribution is advanced and exible. It uses a variety of new tools, including n-DFS and ship/pax

8 Belanger and Krishnamurthy

(described in Chapters 2 and 3, respectively). On the other hand, the technology for testing is not as advanced and automated as that for con- guration, assembling, packaging, shipping, and installation. Work is be- ing carried out on automating and increasing the eectiveness of the testing process, but much is yet to be done. Today, those systems that are large, sophisticated, and/or critical to project operations are typically distributed to production users by downstream technology organizations (usually for a charge). In that case, our process supplies that downstream distribution channel.

1.3.1 A

Process Example

In this section, we present an example of how we coordinate, congure, and track the development of a variety of software components that are distributed to organizations throughout AT&T. We use a tool called ship, which is a collection of KornShell command scripts (see Chapter 4) that packages components of a software distribution for a variety of hardware and operating system platforms in a portable way. ship requires that tools to be distributed be stored in a directory, together with an item le that lists the set of direct dependent entities of the tool (that is, libraries needed to build the tool). ship creates a portable archive (using pax) of the source and nmake (the build tool) les to be sent to the remote site. ship, pax, and nmake are described in Chapter 3.

The example in this section deals with how changes are automati- cally tracked in libast, a library on whichYeastand other tools depend. Taking advantage of the ship directory hierarchy and the item les, we are able to automate the generation of new versions of modied software via the Advsoft process. The automation itself, incidentally, is done via Yeast(see Chapter 9). The example is explained in more detail in Section 9.5.

Figure 1.2 depicts the process that Advsoft manages. In the gure, the circles represent subprocesses and the arrows represent data ow be- tween subprocesses. Tool owners submit a copy of the newest version of their tools to Advsoft in cycles (about twice a year).

Software Reuse: A Decade-Long Experiment 9 Prepare Enhanced T (T’s owner) Approve T (advsoft) Rebuild libast (advsoft) Test (Dep Owners) (Yeast) Approve libast (advsoft) Distribute Approved System (advsoft) Initiate Cycle Announcement of_Success Announcement ofFailure Notification Mail to DependentOwners Announcement of New Cycle Announcement of Acceptance Announcement of_Rejection Announcement of New Cycle Approved T Prepare Enhanced libast owner) New libast Fixes to Tools libast Depends on Fixlibast Fixed libast Approved libast Notify Owners Dependent libast on (libast’s (libast’s owner)

Figure 1.2 A software tool development and distribution process. Portions

10 Belanger and Krishnamurthy

libast (solid lines), while showing that an identical process is carried out for all other toolsT in parallel (dotted lines). The rest of the diagram is

self-explanatory.

The conguration management tasks to be carried out by the Ad- vsoft process include coordinating changes (a new version of a tool requires notifying owners of all tools that depend on the modied tool), tracking the changes (some tool owners may reject the changes), and con- trolling the changes (waiting until all dependent tool owners accept the modied version).

The seemingly simple bookkeeping activities can be time-consuming and error-prone when performed by humans. Thus, automation of these responsibilities was undertaken. In particular, Yeast specications au- tomate those portions of Figure 1.2 that are shown in boldface.

The details of howYeastis used to implement theAdvsoft process are described in detail in Section 9.5. We simply present a brief outline here.

We rst generate specications for all tools and all owners that are under the management of Advsoft. Yeastwatches for the creation of new versions of tools, and generates notications about the successful building of these new versions. In the example, the notications are sent to tool owners dependent on libast, causing them to test their tools with the modied version of libast. Yeast also generates appropriate error messages if the new versions did not build properly.

Tool owners run regression tests on their tools and generate aYeast announcement (a user notication) indicating that they either accept or reject the modied version of libast. Yeast keeps track of all accep- tance and rejection announcements. If all dependent tool owners indicate acceptance, the modied version of libast is considered to be ready for distribution. If any of the dependent owners indicate rejection, the owner of libast is notied and the process involving libast is recommenced.

As dependencies among existing tools change and as new tools and new versions of existing tools come into existence, obsolete Yeastspec- ications are automatically deleted and new ones added.

Software Reuse: A Decade-Long Experiment 11