Visually summarising software change

(1)

Visually Summarising Software Change

Tom Arbuckle

University of Limerick

Limerick, Ireland

[email protected]

Abstract

Many authors have noted the problem of excessive in-formation when attempting to create useful visualisations of software. The problem of visualising change over multi-ple versions of software is more commulti-plex still. We present a means of visualising changes in software, founded on information-theoretic arguments, that easily and automati-cally summarises difference between software versions with respect to their code, their structure or their behaviour. Further, we show, by creating visualisations in experiments on real-world data, that the method is of utility to practi-tioners and has implications beyond the field of software visualisation.

Keywords: software evolution, visualisation of change,

Kolmogorov complexity, similarity metric, CompLearn.

1 Introduction

Software is difficult to visualise: it is both complex and detailed, has multiple levels of structure and both static and dynamic aspects need to be examined for a detailed under-standing of its operation. When we come to consider soft-ware evolution, or, alternatively, the comparison of different versions or releases of software, the problems are further compounded by the chronological nature of the evolution — or simply by the quantities of data for comparison.

Successful approaches to visualising software have ac-counted for software’s inherent properties by mirroring them. Means of presenting data only when needed and in the right quantities have been formulated and several imple-mentations of useful tools for the exploration of software structure have been documented in the literature.

The approach taken in this paper, however, is one of (the-oretically based) summarization and of the comparison of Diagrams in this paper employ colour: obtaining an electronic or colour-printed version will aid comprehension.

summaries. Our interest is in first impressions and in pro-viding big pictures of changes meaningful to developers, re-searchers and managers. How can summarised changes in structure, in behaviour or in the artefacts of the software itself be compared and presented in a graphically useful ‘quick look’ way?

This paper, which provides an answer to that question, is structured as follows. Sections providing a motivational ex-ample and an overview of alternative solutions are followed by a section in which we describe a series of experiments performed on an open-source application. For each exper-iment, we provide a summary illustration and provide an analysis of the meaning of its results. Thereafter, we sketch the information-theoretic basis for the illustrations and ex-plain the procedure by which they were obtained. After a brief section exploring the implications of these findings, we outline future work and conclude.

2 Motivation: Example Scenario

Suppose an engineer has been set the task of evaluating a series of releases of software with which he or she is com-pletely unfamiliar with a view to beginning maintenance ac-tivities. If a complete set of documentation exists, the en-gineer will be very lucky but will still have to pick through the documentation to find the changes. If, as is often the case, the documentation is patchy or non-existent, the only course of action is some form of reverse engineering activ-ity. The engineer needs to find out about both the general structure of the code and its detailed design; to discover as many flaws or defects as possible in preliminary testing or review; to locate where major changes were introduced; and to record these findings before creating suites of regression tests (if none yet exist) or introducing changes. What can the engineer do to expedite progress?

In the case of single versions of the code, reverse engi-neering and re-engiengi-neering patterns such as “Read all the code in one hour” [10] can certainly contribute. Moreover, there is a plethora of tools that allow the interactive explo-ration of source code [39, 38, 27, 21]. In the case of multiple

(2)

1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_bins_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

Figure 1. Comparison of 16 binary versions of slocate

releases of the same software, there are some research tools such as Seesoft [12], Gase [16] and EvoTrace [14] that al-low visualisations of software versions in terms of change log visualisations, comparisons of change history, semantic similarities or clustering of similar entities. However, they offer quite a lot of information in terms of disparate views of the software.

What happens if the engineer just wants a quick sum-mary of several different aspects of the software artefacts in order to allow them to know where to most productively start work? How can they obtain, programmatically, a visu-alisation of summarised software change?

Would a diagram like figure 1 help? This diagram shows (in both 3D and 2D) plots of a distance matrix in which each of sixteen releases of (in this case, the executables for) a software package are compared with each other. The pair-wise degree of similarity between versions is found quickly, automatically, and in a parameter free way. The distance matrix plots are symmetric, a central ridge, value 1.0 de-noting identicality. The similarity between the first and the last versions is shown to be least (graphed numeric result tending towards 0.0).

It is most convenient to think of the plots either in terms of columns or rows. Reading from a point on this ridge either up (a column) or (along a row) to the right corre-sponds to a comparison of that (reference) release with later releases. A flat area (in a row or a column) shows that the elements at that point are equally dissimilar to the reference,

not that they are necessarily similar to each other. Descrip-tions of how the experimental data are derived, common analysis of multiple experiments, and the theoretical back-ground are given in sections 4, 6 and 5 respectively.

3 Visualisation of Software Change

What are the problems associated with the visualisation of software change and what can be done to ameliorate them? The main problem is that there is a surfeit of infor-mation and that a means needs to be found of reducing the cognitive load to make it easier (or possible) for the view-ers to extract undview-erstanding from pictorial representations of the change data. We survey the literature.

3.1 Why and how to visualise

Sillito et al. [36] describe the tasks that a programmer must perform when making changes to software. Develop-ers need to explore the code base to build an undDevelop-erstanding of the software to be modified: gaining a sufficiently broad understanding is difficult. Apart from the cognitive over-load, there is the problem of finding focus points, or land-marks by which to navigate, during the building of the mod-ification. Effective visualisations provide one tool in allevi-ating these problems. What makes a good visualisation is discussed by Amar and Stasko [2] who introduce the notion

(3)

of analytic gaps, “obstacles faced by visualisations in facili-tating higher level tasks”. Erbacher [13] outlines an iterative process by which effective visualisations can be designed and Hungerford et al. [17] discuss how visualisations can be critically reviewed. Kienle and M¨uller [19], on the other hand, study the requirements for visualisation tools. They identify seven quality attributes that they recommend for ef-fective visualisation systems. Finally, Reiss [32] points out that, despite decades of research and the intuitive utility of visualisation to the problems at hand, very few visualisa-tions have had much impact in commonly available devel-opment environments.

3.2 Software Visualisation

In section 2, we mentioned Rigi [39], SHriMP/Creole [38, 27] and CodeCrawler [21] as tools providing visuali-sations of software systems. Many others can be found in the literature. For example, Li et al. [24] discuss how to dis-play software architectures with constant visual complexity. Ducasse et al. [11] employ polymetric views for looking at the large quantities of run-time information. Reiss et al. [33] review visualisations of run-time information.

We focus on tools and projects that are particularly con-cerned with visualising or measuring software evolution. McNair et al. [28], Ratzinger et al. [31] and Voinea and Telea [41] all discuss this topic. Sawant and Bali [35] de-scribe a tool for the visualisation of differences in software architecture whereas van Rysselberghe and Demeyer [34] are concerned with software understanding via the visual-isation of change history. Vasa et al. [40] study which el-ements of software projects change between releases with the aid of the Bhattacharyya metric. Licata et al. [25] inves-tigate how the signatures of program features change with time by comparing pairs of versons of code taken from a code versioning system. While Wu et al. [43] are inter-ested in looking at how the rate of change varies to high-light conspicuous changes, Xie and Notkin [45] investigate the change in value spectra with a view to employing this information in regression testing. Xing and Stroulia [46] employ the UMLDiff tool [47] for examining the evolution of software design. Similar work has been carried out by Wenzel et al. [42]. The paper by Apiwattanapong et al. [3] is well known for its discussion of semantic differencing and the JDiff tool. Fluri and Gall [15] investigate how a classification of the types of changes being made can help in discovering coupling between the items changed.

3.3 Visualisation: Summary

The literature surveyed can be summarised as follows. • There is a broad literature on the visualisation of

soft-ware engineering artefacts with many sophisticated

academic approaches to exploiting visualisation for a wide variety of purposes.

• Despite general agreement concerning its utility, there seems to be limited use of visualisation techniques in the professional development arena.

• Several approaches exist to pairwise compare software releases. In particular, several diff -like tools exist such as JDiff, UMLDiff, SiDiff and DiffArchVis which will provide details concerning the difference between ver-sions.

• There does not seem to be a generic context-free way to compare software artefacts quickly to get a single measure of their degree of similarity.

4 Experiments

In several experiments, we apply the same method to gauge and visualise the similarity of a variety of software artefacts. In each case, we also provide an analysis of the results. The common method and procedure by which these visualisations are obtained from the different forms of ex-perimental data is explained in section 6.

All experiments are performed on the open-source soft-ware slocate [26], which is designed to catalogue and index all files present in a specified area of a filesystem. slocate has a long history with many publicly available releases. Its code, which is short but performs fairly complex operations, has also been redesigned on at least one occasion. This makes the package an interesting candidate for our exper-iments. Source lengths in kilobytes for the slocate releases we studied are shown in table 1.

Table 1. slocate releases and sizes (kilobytes)

Release. 1.0 1.1 1.2 1.3 1.4 1.5 1.6 2.0 Size 26.6 26.7 24.0 23.9 29.7 31.3 32.2 70.3 Release 2.1 2.2 2.3 2.4 2.5 2.6 2.7 3.1 Size 72.3 72.7 70.7 73.6 82.6 91.3 92.0 67.4

4.1

slocate

: Sources, Binaries

In our initial trials of the method [4], we made a direct examination of the source code as well as looking at index-ing and search traces on a known set of data. These results are recreated and revisualised in this section for comparison with the new results.

4.1.1 Experiments

As a first step in our investigation of slocate, we see how much information we can learn from the packages directly.

(4)

1.1 1.3 1.5 2.0 2.2 2.4 2.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_sources_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(a) slocate sources

1.1 1.3 1.52.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_tars_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(b) slocate tarred sources

Figure 2. Comparison of slocate code sources and tar distributions

1.1 1.3 1.5 2.0 2.2 2.4 2.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "create_trace.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(a) slocate database creation

1.1 1.3 1.52.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "locate_trace.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(b) slocate locate in database

Figure 3. Comparison of slocate execution traces: create and locate

In figure 1, we showed a comparison of the binary executa-bles for each release. Figure 2(a) shows a comparison of the sources (concatenated but unmodified ‘.h’ and ‘.c’ files) with figure 2(b) showing an even simpler comparison of the complete tar files as provided by the source distributions.

Figure 3(a), on the other hand, provides a comparison of behaviour. It shows a comparison of the different ver-sions of slocate during the indexing of a set of known data. Likewise, figure 3(b) compares the behaviour of each of the versions when performing the same search on the database created in the indexing phase. The input data used to gener-ate these diagrams were the unmodified system call traces produced by monitoring the applications using the program strace running under Linux.

4.1.2 Analysis

In some situations, we might not have source code so the comparison shown in figure 1 could be quite useful. It clearly shows the similarity between members of the 1.x

series. It further shows a similarity between versions 2.0, 2.1 and 2.2. Versions after that show only similarity to their neighbours indicating continuous development.

Figure 2(a) is most interesting for showing the clear grouping of the sources (only) into their release classifi-cations. Figure 2(b), on the other hand, shows that in the second series of release tarballs 2.0 to 2.4 (makefile only) differed from 2.5 to 2.7 (autoconf files, doc directory).

Now examine figures 3(a) and 3(b). The first of these shows that a quite striking change in behaviour was intro-duced in version 1.6. Creation of the indexing database in-volved similar operations in versions 1.0 to 1.5 and in ver-sions 1.6 to 2.7. The second figure, however, shows extreme similarity between versions during location operations. The first and the last version are the most different and there are hints that versions 1.5 to 2.2 involved similar sequences of system calls as did versions 2.3 to 2.7.

(5)

370: instr 371: if 372: instr373: instr 374: if 375: instr 376: instr 377: loop 378: instr 379: if 381: switch 380: break 382: instr386: instr384: instr

390: instr 392: instr 394: instr 395: if 383: break 387: if385: break 391: break 393: break 388: if 389: instr 396: instr 397: if 401: if 398: instr 399: if 402: return403: return 400: instr 353: instr 354: if 355: instr356: instr 357: if 358: instr 359: instr 360: loop 361: instr 362: if 364: switch363: break 365: instr 367: instr369: instr378: instr374: instr 381: instr376: instr

382: if 368: break370: if379: return 375: break377: break

366: break 371: if 373: break372: instr 380: break 383: instr 384: if 389: if 385: instr 386: if 390: return391: return 387: instr388: instr 309: instr 310: if 311: instr312: instr 313: if 314: instr 315: instr 316: loop 317: instr 318: if 320: switch319: break 321: instr 323: instr325: instr334: instr330: instr 337: instr332: instr

322: break 327: if 329: break328: instr 336: break 339: instr 340: if 345: if 341: instr 342: if 346: return347: return 343: instr344: instr 303: instr 304: if 305: instr306: instr 307: if 308: instr 309: instr 310: if 312: instr311: return 313: loop 314: instr 315: if 317: switch316: break 318: instr 320: instr322: instr331: instr327: instr 334: instr329: instr

319: break 324: if 326: break325: instr 333: break 336: instr 337: if 342: if 338: instr 339: if 343: return344: return 340: instr341: instr 387: instr 388: if 389: instr390: instr 391: if 392: instr 393: instr 394: if 396: instr 395: return 397: loop 398: instr 399: if 401: switch 400: break 402: instr 404: instr406: instr

408: instr 410: instr 423: instr425: instr427: instr412: instr 421: instr419: instr 430: instr 431: if 403: break 405: break

409: break 411: break 424: break426: break428: return413: if 422: break420: break 407: break 414: if 418: break 415: if 416: instr 417: instr 429: break 432: instr 433: if 438: if 434: instr 435: if 439: return440: return 436: instr437: instr 436: instr 437: if 438: instr439: instr 440: instr 441: if 443: if 442: instr 444: if 446: instr 445: instr 447: if 449: instr448: return 450: loop 451: instr 452: if 454: switch 453: break

455: instr465: instr476: instr486: instr490: instr459: instr501: instr474: instr463: instr457: instr488: instr484: instr472: instr461: instr 504: instr

505: if 456: break466: if 477: loop487: break491: loop 502: return475: break464: break458: break489: break485: break473: break462: break 460: break 467: if 471: break468: if 469: instr 470: instr 478: instr 479: if 481: instr 480: break 482: return 483: break 492: instr 493: if 495: instr 494: break 496: if 499: instr 500: break 498: instr 497: instr 503: break 506: instr 507: if 521: return 508: instr 509: if 510: instr 520: instr 511: loop 512: if 514: loop513: break 515: instr 516: if 518: instr517: break 519: instr 437: instr 438: if 439: instr440: instr 441: instr 442: if 444: instr 443: instr 445: if 446: instr447: if 448: if 450: instr 449: instr 451: if 453: instr452: return 454: loop 455: instr 456: if 458: switch 457: break

459: instr469: instr480: instr490: instr494: instr463: instr505: instr478: instr467: instr461: instr492: instr488: instr476: instr465: instr 508: instr

509: if 460: break470: if 481: loop491: break495: loop 506: return479: break468: break462: break493: break489: break477: break466: break 464: break 471: if 475: break472: if 473: instr 474: instr 482: instr 483: if 485: instr 484: break 486: return 487: break 496: instr 497: if 499: instr 498: break 500: if 503: instr 504: break 502: instr 501: instr 507: break 510: instr 511: if 525: return 512: instr 513: if 514: instr 524: instr 515: loop 516: if 518: loop517: break 519: instr 520: if 522: instr521: break 523: instr 445: instr 446: if 447: instr448: instr 449: instr 450: if 452: instr 451: instr 453: if 454: instr455: if 456: if 458: instr 457: instr 459: if 461: instr 460: return 462: loop 463: instr 464: if 466: switch 465: break 467: instr 469: instr 471: instr 473: instr 475: instr477: instr 479: instr 500: instr506: instr517: instr481: instr502: instr 492: instr504: instr488: instr 490: instr 520: instr

521: if 468: break 470: break

474: break476: break 478: break 480: break 501: break507: loop518: return 482: if503: break493: loop505: break489: break491: break 472: break 483: if 487: break 484: if 485: instr 486: instr 494: instr 495: if 497: instr496: break 498: return 499: break 508: instr 509: if 511: instr510: break 512: if515: instr 516: break 514: instr513: instr 519: break 522: instr 523: if 537: return 524: instr 525: if 526: instr 536: instr 527: loop 528: if 530: loop 529: break 531: instr 532: if 534: instr 533: break 535: instr 472: instr 473: if 474: instr475: instr 476: instr 477: if 479: instr 478: instr 480: if 481: instr482: if 483: if 485: instr 484: instr 486: if 488: instr 487: return 489: loop 490: instr 491: if 493: switch 492: break 494: instr

496: instr500: instr515: instr506: instr 531: instr519: instr533: instr508: instr517: instr498: instr546: instr529: instr527: instr502: instr504: instr544: instr 549: instr 550: if 495: break

497: break501: break516: break507: break 532: break520: loop534: loop 509: if547: return518: break530: break528: break503: break505: break545: break 499: break 510: if 514: break 511: if 512: instr 513: instr 521: instr 522: if 524: instr 523: break 525: return 526: break 535: instr 536: if 538: instr537: break 539: if542: instr 543: break 541: instr540: instr 548: break 551: instr 552: if 566: return 553: instr 554: if 555: instr 565: instr 556: loop 557: if 559: loop 558: break 560: instr 561: if 563: instr 562: break 564: instr 494: instr 495: if 496: instr497: instr 498: instr 499: if 501: instr 500: instr 502: if 503: instr504: if 505: if 507: instr 506: instr 508: if 510: instr 509: return 511: loop 512: instr 513: if 515: switch 514: break 516: instr

517: break521: break536: break527: break 552: break540: loop554: loop 529: if567: return538: break550: break548: break523: break525: break565: break 519: break 530: if 534: break 531: if 532: instr 533: instr 541: instr 542: if 544: instr 543: break 545: return 546: break 555: instr 556: if 558: instr557: break 559: if562: instr 563: break 561: instr560: instr 568: break 571: instr 572: if 586: return 573: instr 574: if 575: instr 585: instr 576: loop 577: if 579: loop 578: break 580: instr 581: if 583: instr 582: break 584: instr 575: instr 576: if 577: instr578: instr 579: instr 580: if 582: instr 581: instr 583: if 584: instr 585: if 586: if 588: instr 587: instr 589: loop 590: instr 591: if 593: switch 592: break 594: instr 596: instr 598: instr 600: instr602: instr604: instr 606: instr 633: instr635: instr637: instr650: instr 608: instr620: instr615: instr613: instr653: if648: instr631: instr

654: instr 655: instr 595: break 597: break

601: break 603: break605: break 607: break 634: break636: break638: loop 651: return609: if 621: if616: if614: break 649: break632: break 599: break 610: if 612: break 611: instr 617: instr 618: instr 619: break 622: instr 623: instr 624: loop 625: instr 626: if 628: instr627: break 629: return 630: break 639: instr 640: if 642: instr641: break 643: if646: instr 647: break 645: instr644: instr 652: break 656: if 657: instr 658: if 672: return 659: instr 660: if 661: instr 671: instr 662: loop 663: if 665: loop 664: break 666: instr 667: if 669: instr 668: break 670: instr 596: instr 597: if 598: instr599: instr 600: instr 601: if 603: instr 602: instr 604: if 605: instr 606: if 607: if 609: instr 608: instr 610: loop 611: instr 612: if 614: switch 613: break 615: instr 619: instr 617: instr621: instr 623: instr629: instr 625: instr 627: instr646: if668: instr649: if666: instr 655: instr636: instr 641: instr653: instr 634: instr 671: if

672: instr 673: instr 616: break 618: break622: break 624: break630: if 626: break 628: break647: instr 637: if 642: if 635: break

648: break 650: instr 651: instr 654: break 656: loop 667: break 669: return 620: break 631: if 633: break 632: instr 638: instr 639: instr 640: break 643: instr 644: instr 645: break 652: break 657: instr 658: if 660: instr 659: break 661: if 664: instr 665: break 662: instr663: instr

670: break 674: if 675: instr 676: if 702: return 677: instr 678: if 679: goto680: if 681: instr 701: instr 682: if 683: loop 690: loop 684: if 691: if 685: instr 688: break686: if 689: instr 687: break 693: loop 692: break 694: if 695: instr 698: break696: if 700: instr 699: instr 697: break 594: instr 595: if 596: instr597: instr 598: instr 599: if 601: instr 600: instr 602: if 603: instr 604: if 605: if 607: instr 606: instr 608: loop 609: instr 610: if 612: switch 611: break 613: instr 617: instr 615: instr619: instr 621: instr

623: instr 625: instr

627: instr 644: if 632: instr664: instr653: instr634: instr 666: instr647: if639: instr669: if 651: instr 670: instr 671: instr 614: break 616: break620: break 622: break 624: break 626: break 628: if 645: instr 633: break 635: if 640: if 646: break 648: instr 649: instr 652: break 654: loop 665: break 667: return 618: break 629: if 631: break 630: instr 636: instr 637: instr 638: break 641: instr 642: instr 643: break 650: break 655: instr 656: if 658: instr657: break 659: if 662: instr 663: break 661: instr660: instr 668: break 672: if 673: instr 674: if 699: return 675: instr 676: if 677: instr 685: if 678: loop 686: instr698: instr 679: if 680: instr683: break 681: if 684: instr682: break 687: loop 688: if 690: loop 689: break 691: if 692: instr 695: break 693: if 697: instr 696: instr694: break 1093: instr 1094: if 1096: instr 1095: goto 1097: if 1144: instr 1145: return 1099: if 1098: goto 1100: instr1102: if 1101: goto 1103: instr1118: if 1104: if 1119: if 1141: instr 1105: instr 1108: if 1106: if 1109: instr 1114: instr 1107: goto1110: if 1115: if 1111: instr 1112: if 1113: goto1116: instr 1117: goto 1120: instr 1121: if 1142: goto 1123: instr 1122: instr 1124: loop 1125: if 1127: if 1126: break 1128: instr 1129: instr 1143: instr 1138: if 1130: loop 1139: goto 1140: instr 1131: if 1132: if1136: break 1133: if1135: break 1137: instr1134: break

Figure 4. CIL: control flow for the sequence of slocate main routines (shown in reading order)

4.2

slocate

: CIL Analysis

In this subsection, we compare the logical flow of the program across versions by determining the control flow graph (CFG) for each of the releases. The CFG is a rep-resentation of decision points within the routines.

4.2.1 CIL

CIL [30], C Intermediate Language, is a high level repre-sentation of C code and a powerful set of associated tools intended for code analysis and transformation. For our lim-ited purposes, it can, however, be thought of as a library of Objective CAML routines that can be directed to perform analysis of C code.

Using the CIL system [29], a short OCAML program was written (based on sample code available from [18]) that would accept all of the preprocessed .i files corresponding to program compilation, would merge and combine them and then use its analysis to create a set of dot files describ-ing the control flow within the program. Each decision point in the routine becomes a node in the diagram. We

experi-ment directly on the files describing control flow for each version’s main routine. The flows for each version of slo-cate are sketched in fig. 4. Fig. 5 shows in detail the flow for slocate version 1.0.

We perform two experiments. In figure 6(a), we compare the control flow for the top level main routines for each re-lease by comparing their dot file representations. In figure 6(b), we compare the control flows for each release by look-ing at the concatenations of the dot files representlook-ing all of the routines in each release.

4.2.3 Analysis

First look at diagram 6(a). The lack of plateaux spread-ing out towards corners of the main diagonal is indicative of steady change. The slocate main routines operate differ-ently in each version. Nevertheless, we can see four blocks along the main diagonal: 1.0 to 1.4, 1.4 to 2.1, 2.1 to 2.5 and 2.5 to 2.7. The extreme difference between the versions

(6)

370: instr 371: if 372: instr 373: instr 374: if 375: instr 376: instr 377: loop 378: instr 379: if 381: switch 380: break

382: instr 386: instr 384: instr

390: instr 392: instr 394: instr 395: if 383: break 387: if 385: break 391: break 393: break 388: if 389: instr 396: instr 397: if 401: if 398: instr 399: if 402: return 403: return 400: instr

Figure 5. CIL: control flow for slocate main rou-tine, version 1.0

2.5 to 3.1 and the others is indicated by values close to zero. These final versions are strongly different from their prede-cessors. Also striking are the very low values for the simi-larity. Only for routines in the block 1.0 to 1.4 is their any sign of close similarity. The main routines being analysed are short but steadily increase in complexity as additional functionality is added. This representation, therefore, seems to be less informative than the previous ones (although we will need to carry out additional research to find out why).

Now look at diagram 6(b) that shows the similarity for the concatenation of all routines’ representations. We see that, apart from some slight degree of similarity for the ini-tial few versions, perhaps as far as version 1.5, there seems to be very little similarity detected by the method. The most striking feature is the trough to be seen between versions 2.0 and 2.3. It seems that these versions were quite different from all of the others.

4.3

slocate

: Call Graph Analysis

In this subsection, we consider the program structure in terms of the linkage between its routines. This time we compare call graphs.

4.3.1 ncc

ncc [44] can be thought of as a front-end for the gcc com-piler. It extracts a call graph from the source code before passing it on to gcc for compilation. Again ncc will pro-duce dot files corresponding to the code presented to it for analysis. Each of these represents the call graph for the pro-gram under analysis: each routine becomes a node in the graph. It does not, however, show control flow.

We applied it to the sources for each version of slocate to produce call graphs for the main routine. Each node is the name of a routine: main, rindex, load dir, and so on. For space reaons, we show only one example as figure 7.

Again we perform two experiments. In figure 8(a), we look at the call graphs for the top level main routines. In fig-ure 8(b), although we consider the main routines, this time we omit calls to the system memory management routines malloc, free and realloc.

4.3.3 Analysis

The first item of note about figures 8(a) and 8(b) is the strong similarity between the two. Omitting or including the calls to the sytem memory management functions makes no noticeable difference to the results. The comparison method does not see these as being noteworthy functions.

Looking at the figures in more detail, we see three clear divisions represented as blocks along the diagonal corre-sponding to the major release numbers. It is reassuring to see that the last two members of the 1.x series had some de-gree of similarity to the 2.x series leading to a ‘halo’ effect around the central block. The sole member of the 3.x series is seen to be fundamentally different in structure to the 1.x series and even the 2.x series as we would expect. The 2.x series, however seems to be split into two sub-blocks: 2.0 to 2.3 and 2.4 (perhaps 2.5) to 2.7. Again, this is reflected in the code. 2.0 to 2.3 were indeed structured differently. A patch was required to get these versions to compile.

5 Information Theoretic Approach

All of the experimental results were obtained using an implementation of a similarity metric based on the concept of Kolmogorov complexity. Here we sketch the theory be-hind these, the implementation, and how they were applied

(7)

1.11.31.52.02.22.42.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_dots_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(a) slocate: CFG for main only

1.11.31.52.02.22.42.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_all_dots_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(b) slocate CFG, all procedures

Figure 6. Comparison of slocate control flow graphs produced from CIL data

main

rindex

usage

printf

exit

getuid getgid getopt

parse_exclude malloc strlen fprintf memmove get_gid create_db decode_db __rawmemchr

__builtin_strchr strcat __builtin_strncat __builtin_strcmp realloc fopen _IO_getc

free atoi fclose strtol __strtol_internal _IO_putc lstat strerror __errno_location init_2D_list statmalloc open MYfchdir close match_exclude

load_dir frcode get_cur_dir

chmod chown MYchdir chdir free_right rename __lxstat fchdir strstr

freemalloc add_down add_right opendir readdir seekdir telldir closedir put_short fputs Uput_short calloc Uget_short get_short snprintf

fgetc

Figure 7. ncc: call graph for slocate main routine, version 1.0

1.11.31.5 2.02.22.42.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_dots_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(a) slocate: call graph

1.11.31.5 2.02.22.42.63.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 "plot_no_mfr_blocksort.data" matrix 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1 1.1 1.3 1.5 2.0 2.2 2.4 2.6 3.1

(b) slocate call graph, no malloc/free/realloc

(8)

to produce the final results. In addition to an extensive bibli-ography on these topics, pointers to references on Shannon entropy and Kolmogorov complexity in the context of soft-ware engineering are presented in our earlier paper [4].

5.1 Kolmogorov Complexity

The Kolmogorov complexity, discovered independently by Solomonoff [37], Kolmogorov [20] and Chaitin [5], can be defined as follows [22].

Formally, the Kolmogorov complexity, or al-gorithmic entropy,K(x) of a string x is the length of the shortest binary programx∗ to computex on an appropriate universal computer — such as a universal Turing machine.

Kolmogorov complexity measures messages’ individual information content, rather than, as in the Shannon case, information transfer based on the probabilistic selection of messages from a set. For more details see [23, 8].

5.2 Similarity Metric

A means for comparing how similar two objects are in terms of their shared information content, the Similarity Metric, was published by Li et al. [22]. Their normalised in-formation distance (NID) is a relative metric in that it takes the quantities of information in the objects to be compared into account. It is given by the following equation.

NID(x, y) = max{K(x|y), K(y|x)}_{max{K(x), K(y)}} (1) HereK(x|y), for example, is the conditional Kolmogorov complexity ofx given y. This is the length of the shortest program for a universal Turing machine to outputx when given an input y. The NID has been shown to obey the standard metric properties up to an additive term. It takes values from[0, 1] with 0 indicating identical objects.

5.3 Implementation

The practical difficulty with using Kolmogorov com-plexity is that it is non-computable. Kolmogorov complex-ity is not partial recursive and so it is impossible to find its exact value. This is a consequence of the nonexistence of an algorithm solving the halting problem which is itself closely connected with G¨odel’s incompleteness theorem. Kolmogorov complexity can, however, be approximated.

Cilibrasi and Vit´anyi [7] created the normalised com-pression distance (NCD).

NCD(x, y) = C(xy) − min{C(x), C(y)}_{max{C(x), C(y)}} (2)

wherexy denotes the concatenation of x and y. C(x), for example, denotes the approximation of a Kolmogorov com-plexityK(x) by the length of the compressed data produced by an instance of a real compressor. Provided the compres-sor possesses certain properties (idempotency, monotonic-ity, symmetry and distributivity), they show that the NCD provides an approximation of the NID [22].

The publicly available toolkit CompLearn [6] contains an implementation of the NCD. We employ the blocksort compressor of version 1.0.2. Given two strings to compare, it produces a number constrained to lie between zero and (approximately) one: zero means that the comparands are identical. Performing pairwise comparisons on a set of ob-jects, the program ncd produces a symmetric distance ma-trix which is used for our visualisation.

5.4 Interpretation

The Similarity Metric is a universal comparator. If two objects are close using any reasonable distance measure, then they will also be close using the Similarity Metric. Fur-thermore, it can compare objects not normally considered comparable – a mitochondrial genome and a Unix binary, for example.

The Similarity Metric is also, in some sense, a feature de-tector. It finds the characteristic most similar between com-pared entities as the basis for its similarity measurement. This may not be what is intuitively the most comparable feature and what is chosen may not be the same even when one member of a pair is substituted by a third item.

The NCD has already been applied successfully in a di-verse range of fields from genomics to musical style classifi-cation. Concerning Kolmogorov complexity-based metrics, in a section of the recent paper by Allen et al. [1] discussing ‘size’ metrics, the authors show that they are strongly cor-related with counting metrics.

6 Common experimental analysis

In our experiments, all of the comparisons for each of the different types of input data were performed in the same way. Data samples representing the software releases were compared using the NCD to produce a distance matrix. This matrix denotes pairwise comparison between each of the samples and is symmetrical (about its minor diagonal). The values plotted, both in 2D and 3D, are 1.0-NCD to permit easier viewing of the results. Therefore, a value of 1.0 resents identical samples whereas a value closer to zero rep-resents samples differing greatly from each other.

Note that we performed no filtering on the input data for comparison of sources or behaviours. We made no attempt to find a common alphabet of symbols or make edits of any

(9)

kind1_{. Similarly, in the comparison of logical structure, we}

relied directly on the dot files as being representative of de-cisions made in the software. Finally, the dot files concern-ing program structure in terms of linkages (routine calls) be-tween the different elements of the program were not mod-ified in any way to produce the graphs shown.

We believe that the techniques described in this paper have additional applications in software engineering. Some examples include software clustering, clone detection, im-plications for testing resulting from added software com-plexity, project management and feature detection. Con-cerning visualisation, one possible avenue of investigation might mirror D’Ambros and Lanza’s work showing ways to visualise changes in information content as a means of predicting where bugs may have been injected [9].

7 Conclusions and Future Work

In visualisation, the problem of information overload is well-known and several means of ameliorating it — mul-tiple views, hierarchical views, abstraction, interactive vi-sualisation — have been proposed. In this paper we have shown that, by applying an information theoretic approach to analyze software artefacts, we can create a visualisation that, firstly, finds and highlights the most significant dif-ferences between software versions; secondly, provides a numerical measure of similarity between compared arte-facts; and, thirdly, in conjunction with the visualisations presented here, allows researchers, developers and man-agers to quickly grasp areas of significant change without need for extensive and difficult analysis.

Clearly there are implications for several software engi-neering fields in view of the results of the information the-oretic model sketched in this paper. Within the field of vi-sualisation, we will proceed to make more detailed studies of how much can easily be revealed using these techniques throughout the software development process with the aim of providing as much information for practitioners as possi-ble in their never-ending challenge to understand unfamiliar code.

Acknowledgements

The author would like to acknowledge the helpful com-ments made by the reviewers. The author thanks Adam Balaban, Mark Lawford and Dennis K. Peters for encour-agement, additional comments and insight.

1_{To compile releases 2.0, 2.1, 2.2, 2.3, on line 819 of file sl fts.c ‘#ifdef}

DT WHT’ became ‘#if defined DT WHT && defined S IFWHT’.

References

[1] E. B. Allen, S. Gottipati, and R. Govindarajan. Measuring size, complexity, and coupling of hypergraph abstractions of software: An information-theory approach. Software

Qual-ity Control, 15(2):179–212, 2007.

[2] R. A. Amar and J. T. Stasko. Knowledge precepts for design and evaluation of information visualizations. IEEE

Transac-tions on Visualization and Computer Graphics, 11(4):432–

442, 2005.

[3] T. Apiwattanapong, A. Orso, and M. J. Harrold. A differ-encing algorithm for object-oriented programs. In ASE ’04:

Proceedings of the 19th IEEE international conference on Automated software engineering, pages 2–13. IEEE

Com-puter Society, 2004.

[4] T. Arbuckle, A. Balaban, D. K. Peters, and M. Lawford. Software documents: Comparison and measurement. In

Proc. 18th Int. Conf. on Software Eng. & Knowledge Eng. (SEKE 2007), pages 740–745, July 2007.

[5] G. J. Chaitin. On the length of programs for computing fi-nite binary sequences: statistical considerations. J. ACM, 16(1):145–159, 1969.

[6] R. Cilibrasi. The CompLearn Toolkit, 2003.

[7] R. Cilibrasi and P. Vit´anyi. Clustering by compression. IEEE

Trans. Information Theory, 51(4):1523–1545, April 2005.

[8] T. M. Cover and J. A. Thomas. Elements of Information

Theory. Wiley-Interscience, 2006.

[9] M. D’Ambros and M. Lanza. Software bugs and evolution: A visual approach to uncover their relationship. In CSMR

’06: Proc. Conf. Software Maintenance and Reengineering,

pages 229–238. IEEE Computer Society, 2006.

[10] S. Demeyer, S. Ducasse, and O. Nierstrasz. Object Oriented

Reengineering Patterns. Morgan Kaufmann Publishers Inc.,

2002.

[11] S. Ducasse, M. Lanza, and R. Bertuli. High-level polymet-ric views of condensed run-time information. In CSMR ’04:

Proceedings of the Eighth Euromicro Working Conference on Software Maintenance and Reengineering (CSMR’04),

[12] S. G. Eick, J. L. Steffen, and E. E. Sumner, Jr. Seesoft-a tool for visualizing line oriented software statistics. IEEE Trans.

Softw. Eng., 18(11):957–968, 1992.

[13] R. F. Erbacher. Exemplifying the inter-disciplinary nature of visualization research. In IV ’07: Proc. 11th Int. Conf.

Information Visualization, pages 623–630. IEEE Computer

Society, 2007.

[14] M. Fischer, J. Oberleitner, H. Gall, and T. Gschwind. Sys-tem evolution tracking through execution trace analysis. In

IWPC ’05: Proc. 13th Int. Workshop on Program Compre-hension, pages 237–246. IEEE Computer Society, 2005.

[15] B. Fluri and H. C. Gall. Classifying change types for qualify-ing change couplqualify-ings. In Proc. 14th IEEE Int. Conf. on

Pro-gram Comprehension (ICPC’06), pages 35–45. IEEE

Com-puter Society, 2006.

[16] R. Holt and J. Y. Pak. GASE: visualizing software evolution-in-the-large. In WCRE ’96: Proc. 3rd Working Conf. on

(10)

[17] B. C. Hungerford, A. R. Hevner, and R. W. Collins. Re-viewing software diagrams: A cognitive study. IEEE Trans.

Softw. Eng., 30(2):82–96, 2004.

[18] R. Jones. Using CIL to analyze libvirt, 2007. Canonical URL is http://et.redhat.com/∼rjones/cil-analysis-of-libvirt/. [19] H. M. Kienle and H. A. M¨uller. Requirements of software visualization tools: A literature survey. In Proc. 4th IEEE

International Workshop on Visualizing Software for Under-standing and Analysis, pages 2–9, June 2007.

[20] A. N. Kolmogorov. Three approaches to the quantitative definition of information. Probl. Inform. Trans., 1(1):1–7, 1965.

[21] M. Lanza, S. Ducasse, H. Gall, and M. Pinzger. Code-Crawler: an information visualization tool for program com-prehension. In ICSE ’05: Proc. 27th Int. Conf. on Software

Eng., pages 672–673, 2005.

[22] M. Li, X. Chen, X. Li, B. Ma, and P. Vit´anyi. The similar-ity metric. IEEE Trans. Information Theory, 50(12):3250– 3264, 2004.

[23] M. Li and P. Vit´anyi. An introduction to Kolmogorov

com-plexity and its applications (2nd ed.). Springer-Verlag, 1997.

[24] W. Li, P. Eades, and S.-H. Hong. Navigating software ar-chitectures with constant visual complexity. In VLHCC

’05: Proceedings of the 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC’05),

[25] D. R. Licata, C. D. Harris, and S. Krishnamurthi. The feature signatures of evolving programs. In Proc. 18th IEEE Int.

Conf. on Automated Software Engineering, pages 281–285,

October 2003.

[26] K. Lindsay. slocate, 1998. Canonical URL is http://slocate.trakker.ca/.

[27] R. Lintern, J. Michaud, M.-A. Storey, and X. Wu. Plugging-in visualization: experiences Plugging-integratPlugging-ing a visualization tool with Eclipse. In SoftVis ’03: Proceedings of the 2003 ACM

symposium on Software visualization, pages 47–56, 209.

ACM, 2003.

[28] A. McNair, D. M. German, and J. Weber-Jahnke. Visual-izing software architecture evolution using change-sets. In

WCRE ’07: Proceedings of the 14th Working Conference on Reverse Engineering (WCRE 2007), pages 130–139. IEEE

Computer Society, 2007.

[29] G. C. Necula. CIL home page, 2005. Canonical URL is http://hal.cs.berkeley.edu/cil/.

[30] G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer. CIL: Intermediate language and tools for analysis and transfor-mation of C programs. In CC ’02: Proc. 11th Int. Conf. on

Compiler Construction, pages 213–228, London, UK, 2002.

Springer-Verlag.

[31] J. Ratzinger, M. Fischer, and H. Gall. EvoLens: Lens-view visualizations of evolution data. In IWPSE ’05:

Proceed-ings of the Eighth International Workshop on Principles of Software Evolution, pages 103–112. IEEE Computer

Soci-ety, 2005.

[32] S. P. Reiss. The paradox of software visualization. In

VIS-SOFT ’05: Proc. 3rd IEEE Int. Workshop on Visualizing Software for Understanding and Analysis, pages 1–5. IEEE

Computer Society, 2005.

[33] S. P. Reiss. Visual representations of executing programs. J.

Vis. Lang. Comput., 18(2):126–148, 2007.

[34] F. V. Rysselberghe and S. Demeyer. Studying software evo-lution information by visualizing the change history. In

ICSM ’04: Proc. 20th IEEE Int. Conf. on Software Main-tenance, pages 328–337. IEEE Computer Society, 2004.

[35] A. P. Sawant and N. Bali. DiffArchViz: A tool to visualize correspondence between multiple representations of a soft-ware architecture. In 4th IEEE Int. Workshop on

Visualiz-ing Software for UnderstandVisualiz-ing and Analysis, 2007., pages

121–128, June 2007.

[36] J. Sillito, K. D. Volder, B. Fisher, and G. Murphy. Managing software change tasks: an exploratory study. In Int. Symp. on

Empirical Software Engineering, pages 23–32, November

2005.

[37] R. J. Solomonoff. A formal theory of inductive inference. part I and part II. Information and Control, 7(1 and 2):1–22 and 224–254, 1964.

[38] M.-A. D. Storey and H. A. M¨uller. Manipulating and documenting software structures using SHriMP views. In

ICSM ’95: Proceedings of the International Conference on Software Maintenance, page 275. IEEE Computer Society,

1995.

[39] M.-A. D. Storey, K. Wong, and H. A. M¨uller. Rigi: a vi-sualization environment for reverse engineering. In ICSE

’97: Proc. 19th Int. Conf. on Software engineering, pages

606–607. ACM Press, 1997.

[40] R. Vasa, J.-G. Schneider, and O. Nierstrasz. The inevitable stability of software change. In IEEE Int. Conf. Software

Maintenance, pages 4–13, Oct. 2007.

[41] L. Voinea and A. Telea. Multiscale and multivariate visu-alizations of software evolution. In SoftVis ’06:

Proceed-ings of the 2006 ACM symposium on Software visualization,

pages 115–124. ACM, 2006.

[42] S. Wenzel, H. Hutter, and U. Kelter. Tracing model ele-ments. In IEEE Int. Conf. on Software Maintenance, 2007., pages 104–113, October 2007.

[43] J. Wu, C. W. Spitzer, A. E. Hassan, and R. C. Holt. Evolu-tion spectrographs: Visualizing punctuated change in soft-ware evolution. In IWPSE ’04: Proc. 7th Int. Workshop on

Principles of Software Evolution, (IWPSE’04), pages 57–66.

IEEE Computer Society, 2004.

[44] S. Xanthakis. ncc home page, 2002. Canonical URL is http://students.ceid.upatras.gr/∼sxanth/ncc/.

[45] T. Xie and D. Notkin. Checking inside the black box: Re-gression testing by comparing value spectra. IEEE Trans.

Softw. Eng., 31(10):869–883, 2005.

[46] Z. Xing and E. Stroulia. Analyzing the evolutionary his-tory of the logical design of object-oriented software. IEEE

Trans. Softw. Eng., 31(10):850–868, 2005.

[47] Z. Xing and E. Stroulia. UMLDiff: an algorithm for object-oriented design differencing. In ASE ’05: Proc. 20th

IEEE/ACM Int. Conf. on Automated Software Engineering,