Interaction of distance measures

Having presented some ways to measure distance between declarations and where they are used, it is important to ask how these attributes may interact. This is done for a larger selection of programs in Section 5.1, but it is interesting to look at this for the two case study programs here. The first step towards this is to produce a correlation matrix for the metric results. These are shown in Table 15 in Appendix C.

The correlation matrices for the two case study programs each show two clus- ters of strongly correlated measures. The clusters consist of the same metrics in

each programs, with the first cluster consisting of

“Distance by the sum of the number of scopes” (d1)

“Distance by the sum of the number of declarations in scope” (d4) “Distance by the sum of the number of source lines” (d7)

“Distance by the maximum number of source lines” (d8) “Distance by the average number of source lines” (d9)

“Distance by the sum of the number of parse tree nodes” (d10) “Distance by the maximum number of parse tree nodes” (d11) “Distance by the average number of parse tree nodes” (d12) and the second cluster consisting of

“Distance by the maximum number of scopes” (d2) “Distance by the average number of scopes” (d3)

“Distance by the maximum of the number of declarations in scope” (d5) “Distance by the average number of declarations in scope” (d6)

The first cluster reaffirms that there is little difference between measuring distance by the number of source lines or by the number of parse trees nodes, and shows that measuring the sum of the number of scopes or declarations in scopes does not give much more information than measuring the number of source lines. This might be because declarations that are further away in scope tend to be further away in the source code. Likewise, as the number of declarations increases, so the distances in the source code between declarations and where they are used tend to increase.

However, it may also be that programmers might tend to write functions close to where they are used in the source code. In future work it may be interesting to see if it is possible to permutate the source code of a program to produce a program which has minimal distance measures, or indeed to discover if a program is already “minimal”.

The second cluster shows that the distance measured by the maximum or av- erage number of scopes or declarations in scopes is not strongly correlated with

measuring the sum of the number of scopes or declarations in scope. One expla- nation for this might be that the declarations, or variables, used in a function are generally similar distances from their declarations, for instance, all the uses of a pattern variable in a function might have a similar distance measure. This would cause the average and maximum values to be similar between functions, while the sum measure would vary much more between functions because it also measures how many declarations are used.

Having looked at the correlation matrix for the distance measures it is possible to replace the clusters of strongly correlated metrics with single representative metrics and perform a regression analysis on the z-scores of the metric values to see if combining the measurements can increase the correlation with the number of changes.

The metric with the highest correlation with the number of changes was chosen from each cluster as the representative of that cluster. The chosen representative metrics are “Distance by the sum of the number of source lines” (d7) and “Distance by the average number of declarations in scope” (d6) for the Peg Solitaire program and “Distance by the sum of the number of scopes” (d1) and “Distance by the maximum number of scopes” (d2) for the Refactoring program. The results of the regression analysis are shown in Table 22 in Appendix D.

The regression analysis shows a multiple correlation coefficient (R) of 0.1957 for the Peg Solitaire program and 0.6829 for the Refactoring program, both of which are statistically significant. These are slightly higher than the correlation values of any of the individual measurements which suggests that there is a little interaction between them. However the regression for the Peg Solitaire program still only accounts for a small proportion of the variance of the number of changes. The values for the Refactoring program show that distance measures can ex- plain nearly 50% of the variance of the number of changes in that program, in- dicating that it may be possible to obtain good predictions from these measures. This suggests that these metrics might be usefully combined in a visualisation tool which could highlight functions with particularly high values of these metrics.

Looking at the coefficients from the analysis of the Refactoring program it seems that “Distance by the sum of the number of scopes” (d1) and “Distance by the maximum number of scopes” (d2) make reasonably even contributions to the correlation with the number of changes. This is probably because the two measures are partially correlated and therefore combining them adds only a small amount of information.

Interestingly, the coefficients from the analysis of the Peg Solitaire program show that the “Distance by the sum of the number of source lines” (d7) metric has a negative coefficient. This is interesting because it suggests that if the functions used are a long way away in the source code they are less likely to introduce errors. This result may be caused by cross-module function calls, which imply that the calling function is using some well defined and stable interface, and hence is less likely to have to be changed as a result of the called function being changed. This may suggest that when measuring attributes across modules, the behaviour of the attributes might be different than when considering them inside a single module.

4.2.7

Summary

This section has presented a selection of metrics that measure the distance between declarations and where they are used in a variety of ways. Performing these measurements has revealed a number of discrepancies between results for the Peg Solitaire and Refactoring programs, and possible explanations for these have been presented.

Investigating the cross correlation of the measures has shown that a large proportion of the measurements are strongly correlated, suggesting they are mea- suring the same or similar attributes.

Examining the regression analysis of the measures suggests that the attribute that contributes most to the complexity is the semantic, “conceptual”, distance in the program code, rather than the “spatial” distance. This suggests that errors are more likely to be introduced into programs when there are lots of nested scopes, although since there is a strong correlation between the metrics it is not clear from

these results if this is an artifact of a different attribute, such as program size. The regression analysis also showed that in some circumstances the further away a called function is defined in the source code, the less likely an error is to occur. This implies that cross-module function calls should perhaps be treated differently to intra-module function calls.

fac :: Int -> Int fac 0 = 1

fac n = n * fac (n - 1)

Example 13: An example of trivial recursion.

4.3

Measuring Attributes of Recursive Functions

The Haskell language does not provide an explicit loop construct, instead pro- grams use recursion to achieve looping. In many cases such recursive functions can be optimised by the compiler into a loop. Because recursive functions are the only way to program loops in Haskell they are used extensively, and so it is interesting to look at what can be measured about recursive functions. This section considers only explicit recursion and does not consider other mechanisms for achieving looping, such as the use of foldr and other higher-order functions, which can be considered as “black boxes” that can be trusted. Instead this section concentrates on the visible structure of the code.

There are two ways in which a function may be recursive, which one might term “trivial” and “non-trivial” recursion. In trivial or “direct” recursion the function directly calls itself, as shown in Example 13.

Non-trivial or “indirect” recursion is potentially more complex. In a non- trivial recursive function, foo, a second function, bar, is called which in turn calls foo, producing a cyclic callgraph, or strongly connected component, although the strongly connected component may be larger in a real program. This type of recursion is demonstrated in Example 14.

Thus non-trivial recursion is much less obvious in the source code than trivial recursion, and therefore may affect the complexity of the functions involved. It is also worth noting that functions may have more than one execution path that cause recursive behaviour, and so a function may contain both trivially recursive execution paths and non-trivially recursive execution paths.

qsort :: [Int] -> [Int] qsort [] = []

qsort (a:xs) = qsortLE a xs ++ a ++ qsortG a xs qsortLE :: Int -> [Int] -> [Int]

qsortLE a xs = qsort [x | x <- xs, x <= a] qsortG :: Int -> [Int] -> [Int]

qsortG a xs = qsort [x | x <- xs, x > a]

Example 14: An example of non-trivial recursive behaviour.

which functions may be recursive. There are several attributes that one might wish to measure from recursive functions. These will be briefly introduced here and then explained and analysed in detail later in this section.

• Binary measure of recursion. It is not always obvious if a function is recursive because its cyclic callgraph might be large. In such cases it may be useful to know if a function is recursive without knowing how the function is recursive. Such an indication of recursion can be thought of as a binary recursion measure. This method is discussed in more detail in Section 4.3.1. • Number of recursive call paths. A call path is a chain of function calls. e.g. Function a calls function bar which calls function c. If the call path is cyclic it indicates that the functions involved are recursive. A function may have more than one recursive call path and so functions with greater numbers of recursive paths may be more complex. This method is discussed in more detail in Section 4.3.2.

• Number of trivial recursive call paths and number of non-trivial recursive call paths. Trivial recursive paths, those where a function directly calls itself, may be easier to understand than non-trivially recursive paths, where a function calls a second function that calls the first, for instance, and so it is interesting to count each type of recursive path separately. This method is discussed in more detail in Section 4.3.3.

• Sum or Product of the lengths of the recursive paths. As the recursive paths within a function have a length it is interesting to measure these lengths, because it may be that longer path lengths indicate increased complexity. If a function has more than one recursive path one must decide how to combine the lengths of the paths. For this work we have chosen to take the product and the sum of the lengths because these methods take account of how many recursive paths there are as well as the lengths, although one might also choose to take the maximum, or the average lengths, for instance. When investigating distance measures in Section 4.2 the use of products to combine metric values caused problems with large values being produced. However, no such problems were encountered in the use of products for the recursion metrics because the values being combined were much smaller than those of the distance measures.

This method is discussed in more detail in Section 4.3.4.

In the following sections these measures will be examined in more detail, using the case study programs described in Chapter 3. There are also other measures that can be taken from the callgraph of a program that measure attributes of recursion, such as the size of any strongly connected components. Measurements of callgraph attributes are discussed in more detail later in Section 4.4.

The rest of this section is structured in the following manner. • Section 4.3.1 looks at the binary measure of recursion.

• Section 4.3.2 investigates measuring the total number of recursive paths in a function.

• Section 4.3.3 examines ways of measuring the number of trivial and non- trivial recursive paths separately.

• Section 4.3.4 studies the lengths of the recursive paths present in a function. • Section 4.3.5 discusses the possible interactions between the recursion mea-

• Section 4.3.6 presents the conclusions that can be drawn from this study of these recursion metrics.

4.3.1

A binary indication of recursion

Sometimes it can be difficult to recognise that a function is in fact recursive because its strongly connected component may be quite large. Because of this it may be useful to have an indication that a function is recursive without actually knowing how that recursion occurs. To do this a metric was developed that returns either one or zero, indicating whether the function is recursive or not. The correlation values for this “Binary recursion” (r1) metric are shown in Table 11 in Appendix B.

The interesting observation from the results is that, for the Peg Solitaire pro- gram, this very simplistic metric appears to give the highest correlation of any of the recursion metrics. This suggests that it is more important to know that a function is recursive than to know exactly what makes the function recursive. Although it is not significant at the 5% level it is significant at the 10% level.

The results for the Refactoring program are less striking. There are only 21 out of the 540 functions that make up the Refactoring program that are recursive. Most of these recursive functions are small and have not been changed much, resulting in the low, statistically insignificant, correlation value seen.

Recursive callgraph paths can be either trivial or non-trivial. Closer inspection of the table shows that, for the Peg Solitaire program, there is a very similar correlation value for the “Number of trivial recursive paths” (r3), suggesting that the binary metric is mainly measuring the number of trivial recursive paths. This was confirmed when the raw data from the metrics was inspected, which showed that there were 28 functions with trivial recursive paths but only 7 functions with non-trivial recursive paths.

This is also shown in Section 5.2.2 of Chapter 5 which studies the values of the recursion metrics on a wider selection of programs, showing that typically the amount of non-trivial recursion is much lower than the amount of trivial recursion.

foo :: String -> String foo [] = []

foo (c:cs) = toLower c : foo cs bar :: String -> String

bar [] = [] bar (c:cs)

| isUpper c = toLower c : bar cs | otherwise = toUpper c : bar cs fib :: Int -> Int

fib 0 = 0 fib 1 = 1

fib n = fib(n-1) + fib(n-2)

Example 15: Examples of multiple recursive paths in functions.