MCD Complexity (Question (ii))

If a defect requires one or more accompanying changes (as per definition in Sec- tion 4.2), we can assume that this defect is relatively more complex to fix, in some respect, than a defect that does not require any accompanying change. In the study, we find the following profile for defects that require one or more accompa- nying changes over six releases: 10% defects of type A(all), 8% defects of type D (development), 9% defects of type T (testing), and 9% defects of type F (field)6_. Due to the “multiple-component” nature, MCDs are generally considered more complex to fix than other types of defects. The complexity of a MCD can be thus 6_{Here, we see that the percentage 10% for typeAdefects, which is larger than that for types} D,T, andF defects. The reason follows. TypeAis a set of typesD,T andF; and there exist defects requiring accompanying changes across phases (development, testing and field). These defects are considered only when the percentage for type A– 10% – is counted.

measured by the number of accompanying changes required to fix this MCD. Table 4.3 shows the proportions of MCDs (see column “%MCDs”) and their accompanying changes (see column “%Accompanying Changes for MCDs”) in the six releases. For example, MCDs account for 7% of all defects but these defects consume 50% of all accompanying changes in release 1 (see row “r1”).

Table 4.3: Proportions of MCDs and their accompanying changes. Release %MCDs %Accompanying Changes for MCDs

r1 7% 50% r2 11% 60% r3 5% 45% r4 8% 57% r5 5% 46% r6 10% 55% Mean 8% 52% StDev 3% 6%

Table 4.3 shows that on average over the six releases, MCDs account for ap- proximately 8% of the defects (with standard deviation 3%), but consume ap- proximately 52% of the all accompanying changes (thus 52% of the all children defects) (with standard deviation 6%). This means that, on average, a MCD consumes many more accompanying changes than a non-MCD, indicating the increased complexity in attempting to fix MCDs in the subject system.

Table 4.3 also describes a near-linear, positive, correlation between the propor- tion of MCDs and that of accompanying changes for MCDs; the corresponding Pearson-value is 0.93 (over the six releases)7_{. This indicates that the greater} the MCD proportion of the system, the greater the proportion of accompanying changes required for fixing the MCDs in the system.

Considering Tables 4.2 and 4.3 together, we find that there is “no” correlation between the system size (see column “#Components” in Table 4.2) and the MCD 7_{The critical value for 6 data pairs (because there are six releases) based Pearson value}

is 0.622 (see http://www.une.edu.au/WebStat/unit_materials/c6_common_statistical_ tests/test_signif_pearson.html (last access in November 2010)). Here, obviously, the 0.93 Pearson-value indicates a significant correlation.

proportion (see column “%MCDs” in Table 4.3); the corresponding Pearson-value is -0.158_{. This indicates that the increase in system size does not substantially} affect the MCD proportion of the system.

In particular, Table 4.4 depicts the average numbers of accompanying changes for MCDs and all defects in development phases, testing phases and all phases. Each cell in the “MCDs” and “All Defects” columns is a Mean/Standard devia- tion (StDev) pair. For example, value “2.10/0.10” in column “MCDs” means that the average number of accompanying changes for a MCD is 2.10 with standard deviation 0.10. Column “MCDs:Non-MCDs” shows the average ratio of the num- ber of accompanying changes for MCDs to that for non-MCDs. The last column “p-value” shows the significance level – the p value9 _{– for the difference between} the values for MCDs and “All Defects”. Thepvalue “<0.001” (for the testing of defects of types: All, Development and Testing) means that there are statistically significant differences between all defects and MCDs with respect to the quantity of accompanying changes required.

Table 4.4: Accompanying changes required for MCDs.

Defect Type MCDs All Defects MCDs:Non-MCDs p-value All 2.10/0.10 0.10/0.03 26.3 <0.001 Development 2.10/0.17 0.08/0.02 30 <0.001 Testing 2.15/0.07 0.09/0.03 27 <0.001 Table 4.4 shows that the average number of accompanying changes for MCDs is about 2.1 (see column “MCDs”), which is 26-30 times (see column “MCDs:Non- MCDs”) as much as that for non-MCDs. This also means that a MCD has an average of 2.1 children defects, but a general defect has an average of only 0.1 8_{Because the critical value for 6 data pairs based Pearson value is 0.622 (see the previous}

footnote), here, the -0.15 Pearson-value thus indicates an insignificant correlation.

9_{In order to determine if there is any significant difference between all defects and MCDs}

with respect to the accompanying changes, we conducted a Wilcoxon paired signed rank test. Each test here is based on 6 data pairs (because that there are six releases). Thepvalue is “the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.” (seehttp://en.wikipedia.org/wiki/P-value(last access in November 2010))

children defect. This further indicates that, on average, a MCD requires to be decomposed into nearly 3 times as much as the simpler defects for piecemeal correction than a non-MCD (i.e., fixing a MCD requires nearly 3 times changes (based on components) as much as that for fixing a non-MCD.). This confirms the early finding (of Table 4.3) that MCDs are more complex to fix than non-MCDs, which thus answered question (ii) posed in Section 4.1 – To what extent are MCDs more complex than other types of defects?

We note that 20% of defects consume 60%-80% of total correction effort (Ebert et al., 2005, ch. 9). Usually, these 20% of defects are more complex to fix than the remaining defects. We also note that, in (Eick et al., 2001), the authors use change factors (includingsize (number of added/deleted SLOC),span (number of components spanned), time interval (calendar time required), etc.) to measure a change’s effort. Fixing a defect usually requires changes in a code base; therefore, these change factors can be used to measure the defect – its “complexity”. As shown above, we measured defect (MCD) complexity with the number of accom- panying changes. This can complement the previous work (e.g., (Eick et al., 2001) and (Ebert et al., 2005, ch. 9)).

Furthermore, considering the relationship between MCDs and architectural degeneration (see Figure 3.1), we can infer that the architectural degeneration increased the (fix) complexity of defects in the system. This is mainly due to the profile of over 50% of accompanying changes for MCDs (see Table 4.3) and the “26-30” factor of the MCD complexity over that for non-MCDs (see Table 4.4).