This section provides an overview of the performance of the direct and EPM-based selectors developed for the 33 kV meshed distribution system and the IEEE 14- and 57-bus systems. The overview is provided via three means. Firstly, a table (Table 8.1) lists the numbers
of selectors created for each selector design variant, the number of those that provide performance benefits, and the number of selectors that provide statistically significant performance benefits. The definition of “performance benefit” depends on the system and the performance measure considered, and is explained in the subsequent text for each system. Secondly, a series of plots (Figures 8.1 to 8.5) present the performance of the selectors graphically, alongside the performance of the oracles and the power flow management algorithms. The plots include general views (such as Figure 8.1a) of the entire performance space, showing the performance of all the selectors, and detailed views (such as Figure 8.1b) that show the performance of the selectors whose performance is close to the both the oracles and the most effective power flow management algorithms. Thirdly, the text provides an explanation of the table and figures, and draws out the key findings about the overall performance of the selectors.
Whether a performance benefit is statistically significant is assessed using the same method as used for comparing the performance of the power flow management algorithms and the oracles in Chapters 4 and 5 (described in Section 4.1.3). The performance of a selector is compared to that of the most effective algorithm for the performance measure of interest using a paired t-test, yielding a p-value. The p-values of all the comparisons of performance made for a particular system and performance measures of interest are then collected together (they are treated as a single family of hypotheses for each system) and the Benjamini-Hochberg procedure is applied to determine which represent statistically significant results, while correcting for making multiple comparisons. As in the previous chapters, a significance level of α = 0.05 is used. Note the families of hypotheses in this chapter are considered to be separate to those in Chapters 4 and 5, as the hypotheses in those chapters were comparing the performance of all the power flow management algorithms (and oracles) for all performance measures, whereas the hypotheses in this chapter compare the performance of algorithm selectors to that of one or two power flow management algorithms for the performance measures of interest.
A summary of the overall performance of the direct and EPM-based selectors created for the 33 kV meshed distribution system is shown in the first group of four rows of Table 8.1. As described in Section 7.1.2, no (untuned) weighted selectors were created for this system as two objectives needed to be considered by the selectors. For this system, the performance benefit is when a selector applies less curtailment than PFM-OPF, while still removing all overloads (the same as PFM-OPF). So, for example, the table shows (in the first row) that 1.47% of the unweighted direct selectors apply statistically significantly less curtailment than PFM-OPF, while still removing all overloads.
Table 8.1 Summary of direct and EPM-based selectors giving performance benefits for each system and overload objective
System & (overload objective) Selector design variant Total
selectors giving a performanceNumber of selectors benefit
No. of selectors with statistically significant performance benefit 33 kV meshed (either) Unw. direct 1628 24 (1.47%) 24 (1.47%) Wei. direct - - - - - Tuned direct 419 20 (4.77%) 15 (3.58%) EPM-based 21,175 186 (0.88%) 143 (0.68%) IEEE 14-bus (number) Unw. direct 2144 892 (41.60%) 738 (34.42%) Wei. direct 530 298 (56.23%) 196 (36.98%) Tuned direct 526 282 (53.61%) 195 (37.07%) EPM-based 21,175 5896 (27.84%) 2887 (13.63%) IEEE 14-bus (energy) Unw. direct 2144 612 (28.54%) 402 (18.75%) Wei. direct 530 222 (41.89%) 181 (34.15%) Tuned direct 526 216 (41.06%) 168 (31.94%) EPM-based 21,175 5316 (25.11%) 3165 (14.95%) IEEE 57-bus (number) Unw. direct 1288 1040 (80.75%) 1009 (78.34%) Wei. direct 301 165 (54.82%) 163 (54.15%) Tuned direct 292 159 (54.45%) 157 (53.77%) EPM-based 12,705 6443 (50.71%) 6301 (49.59%) IEEE 57-bus (energy) Unw. direct 1288 298 (23.14%) 172 (13.35%) Wei. direct 301 38 (12.62%) 27 (8.97%) Tuned direct 292 35 (11.99%) 26 (8.90%) EPM-based 12,705 1453 (11.44%) 1248 (9.82%)
Figure 8.1 plots the performance of every selector created for the 33 kV meshed distri- bution system, in terms of the number of overloads and the curtailment applied. The figure shows that some of the selectors always select the same algorithm, which is shown by the markers indicating the performance of selector overlapping with the marker of one of the power flow management algorithms. However, numerous markers of selectors’ performance do not overlap the markers of the algorithms’ performance, which shows that those selectors are selecting at least two different algorithms for a number of states.
For the IEEE 14-bus system, the definition of “performance benefit” depends on which overload objective is considered. If minimising the number of overloads is considered (the second group of four rows in Table 8.1) then a selector delivers a performance benefit if it leaves fewer overloads than PFM-OPF. If minimising overload energy is considered (the third group of rows in Table 8.1) then there is a performance benefit if a selector leads to lower overload energy than PFSF-TMA. The performance of the selectors created for the IEEE 14-bus system is shown graphically in Figure 8.2, in terms of the number of overloads and the amount of curtailment, and in Figure 8.3, in terms of overload energy and curtailment.
For the IEEE 57-bus system, the same definitions of “performance benefit” are used as for the IEEE 14-bus system, however, the comparisons for both overload objectives are made relative to the performance of PFM-OPF. The fourth and fifth groups of four rows in Table 8.1 provide a summary of the selectors created for this system, according to the overload performance measure considered. The performance of the selectors for the IEEE 57-bus system is shown graphically in Figures 8.4 and 8.5 for the number and energy of overloads, respectively, against the amount of curtailment applied.
The results summarised in Table 8.1 show that each of the selector design variants considered is able to produce selectors that offer statistically significant performance benefits for each of the case study systems. There are variations in the proportion of selectors that give a performance benefit across the different systems, design variants and overload objectives considered. For example, for the 33 kV meshed distribution system, relatively few selectors provide performance benefits, which is likely due to the curtailment performance benefit also requiring algorithms to be selected that remove all overloads. Furthermore, for the IEEE 14- and 57-bus systems, each selector design variant is less likely to produce a selector that offers a performance benefit for overload energy, than for the number of overloads.