State Variable H Discoverer Vulnerability Activity

Linked closely to the number of vulnerability discoverers within the VDDS are the discovery activities of the discoverers. To discover a vulnerability, one must expend effort to do so, and have a level of technical competency. The effort that is expended to find a faults and vulnerabilities within software has been extensively studied, with vulnerability discovery effort being considered to be analogous to the testing effort that is needed to test and debug software systems (Alhazmi, 2006; Alhazmi et al., 2005; Holm et al., 2013; Johnson et al., 2016; Pham, 2006; Rescorla, 2005; Sommestad et al., 2012; Wang et al., 2014; Woo et al., 2011). Pham (2006) defines two resources that direct the pace of testing, manpower, including failure and correction identification personnel, and computer time (Pham, 2006, p.482). Consequently, identification and fault correction can therefore be considered to be analogous to vulnerability discovery and vulnerability patching.

To estimate the effort a discoverer expends to uncover a vulnerability we must calculate the frequency of discovery per individual and the time that is taken. A conservative estimate is that a period of 3 months is required to find one vulnerability, whereas other estimates state range 1.5 … 3 days dependent upon experience. (Holm et al., 2013; Nom, 2015; Sommestad et al., 2012). An effort model can be considered as a six stage process, with four distinct categories of discoverer, expert, intermediate, beginner and novice, each attracting a number of days to develop a vulnerability and exploit (McQueen et al., 2006). The mean time to discovery for the categories of discoverer range 0 … 122 days for expert, range 0 … 165 days for intermediate, and in excess of 200 days for beginner and novice (McQueen et al., 2006).

Study / Evidence Effort Expended (Days) (Sommestad et al., 2012)

{1.5 … 3} (McQueen et al., 2006) (Expert)

{0 … 122} (McQueen et al., 2006) (Intermediate)

{0 … 165} (Nom, 2015)

120

Table 31– Summary of Effort Ranges

Given the wide range of recorded durations to discover a vulnerability the assumption is made that only expert and intermediate discoverers will be considered from the McQueen et al., 2006 study.

5.4.8.1 Reference Mode and Analysis

As we are looking to derive the shape and behaviour of key variables from within the VDDS, two key aspects, the rate vulnerabilities are disclosed by researchers, and inter arrival rate are important. The importance of these is highlighted as System Dynamics models aim to show the flows of information or resources around the system, and accurately representing these within the model is key. By understanding the rate which vulnerabilities enter into and exit the VDDS allows the flow of vulnerabilities and disclosure choices to be made. Typically, the process of arrival is described as a queueing system, and specific notation known as Kendall notation is used (Ibe, 2011, p.64). As such we will adopt the same nomenclature, however this is for descriptive purposes only. Arrival rate is defined as per the number of occurrences per unit time, (Ibe, 2011, p.31) and inter-arrival time as the time between occurrences. The unit of time is disclosures per calendar month, 30 days.

Gaining a reliable figure on the number of vulnerability researchers that are within the system and active is difficult, but not insurmountable. As stated achieving a reliable figure on the number of vulnerability researchers that are active within the system is difficult, but not insurmountable. Again, we return to ExploitDB.com dataset which records the submission name and date of the discovered vulnerability. Utilising ExploitDB.com provides a reasonable indication of the number of vulnerability researchers that are active that have

reported vulnerabilities on the ExploitDB.com platform. The activity of vulnerability researchers is a key indicator relating to the number of vulnerabilities that are eventually disclosed. As such the shape of the data is characterised as a series of disclosures, indicating the flow of vulnerabilities within the VDDS. Statistics were calculated showing the mean number of days participants took to discover a vulnerability, which was derived looking at published reports within on the vulnerability database ExploitDB.com. The calculated mean is 256 days, with the median value of 1 day, outlined in Table 32 below. Therefore, the most common type of submitter only discloses one vulnerability to the ExploitDB.com platform in their lifetime, with this category accounting for 61.6% of all submitted reports. Inspecting the data, we can see that there is a heavy positive skew of +4.09, suggesting that the median value of 204 days is a more representative figure for the number of days a researcher is actively discovering vulnerabilities when we normalised and remove the single discoverer records skewing the data.

Vulnerability Discoverer Name1 First submission Last Submission Days active Frequency Current Probable State 1 Metasploit 16-06-08 21-11-16 3080 1450 Active 2 Google_Security_Research 04-04-13 20-12-16 1356 416 Active 3 Luigi_Auriemma 17-12-02 29-06-12 3482 416 Inactive 4 High_Tech_Bridge_SA 13-04-10 29-04-16 2208 409 Active 5 anonymous 01-08-88 13-04-15 9751 359 Active 6 LiquidWorm 22-07-08 16-12-16 3069 353 Active 7 rgod 21-05-05 11-12-13 3126 333 Inactive 8 indoushka 23-12-09 08-05-14 1597 294 Inactive 9 r0t 23-03-03 14-03-10 2548 257 Inactive 10 ZoRLu 17-02-08 24-09-14 2411 221 Inactive 11 ajann 26-05-06 15-01-09 965 204 Inactive 12 Lostmon 20-05-03 28-03-12 3235 188 Inactive 13 shinnai 11-12-06 23-11-16 3635 176 Active 14 Moudi 07-01-09 10-09-10 611 170 Inactive 15 laurent_gaffie 15-09-06 09-11-16 3708 161 Inactive 16 GoLd_M 07-01-07 10-08-12 2042 152 Inactive 17 Kacper 12-05-06 15-06-09 1130 149 Inactive 18 Stack 17-01-08 22-01-10 736 147 Inactive 19 S@BUN 22-01-08 25-11-09 2411 143 Inactive 20 cr4wl3r 03-08-09 24-12-13 1604 130 Inactive

Table 32 – Vulnerability Disclosure Activity Timeline

1 _{Groups of researchers exist within the dataset, for example Metasploit, High_Tech_Bridge_SA and}

Using the amended data mean lifetime of a vulnerability discoverer is 1,258 days, with a median of 866 days. Using the frequency of discovery, we can calculate the most prolific vulnerability discoverers find between 416 and 130 vulnerabilities during the active lifecycle. Table 33 below outlines the top 20 vulnerability durations with relative rank, frequency counts, active days and percentage accounted.

All Data Single Discover Removed

Rank Days Active on platform

Frequency Count

Percentage (%)

Days Active _Frequency

Count Percentage (%) 1 1 4830 61.61 2 50 1.66 2 2 50 0.63 3 40 1.33 3 3 40 0.51 11 38 1.26 4 11 38 0.48 _{4 34 1.13} 5 4 34 0.43 _{21 34 1.13} 6 21 34 0.43 _{5 30 0.99} 7 5 30 0.38 _{6 30 0.99} 8 6 30 0.38 _{7 25 0.83} 9 7 25 0.31 _{111 23 0.76} 10 111 23 0.29 13 21 0.69 11 13 21 0.26 14 20 0.66 12 14 20 0.25 12 19 0.63 13 12 19 0.24 _{9 18 0.59} 14 9 18 0.22 _{31 18 0.59} 15 31 18 0.22 8 17 0.56 16 8 17 0.21 23 15 0.49 17 23 15 0.19 15 14 0.46 18 15 14 0.17 41 14 0.46 19 41 14 0.17 51 14 0.46 20 51 14 0.17 113 14 0.46

Table 33 - Top 20 vulnerability discoverers with relative rank, frequency counts, active days and percentage accounted