Chapter 4. Analysis Methodology
4.3 Vulnerability Disclosure
Learning from the case studies and leveraging our previous experience in performing vulnerability disclosure, in this section we look at how we communicate the vulnerabilities to the vulnerable system owners with the aim of getting the vulnerabilities efficiently fixed. We broadly categorise
vulnerability disclosure into two categories: (1) Full Disclosure (publishing the vulnerabilities on the Internet without any restriction), and (2) Responsible Disclosure (all relevant stakeholders agree to allow a period of time for the loophole/vulnerability to be patched before publishing the details on the Internet).
Once we decide to notify the vulnerable stakeholders, there are several other crucial decisions to be made in order to provide a proper structure to the disclosure exercise. All of these decisions need to consider the ethics and morals of passing vulnerability information to the affected third party, and these considerations are not widely explored in the scientific community yet. There are questions that need to be addressed, such as: To whom do we disclose the vulnerability information? How should we handle the responses from the affected third party? How much information shall we divulge in the notification message?
In an approach to find a consistent and coherent procedure to achieve effective vulnerability exercise, we will discuss in the following sub-sections two key steps: disclosing the findings and following this up with post disclosure activities.
4.3.1.1 Disclosing the Findings
The following issues were considered when disclosing vulnerability findings.
Identifying whom to disclose to. It may be complicated to determine which parties to disclose the vulnerability to, and even if stakeholders have been identified, it is not always straightforward to actually contact the party. Large companies (like card issuing banks and payment acquirers) typically provide customer service contact details, but they might not be the right people to approach for vulnerability disclosure. In some cases, there might be a dedicated contact email or number for security-related issues (such as for reporting phishing and spam), but in others, there is nothing obvious as a port of call.
The timing of the disclosure. Passing the vulnerability information forward to the affected
stakeholders could have helped them to put down the adversarial activities. However, not disclosing or delaying passing the vulnerability information to the affected stakeholders may benefit the research in many other ways. For example, this would allow researchers to measure the behaviour and
intentions of attackers, or to study how long the attackers would take to exploit the vulnerability [77][83]. This is a key factor in determining the likelihood of an attack and will give research a prominence dimension from which cyber-security can be studied, and if not followed may scale down eminence of the research. In general, data collection strategies and studies on attacker interest and
behaviour in exploiting the vulnerability are recommended by research on vulnerability disclosures [51][52][58][59]. However, it should not be extended beyond once the conclusions are made.
Researchers should avoid interference with middle-level personnel and should commute with the most relevant managers whenever available[77].
Importance of the First Notice. With our research on the CNP payment system we intended to
perform multiple disclosure exercises with the hosts that remained vulnerable did not respond to our first notification. We derived two important findings.
Firstly, those contacts who accepted the vulnerability on our first notice patched their systems we found that only the earliest notice was likely to show more effect on the patching rate. Secondly, systems that did not patch or remediate after the first notification chose not to remediate and remained vulnerable.
Level of Detail of Disclosure. Disclosure reports that lack the details of compromise will have a very limited response from the notified hosts [77]. For efficient notification process, the vulnerability disclosure report in our research included
A detailed description of every successful attack or any weakest link found during the assessment process
Features of the attack whether if it's practical or theoretically demonstrated. Disclosure report also mentioned the cost and time required to devise the attack.
The report clearly detailed the technical sophistication, and in detail, the tools used or are necessary to practice the attack.
A detailed description of any publicly available information or insider helps that may be required to make the attack practice. Is there any inside information about the system needed for the attacks to be exploited?
Samples of the defeated security devices were provided if practical and appropriate.
A statistical summary of the level of effort made during the vulnerability assessment was detailed. The number of times the attack was successfully performed, time to develop them, time to execute them, type of defeats.
Suggested Counter Measures. The vulnerability disclosure report included not just discovering and detailing the weakness, but also suggested effective countermeasures, if practical. By providing suggested countermeasures, we will lend more credibility, but have to be careful that these potential fixes will not make the situation worse [84].
During the course of our disclosure exercise research, we had learned that the affected hosts were more amenable to deal with the vulnerability when a solution was offered along. It is essential that the
vulnerable hosts from their core deliverables. Apparently, there are other researchers [85][86] that had expanded in detail the effects of patching when the solution was suggested. Li et al. in [86] found that the hosts were more open to fixing their system when the solutions were suggested.
Tracking the Patching Behaviour. We could collect relevant data on characteristics of the defender. Defender features might include attributes of the hosting provider (e.g., large vs small, shared vs dedicated hosting, country headquarters), site owner (company size, company vs individual, country headquarters) or associated registrar.
Handling the Response Messages. We argue that those involved in take-down should consider how
to protect individuals from harm while creating an opportunity for research to advance the
understanding of how to better perform take-down. Opting to keep information private can be even more dangerous than the reckless publication of information that aids attackers. The harm may be harder to observe directly (slowed take-down speed, lack of pressure to improve practices, etc.) but equally destructive.
However, not all issues addressed during vulnerability assessments are mitigated because a lot may depend on the business decisions involved. In such a case, publishing the details of a vulnerability in the public will ‘name and shame’ responsible parties and victims [84].
4.3.1.2 Post Vulnerability Disclosure
While most of the research we have looked into so far have only carried out the assessment until the disclosure of the vulnerability, a select few sought a further investigation into post vulnerability disclosure. Experiments carried out after the disclosure are useful to determine the adversarial attractiveness against the affected systems. Understanding the insidious tactics against the affected systems at this stage will provide an in-depth insight into the methods and psychology of the attackers. In that regard, there are three useful activities that can be performed post vulnerability disclosure:
Selecting hosts for further analysis: this will allow the research to continue to explore other, similar systems that might be affected by the same vulnerability, and to see if they get attacked too, or if some remedial actions can effectively thwart potential attacks.
Adversary Identification: in an ideal world, it would be desirable to be able to identify the attackers, so that they can be brought to justice. This activity will be closely related to electronic forensic investigation that law enforcement agencies or certain security companies have the capability of performing.
Adversary Characterisation: when it is not possible to identify the attackers, it would still be useful to be able to characterise attackers so that we can understand their profile better in order to come up with a more effective countermeasure.
4.4
Conclusion
In this Chapter we constructed a framework of terminology, methods, and recommendations for the use of vulnerability assessment, attack landscaping and the associated disclosure processes as a research method.
In the next part of this PhD we will demonstrate the application of our analysis methodology, to evaluate the security of CNP payment system. In Chapter 5 and Chapter 7, we will apply the first requirement of our methodology and perform security architecture assessment of the CNP payment system. In Chapter 6 and Chapter 8 we demonstrate the application of attack landscaping and vulnerability disclosure over CNP payment system.