Intuition, gleaned from decades of secure software engineering best practices, sug- gests a tension between the rapid deployment of new software features and the avoid- ance of software defects, particularly those affecting security.
The rapid release employed by Mozilla for Firefox, in which new software releases, with new features, are rolled out on an aggressive schedule, seems as if it could only come at the expense of security. Users concerned with security, one might assume, would be better off eschewing the latest features in favor of the more mature, stable ESR releases. Agile programming methodology, particularly in an application as
exposed as a web browser, should be a security disaster.
At least with respect to vulnerabilities disclosed during the lifecycle of each Fire- fox release, my results suggest that this intuition appears to be wrong. Vulnerabilities are disclosed in the older code at least as often as they are in the newer code. This is both surprising and encouraging news. It suggests that during the active lifecycle,
the adversary’s ability to discover security defects is dominated less by the intrin-
sic quality of the code and more by the time required to familiarize themselves with
it. It suggests that the Firefox rapid-release cycles expose the software to a shorter
window of vulnerability. [AFM00] Frequent releases of new features appear to have
provided the Firefox developers with new grace periods orsecond honeymoons(using
terminology I coined in [CFBS10]). While there may also be other factors affecting vulnerability discovery which are changing over the duration of software evolution I studied, it is clear that the net effect of RRC, seen in the data, has been the attenuation of the attacker.
Even while generalization remains an open question, in Firefox, the unexpected benefit of frequent large code releases appears to be a lengthening of the attacker’s
learning curve. The findings reported in this chapter further support the ideas
that familiarity with a codebase is a useful heuristic for determining how quickly vulnerabilities will be discovered and, consequently, that software reuse (exactly because it is already familiar to attackers) can be more harmful to software security than beneficial.
These results are consistent with a “Honeymoon Effect”, and suggest that the pattern exhibited by vulnerability disclosure in Firefox could result from would-be
attackers having to re-learn and re-adapt their tools in response to a rapidly chang- ing codebase. [CFBS10] These results should lead software developers to question conventional software engineering wisdom when security is the goal.
The “S” learning curve shape seen in Figure 4.8 could be construed as a measure
of the increase in proficiency as result of repeated exposure. In other words, a
measure of the skill and expertise resulting from increased familiarity over time. As demonstrated by the Honeymoon Effect presented in Chapter 3, this learning curve appears to play a significant role in the software vulnerability lifecycle. As demonstrated above, rapid code changes may affect an adversary’s familiarity with the codebase, resulting in a longer, or even a second honeymoon.
I chose Firefox software as the basis for analysis, as it was originally architected using the traditional development model and switched to rapid-release midstream. It will be interesting to see if other software systems, including those that have been
designed and developed usingonly Agile methods share the same properties. It will
also be interesting to see what effect the switch to silent auto-updates has had on the vulnerability life-cycle. The dataset that I integrated for Firefox with its large code-base, and large user-base, coupled with its prominence as an attack target is strongly suggestive that the rapid release strategy has significant and unexpected security advantages in real world systems.
Chapter 5
Testing the Effect of a Single
Intrinsic Property
“The only relevant test of the validity of a hypothesis is comparison of
prediction with experience.”
(Milton Friedman)5.1
Introduction
Recall that each time Mozilla releases a new version of Firefox, hundreds of thou- sands of LOC were being added, modified or removed and thousands of files were affected.(see Tables 4.1 and 4.2 )
Because these changes are primarily the result of adding new features and not merely fixing defects, it is important to recognize that each time a new version is released, its substance has altered in some way. Whether it is measured as lines of code, or numbers of files, these changes are in effect altering some of the intrinsic properties of the product as a whole.
At the same time, it was demonstrated in the previous chapter, that even after Mozilla moved from a traditional development model to a rapid release development
cycle, the Honeymoon Effect is still predominant. New Firefox versions are likely to experience a positive Honeymoon twice as often as not.
This begs the question, do releases with more changes directly correlate with an increase in the time from release until the first vulnerability is discovered and exploited?
This chapter explores whether coarse-grained changes to intrinsic software prop- erties are the sole cause of the Honeymoon Effect and attempts to answer the fol- lowing questions:
• Can the Honeymoon Effect be directly correlated with the total LOC or file
changes? That is, does the addition, modification or removal of code alone account for the increased likelihood of having a positive Honeymoon?
• Is there a minimum number of files that must be changed in order for the
Honeymoon Effect to appear?
• If either of the above is true, how large a change is required?
• Does one type of change matter more than another? (i.e., adding new code,
modifying existing code, or removing old code)
• Do code changes in one type of programming language affect the Honeymoon
more than another?
• Does the frequency of file changes matter?