There is relatively little literature on how programmers learn, whether about novices or professional programmers. Johnson and Senges [59] studied how programmers learned to function in a complicated organisation, Google. They concluded that the majority of programmer learning there was peer learning, facilitated by strong corporate standards and culture; obviously the results are limited to that organisation. Other studies have
Process Improvement (SPI). So, for example, a wide-ranging quantitative study by Dyba [36] examines learning as one aspect of SPI; it differentiates Exploitation, the dissemination of existing knowledge, from Exploration, the gaining of new knowledge; it concludes that both have a positive effect on productivity but does not explore mechanisms.
A little-known work by Enes and Conradi [40] used interviews to discover how professionals, including programmers, acquire their expert knowledge. It concludes that the preferred learning mechanisms are all informal ones: especially on-the job training and personal interaction. It also highlights, as an important factor, professional pride in having ‘expert areas’ of competence. Unfortunately for our purposes, the results include only a very small sample of programmers (4) in a limited area (Trondheim).
Murphy-Hill et al. explored how developers find new software tools [72]; the research was more wide-ranging and included both an initial survey and a larger scale, 79 participant, diary-based survey. They concluded the event is relatively rare, and happens more through joint working than recommendations, though the only solution proposed was to create a tool recommendation website. Proksch et al. used an extensive and carefully analysed literature survey to explore such a recommendation site in detail [85]; the result reads as a requirements specification. So far there is little evidence of such systems being trialled in practice.
In the context of learning about software security, a particularly important finding is that of Conradi and Dyba [26]. Based on interviews with 21 development team members in 5 Norwegian companies, they analysed carefully the effectiveness of written routines on software process improvement. They identified that programmers have difficulty with learning from the output of process improvers, and particularly with learning from formal written routines. As a result, developers tend to avoid such approaches:
Developers are rather sceptical at using written routines, while quality and technical managers are taking this for granted. This is an explosive combination. (Conradi and Dyba [26])
This suggests an ‘impedance mismatch’ between those who write instructions and processes for developers, and the developers themselves who are expected to carry them
anecdotal evidence and personal experience of the authors tends to confirm the finding in respect of UK and US developers. We can speculate that security experts tend to think in terms of complete lists of issues and ways to break software; developers think in terms of simplest ways to create desired functionality.
Thus though one might expect the most effective approach to teaching security to be a prescriptive set of instructions to programmers what to do, a ‘Secure Development Lifecycle’ such as those promoted by Microsoft [69] and others, in practice those do not appeal to developers. This conforms to the author’s own experience; he has never encountered an app development team voluntarily adopting a process of that kind. This means that for app developers we need to find a more lightweight, less prescriptive approach.
A different approach to teaching programmers was the ‘patterns’ movement, discussed in more detail in section 4.2. Yskout et al. tested experimentally the effect of using security patterns in server design; though the paper states there was no benefit, in fact the results suggest a benefit but were statistically inconclusive [116].
Recently several teams have investigated how app developers learn security. Balebako et al. interviewed a dozen app developers in small to medium sized US companies, and surveyed over 200 US app developers to find out their approach to security and privacy issues. The survey was carefully constructed to avoid bias, and concluded that most developers approach these issues using web search, or by consulting peers [10]. Interestingly they also found that security and privacy behaviours were only weakly correlated.
A survey by Acar et al. [2] also concluded through a survey of nearly 300 successful app developers worldwide that they learned security through web search and peers. They went on to use a well-crafted practical experiment with over 50 Android developers to evaluate the effectiveness of the different ways of learning app security; this produced the surprising result that programmers using digital books achieved better security than those using web search.
Two projects, by Xie et al. and Nguyen et al., have developed IDE-based tools to teach programmers by detecting possible security flaws in Android developments [77,111]. The
al.’s project is now finished and Nguyen’s has not yet reached the stage where its effectiveness can be tested. Others, such as Near and Jackson, and Lerch et al. [65,76], have code analysis tools to detect security defects; these work but provide only limited feedback to developers. So far we are not aware of any literature analysing the effectiveness of such approaches.