Section 3.2 explains the choice of analysis method: Grounded Theory. The method is described in detail in Section 3.3.
4.2.1 Research Sub-questions
The wide scope of the questionRQ 2 What interventions can change the environment for members of the development team to achieve good security, considering cost-efficiency, motivational factors, choice of tools, supporting processes, culture, awareness, training and skills?
led to a need for a more specific focus. Accordingly, in the analysis the author focussed on two further questions, one addressing the approach to creating and introducing interventions:
RQ 2.1 What approach to interaction with software development teams leads to the best results in encouraging secure development?
And one addressing the practical aspects:
RQ 2.2 What specific intervention techniques do specialists consider most cost-effective in helping developers improve security and privacy in their code?
4.2.2 Research Participants
Interviewees were chosen opportunistically; the author’s personal connections in industry provided introductions to a number of successful, and mostly senior, practitioners with considerable experience of helping teams achieve software security. Table 6 shows the interviewees, with an indication of organisation size: Solo for consultants working with a variety of organisations, Medium for organisations between 10 and 1000 people, and Large for larger and government ones; also, a description of the interviewee’s main day-to-day role. Each interview was with a different organisation, other than P14/P15; and P12/P16 who were interviewed together. Most are based in the UK other than P10, P13 based in Germany, and P5 in the USA.
Each expert worked with a different set of software development teams. The table also gives a subjective indication of the ‘secure software capability maturity’ [90] of those teams, based on the author’s observations during the workshops: an indication of how expert we might regard the teams involved at delivering secure software, and therefore at what level the expert was normally working:
Low Teams had little or no awareness or activity related to software security Medium Teams were aware of and addressing security issues, typically including
some developers with good security knowledge.
High Teams are expert at software security, within an organisational culture that assigns it a high priority.
Figure 11 visualises the same data, illustrating the range of participants. The horizontal axis indicates their organisation size; the vertical axis the ‘secure software capability maturity’.
To show further the range of participants involved, Figure 12 summarises the recruitment process. Various people were approached; the horizontal axis indicates how long the author had known the people approached. Some were interviewed directly, and these are shown closest to the axis. Professional and personal contacts are shown above the axis, and contacts encountered via academia—mostly encountered at industry-academic workshops—below. In other cases, the contacts approached referred others, and the resulting interviewees are shown further from the axis.
Table 6: Experts Interviewed
ID Org. size Organisation type Est SCMM Main Role P1 Medium Outsourced software
developer and consultant High CEO
P2 Solo Security consultant Low–Med Consultant
P3 Large Security and military supplier High Team leader P4 Large Research organisation Medium Research and
support P5 Large Operating system supplier High Security team
leader
P6 Large Security and military supplier High Security expert P7 Medium Software security tool
supplier Medium CEO
P8 Large Telecommunications provider Medium Security expert
P9 Solo Security consultant High Consultant
P10 Large Software package supplier High Security expert P11 Medium Software security service
supplier Low-Med Training and consultancy P12,
P16 Medium Telecoms service provider High Security expert, Team lead P13 Large Research organisation Low-High Research and
consultant P14 Medium Outsourced software
developer High Principal
engineer P15 Medium Outsourced software
developer High Security expert
Chapter 4: Expert Survey
Those chosen were experts in secure software development; many were therefore predominantly developers first and security experts second. As Table 6 shows, only ten of the fifteen had roles or company missions specifically related to security. Their qualification as experts was based on their reputations, either directly from the researchers’ knowledge; or as validated by the people who referred them.
4.2.3 Interview Questions
We consulted the interviewees as experts, rather than analysed them as subjects. Our questions aimed to draw out what they themselves had found most effective, and what they had seen to be most effective in other teams. Figure 13 shows the main questions we used. These were generated with an initial ideation session, adding further open questions based on the Appreciative Inquiry (Section 3.3.5), and detailed sub-questions based on findings in the author’s earlier interviews of App Security Software Specialists [183].
In the following sections, quotations from the interviewees are in italics. Quotations are edited to protect confidentiality and indicate context: square brackets show additions and replacements; ellipses show removals.
Figure 11: Organisation Size and Security Capability
Small
Figure 12: Recruitment Methods for Expert Survey
Introduction – establish context
• What is your current role, and what do you find yourself doing day-to-day?
Tell me about a typical day at work?
• Briefly, how did you first get involved with secure software development?
Exploration
• What’s your interest in security? What do you do about it, and how do you deal with it day-to-day?
• What do you want to achieve when you’re helping a team improve software security? How do you define and measure success?
• What is the most successful intervention technique you’ve found? Where do you concentrate your efforts?
• Can you think of a particular triumph in your work – where you’ve worked with a team that has improved their security? How did you achieve that?
• Have any of your teams used code checking tools? How happy were you with their effectiveness at finding problems; and their ease of use?
• What do you find effective as motivation for secure development?
• How do you frighten developers into security, or emphasise the positive aspects?
• To what extent are laws and standards helpful in getting teams to be effective at software security? How do you find out about them and keep up to date?
• When new people join an existing team, how do you motivate them and how do they learn what’s required? Do you encourage double checking of contributions from new people or treat them 'as usual'?
• What are the best ways you’ve found to get teams to tackle specific things:
• Security coordination with other teams;
• Reviews and penetration testing;
• Designing to get feedback from the users?
• What else?
• Have you had a nightmare scenario? Or consider this nightmare scenario.
You’re working with a team that’s just learned they have a security flaw in a website that’s very heavily used. Have you even had a situation like that (no details required)? What did or would you do to help the team tackle it?
Vision
Let’s imaging we’re a few years in the future, and the problem of getting teams up to speed with app security has been licked; it’s now a part of everyday software development life. How was it done? What were the first small steps?
Clarification (as appropriate)
• And how did you achieve that?
• Oh I see. Could you give an example?
Figure 13: Expert Survey Questions
Chapter 4: Expert Survey