The third, and the last phase of our research consisted in applying our own findings and models we have established during previous phases and validate them by creat-
CHAPTER 3. RESEARCH OVERVIEW
ing an effective visualisation to effectively identify several parallel performance prob- lems. We decided to address data locality issue and use our observational model along with the diagnosis model for the design.
The visualisation we have designed consists of three main components and is ex- plained in detail in Chapter 7; the main design goal of the tool is to allow effective identification of data locality issues. In the past two decades processing speeds in- creased by around 50% per annum, whereas the time to access DRAM memory fell by only 10%-15% per year. The result is that it now takes hundreds of processor cycles to read a value from main memory. This phenomenon is often called the “memory wall” and efficient use of CPU caches is required to achieve good performance; this recent trend influenced our decision to address data locality problem in our visualisation.
Once we had a visualisation, an evaluation was performed using a convergent basic design as part of our mixed methods research, to assess whether programmers could identify correctly data locality problems. The visualisation was compared to a simple source-code reading exercise, since there is a large percentage of programmers (+- 60% as found by one of the studies sponsored by Intel) who do not use any parallel performance tools and analyse their performance problems by simply looking at the source code. This was consistent with the feedback we received during the interviews. In our experiment we have found, among other things, that the correctness increased significantly across the board (experts and non-experts alike), along with an overall confidence boost. Chapter 7 goes into more detail on the experiment and the findings.
Chapter 4
UnderstandingtheProgrammer
A qualitative study was carried out to better understand how developers approach parallel programming, identify the issues that they are trying to address, and how software performance analytics systems could help them in their work. We conducted a range of interviews across various organisations including a large corporation pro- ducing operating systems, one of the largest B2B software corporations, universities, research laboratories and small to medium sized software companies. A broad spec- trum of organisations was targeted in order to obtain a general overview of the field of parallel programming practices and problems.
4.1
Methodology
Interview participants were practicing software developers, engineers and academics. All of the interviewees were practicing parallel programming in some way, including high performance computing (HPC), Graphics Processing Unit (GPU) accelerators, many-core and/or multi-core programming. The goal of the semi-structured inter- views was to explore and understand the daily activities and challenges faced by pro- grammers and to explore the techniques they use in order to tackle those challenges. The interviewees were asked to describe the nature of their work and recent projects involving parallel programming.
Overall, 22 people were interviewed. 14 of them were recorded, resulting in over 8 hours of audio. Interviews were transcribed (around 47,000 words), open coded (582 open codes, unified into 252 codes), categorised in 8 major categories and sub- categorised in 23 sub-categories. Interview participants are summarised in Table 4.1.
Interviews were semi-structured, and interviewees were asked to talk about the challenges they had to overcome in their every day work related to parallel program-
CHAPTER 4. UNDERSTANDING THE PROGRAMMER
Participant Organisation Activity Years P1, Male Corporate Engineering 0-5 P2, Male Corporate Engineering 0-5 P3, Female Corporate Engineering 5-10 P4, Male Corporate Engineering 5-10 P5, Male Corporate Engineering 10+ P6, Male Corporate Engineering 10+ P7, Male Corporate Engineering 10+ P8, Male Corporate Engineering 10+ P9, Male Corporate Engineering 10+ P10, Male Corporate Research 10+ P11, Male Corporate Research 10+ P12, Male Corporate Research 10+
P13, Male SME Consultancy 5-10
P14, Male SME Consultancy 5-10
P15, Male SME Consultancy 10+
P16, Male SME Engineering 5-10
P17, Female SME Engineering 10+
P18, Male SME Engineering 10+
P19, Male University Research 0-5 P20, Male University Research 5-10 P21, Male University Teaching 5-10 P22, Male University Teaching 5-10
Table 4.1– A table of participants with their years of experience, main activity and the type of organisation.
ming, the tools they had employed, and practices they used. While this analysis method has its roots in Grounded Theory [157], we would like to highlight that it was not our goal to build a holistic theory from the data. Following common practice in HCI [115], we used the approach as the foundation for a focussed analysis of the transcribed interviews.
Listed below are the some of the questions that we asked developers during our interviews. In keeping with the methodological background, the questions were de- liberately kept general and we tried to ”let them talk” from these starting points:
• Could you introduce yourself, describe your background and professional expe- rience?
• Could you describe some projects you worked on and what kind of challenges you have met in these projects?
CHAPTER 4. UNDERSTANDING THE PROGRAMMER
• What kind of interesting projects have you had and how did you solve them?
• What kind of challenges, what kind of problems do you face on a daily basis?
• When you’ve got a piece of code, how do you find where it should be optimised?
• What kind of profiling, tracing tools are you using?
• How do you usually determine where the bottlenecks are in the code?
• You spoke about knowledge of hardware. Could you specify what kind of knowledge of hardware you needed?
• You mentioned patterns in parallel programming, do you use them? Are they relevant?
• Alright, do you have something else to add?