Sampling, who and size of sample

One of the considerations before starting the interviewing process was how I would analyse the data, a question which according Kvale and Brinkman (2009) should not be left until after the data has been collected. Weighing up how much data was

manageable (in terms of collection, transcription and analysis) and whether a small number of informants could offer enough data was part of the process of designing the data collection. Having sought advice from my supervisor I decided on a small sample size. According to Patton (1990), the use of relatively small, purposefully selected samples is a key feature of qualitative research and marks its difference with quantitative methods of data collection (Ibid, p. 169). In the case of my research, purposeful sampling was based on the following criteria: (a) engagement in the SDG4 process over a period of more than a year; (b) participation in negotiations, key

meetings and/or sustained and high-profile advocacy, direct involvement with lobbying and networking with key decision makers; (c) involvement in either UN, government or civil society organisations; (d) at least mid-senior level in their professional posts (this gave authority in the engagement). In other words, they had to be close to the process of SDG4’s formulation. I considered these criteria as a means of strengthening the reliability of the data.

The research was what Maguire and Ball (1994) refer to as “‘elite’ studies or what might be called situated studies of policy formation” (Ibid, p. 279). Interviewing such ‘elites’ creates its own challenges and constitutes what Cohen et al. (2011) refer to as sensitive research (this is something I had not considered until my ethical review – see below). They point out that where there is high profile debate and negotiation over content that is politically sensitive, it can lead to a refusal of policy makers to participate in research (Cohen et al., 2011). This last point is extremely relevant to my research, given the timing of data collection, with SDG4 still in its early days.

Lancaster (2017) suggests:

the issues associated with interviewing ‘elites’ while conducting research in a contested policy domain, especially if policy processes are being studied as they play out in real time, remain underexplored. While the broader extant literature on ‘elite’ interviewing has begun to grapple with the notions of ‘power’ and ‘vulnerability’, the question of how these notions might need to be rethought in the context of a politicised policy domain where

professional, personal and political stakes are high, remains open for examination (ibid, p. 94)

This was an important ethical consideration. During the interviews I was mindful of the politicalised context of discussions, with SDG4 in the early days of its implementation and debates over indicators continuing. I was also mindful of the ‘vulnerability’ of the participants, who as professionals need to maintain discretion and trustworthiness. The perceived ‘risk’ that their voices could somehow be identified may explain why a

number of those I approached chose not to participate.

It was my intention to ensure a balance of professional affiliations - i.e. UN, government or civil society, nationality and gender. Participation was voluntary and dependant on willingness and availability. I approached prospective interviewee via email, with a formal cover letter and information sheet about the research and the research process (see appendix B). A number of those I approached for interview did not respond (despite more than one request) while others did not have the time. Ultimately, this limited the total number of participants - my aim was to interview 10 people, in the end the total was seven. It also proved harder than anticipated to balance gender, with only 2 of the interviewees being women. On one hand, this can be seen as a limitation, but on the other it reflects how men are more often in senior positions and/or leading policy discussions. Although I am aware of the gendered nature of power, I did not focus on the gendered aspects of policy making in my research in a purposeful way, and it perhaps is a consideration for future research. 13 _{I was also cognisant of the power}

dynamics of the interviews, especially given that most interviewees were in more senior professional positions than me. In the majority of cases they were men, which also raised issues of gendered power dynamic in the interview process (although, as I discuss briefly in the section below, this was not apparent).

The participants represented 6 different countries (two were from the same country, which was in the in Global North) but only two of the interviewees were from the Global South. Again, this is a limitation of this research and something to consider in future research (see chapter 7).

13_{Given the small sample size I made a conscious decision that gender, age or ethnicity would not be part of}

The thesis overall is informed by all of the interviews conducted. I aimed to provide a balanced view and used summarised discussion points from interviewees, as well as a large number of direct quotes within the thesis. Although I numbered the interviewees for my own purposes, I decided not to include this numerical identification when presenting quotations from the research. I made this decision to minimise the risk of identification - for example, if I assigned quotations to Interviewee 1, and it is also known that this participant is female and comes from a civil society background, it could feasibly make it easier to identify that person. I made this decision based on

considerations of ethical permissions (see section 4.6 on ethics below). It is common practice to have a numerical or alphabetical identification, and it can be interesting to track a particular interviewee and see coherence in their input (although of course the quotes only represent a small part of all rich discussions). However, being able to track each interviewee makes the probability of identifying them much higher. On balance, given the initial feedback I received from the ethics committee prior to data collection, and the assurances I gave to the interviewees, I decided to leave the quotes completely anonymous.

While this may not be fully satisfactory, I have indicated how many direct quotes are drawn from the different interviews (in the last column on the right of Table 1 below). I felt this offered a constructive way to show that the direct quotes are drawn from all the interviewees. Some have fewer direct quotes, which may be due to the wording used or length of the interview.

In addition to the quotes, the data from all the interviewees was important in shaping the overall findings. The views and voices of all the interviews are reflected in this thesis.

Table 1. Interviewee sample profiles Interviewee Organisational background Region Sample criteria met Direct quotes

used in thesis14 1 International Civil Society Global South All 5 2 Non- governmental Global North All 12 3 Government Global North All 6 4 Government Global North All 9 5 International Civil Society Global North All 10

6 United Nations Global

South All 12 7 Non- governmental Global North All 12

One interesting point to note (in hindsight) is that those interviewed identified themselves with education. As a group, they were closer to the education policy process (described in chapter 5) in terms of engagement, although some were focused on the processes in New York linked to the wider 2030 policy discussions, and some in both. This can be seen as a strength in that the interviewees had strong views on education and proceeding knowledge, which for my research on education was important. On the other hand, it might also be seen as limitation and something to be considered for future research.