Methods of data collection

Party positions, including those on the EU, cannot be observed directly and, as Bakker at al. (2012) argued, researchers must rely on party material or behavioural evidence to infer their stances. This evidence may be collected using a number of quantitative and qualitative methods. The most widely used methods in the comparative literature include: analysis of what parties say their positions are in programmatic documents and election manifestos (quantitative or qualitative content analysis), conducting surveys of country specialists who are asked to interpret party stances (with or without quantifying their answers), observing how parties behave in parliaments (that is, which legislative proposals they support) or interviewing politicians in order to examine their positions. The choice for a certain method of data collection is the result of the nature of the research question, the type of data needed to answer the research question and the chosen research design. This research thus explicitly draws on qualitative data due to the essentially qualitative nature of party stances on the EU as well as the research design – that is, a small-N qualitative comparative study, as discussed above. Although it uses some quantitative data, such as election results and voters’ stances on the EU, it does not employ statistical analysis other than in simple

descriptive terms. The research therefore utilises primarily a qualitative analysis of party programmatic documents and semi-structured elite interviews to collect data. Each of these methods has particular advantages, but also some drawbacks. This section first discusses the positive and negative aspects of quantitative methods and explains why they are not used in this study, followed by an examination of the qualitative methods on which the study does rely.

Party responses to Europe have been widely studied using quantitative methods of data collection, primarily by conducting expert surveys (Ray 1999; Rohrschneider and Whitefield 2007; Steenbergen and Marks 2007; Hooghe at al. 2010; Bakker at al. 2012; Benoit and Laver 2006) and a quantitative content analysis of party documents (Budge et al. 2001).

Expert surveys have been extensively used in the comparative literature. Their key advantage is that such quantitative data on party stances on the EU are generalisable across a large number of cases, and thus appropriate for theory generating. The results of expert surveys also appear to be reliable and valid (certainly more than qualitative content analysis) when cross-validated with alternative sources of information about party positions (Bakker at al. 2012). Moreover, numerous scholars who analysed the quantitative data collected by expert surveys produced a number of plausible theoretical assumptions about party responses to Europe (Hooghe at al. 2002; Marks at al. 2006; Marks and Steenbergen 2002).

However, there are a number of problems with this method and the data it generates. It is questionable to what extent it is methodologically justified to convert qualitative experts’

judgements on party stances into the quantitative information provided by these surveys. It is especially problematic that these surveys – for example, Benoit and Laver (2006), quantified expert’ responses on party stances on the EU in a form of ratio data (on a scale from 0 to 20), given that both party positions and experts’ stances on the party positions are fundamentally qualitative data. This raises the issue of whether a given quantifier has the same meaning in different national contexts? Similarly, it is not clear on what indicators experts assessed party positions on the EU, which begs the question of whether this data is essentially comparable. In other words, did country experts base their assumptions on party programmatic documents, public speeches of party leaders, interviews conducted with party officials or on their general knowledge about these parties?

Crucially, data obtained in this way do not say much about the reasons behind party stances on the EU, and neither do they provide the context that would supply the reasoning

behind parties’ adoption or change in their stances. Admittedly, scholars, using this data (Hooghe at al. 2002; Marks at al. 2006; Marks and Steenbergen 2002), did examine party ideology as a driver of their responses to Europe. They argued for party ideology as the single most important causal factor that determines party stances on the EU. However, this assumption may be the result of the chosen method of data collection. Specifically, they found it easier to quantify party ideology and thus focused on it. At the same time, they disregarded other potentially significant drivers that cannot be easily quantified and obtained through expert surveys, such as transnational party relations, relations with socio-economic groups, position in the party system or parties’ strategic considerations. In other words, scholars utilising expert surveys cannot look at the interaction between explanatory variables and only examine a limited number of quantifiable factors. As a result, they may overlook the factors that cannot be quantified and therefore may reach biased conclusions.

On the other hand, other scholars (Szczerbiak and Taggart 2008a, 2008b) used expert surveys qualitatively – that is, without attempting to quantify country experts’ judgements.

This approach seems to be more justified when assessing qualitative party responses to Europe. It allows for an assessment of more potentially significant variables (including those that are essentially non-quantifiable) as well as their interaction. As a result, a more complete general observation about party stances on this issue, including a detailed assessment of peculiar individual cases, can be achieved using this method. This research acknowledges the importance of experts’ opinions on party positioning on Europe in general. However, it does not rely on expert surveys (due to the shortcomings outlined above) and rather conducts a number of interviewees with country experts. These interviews provided very useful contextual information and, importantly, allowed for verification of the key thesis findings. The authors’ judgements are tested by interviewing country experts, but also officials of the EU and European party federations responsible for Serbian and Croatian parties.

Similarly, this research does not conduct a quantitative content analysis of party documents. This method has a number of limitations and does not appear to be suitable for discerning party responses to Europe. Due to the same reasons discussed above, extracting party positions from their programmatic documents and quantifying them does not provide any insight into motives behind party stances or the contexts in which these positions were adopted. In other words, the sole observation of a party mentioning the EU does not give us any information on the factors that are behind such stances, and it may

only reveal the salience of this issue for a party specifically during the election campaign (if party manifestos are examined). The most extensively used Manifesto Research Group/Comparative Manifestos Project (Budge et al. 2001), which does quantitative content analysis of parties’ election programmes does include Serbian and Croatian parties.

It quantitatively measures both positive and negative party references on Europe and their ideological positions across time. However, focusing exclusively on party manifestos that are adopted in the context of electoral competition does not always reveal parties’ real positions. As Ray (2007, p.16) argued, the manifesto may be advertising and parties may not be expected to adhere to every line in their manifestos; at the same time, they may be prone to an exaggeration of policy differences or a proliferation of vague or unrealistic promises. In addition, some parties tend to avoid the EU as a topic in manifestos for strategic electoral reasons or make too-general references to European integration, which is particularly the case in (potential) candidate countries. Other party programmatic documents, primarily party programmes, may uncover (if qualitatively assessed) more about parties’ positions since they are usually adopted outside electoral competition. Finally, as a result of the mechanistic and statistical approach to fundamentally qualitative data, this method does not always provide empirically reliable data. For instance, the Comparative Manifestos Project (Budge et al. 2001) identified that the hard Eurosceptic Serbian Radical Party had a positive attitude towards the EU in its 2008 manifesto, despite the party’s outright rejection of Serbian EU membership and opposition to the EU as such.

Finally, this research does not largely examine voting in parliament as a source of information. Although this may provide qualitative data and contextual information, Benoit and Laver (2006) were right to note that political systems characterised by very high levels of legislative party discipline are not suitable for this method. In both cases analysed in this study, coalition governments are the norm and members of such cabinets ‘bound together by collective cabinet responsibility are likely to vote in the same way, despite having different policy positions’ (Benoit and Laver 2006, p.106). Similarly, they argued, all members of a diverse opposition parties would most likely vote in a unified way and against the government, despite potentially significant political differences between them.

In summary, quantitative measures of party attitudes towards the EU may be misleading and may not provide insight into the real motives behind party positions. This research, therefore, employs methodological triangulation and uses three qualitative methods to gather data. These are namely:

1. qualitative analysis of party programmatic documents (party statements, election manifestos) and political speeches of representative party leaders;

2. semi-structured elite interviews;

3. analysis of the secondary sources and academic literature on the cases examined.

Both qualitative content analysis of party documents (Batory 2002, 2008a; Henderson 2008;

Szczerbiak 2008) and elite interviewing (Krašovec and Lajh 2009; Szczerbiak and Bil 2009;

Konitzer 2011) are well established and widely used in the comparative literature as a method of data collection. In order to directly examine the stances on the EU among party elites, some authors also surveyed politicians, mostly parliamentarians (Alexandre and Jardin 1996; Baker et al. 1999; Baker and Seawright 2000). However, these approaches have a number of limitations. The most important one that relates to both a qualitative content analysis and interviewing is that they are greatly subjective. In other words, they rely on the author’s judgements and assessment of party stances on the EU in their programmatic documents and public statements, as well as the author’s interpretation of the interviews conducted with senior party officials. The author has therefore attempted, as far as possible, to cross-validate these judgements by interviewing a number of country experts and officials of the EU and European transnational parties that represent authoritative sources of information, and by using the relevant secondary sources.

As stated above, the data was first gathered by a detailed qualitative examination of Serbian and Croatian political parties’ programmatic documents. The study looked at different types of documents adopted over time in order to capture the underlying party stances present in all of them as well as to address the problem with focusing only on party manifestos. It analysed the content of party programmes, manifestos, electoral campaign documents (such as brochures, posters and pamphlets), as well as public statements of senior politicians and party leaders. The statutes and other relevant documents of European transnational parties were also qualitatively analysed. The main limitation of this method was that some parties did not explicitly state their stances in the programmatic documents. That was particularly the case with party positions on the substance of the European integration and the principles underpinning the process of European integration;

however, their stances on their countries’ EU membership were much easier to detect in all examined documents. For example, the hard Eurosceptic Serbian Radical Party adopted the programme that did not contain any references to the EU, although this was a crucial feature of the party’s politics. Data collected this way was therefore complemented by

information gathered through other means, primarily elite interviewing. Interviews were also used to prevent the negative impact of the author’s subjectivity when interpreting party documents, which is the main drawback of a qualitative content analysis. Qualitative content analysis proved to be an important source of data for assessing the impact of some independent variables. Specifically, party programmatic documents were mostly used:

1. to identify parties’ ideology, and locate their positions on both the socio-economic left-right and traditionalism-cosmopolitanism axes accordingly;

2. to assess the direct effect of the linkages with European transnational parties – i.e., programmatic transformations that may have been result of requirements of European party federations.

The second key source of data was 47 interviews with political elites (26 in Serbia, 14 in Croatia, 7 in Brussels) conducted by the author. Elite interviewing as a method of collecting data has a number of advantages. Richards (1996), for example, argued that elite interviews provide information which may not be recorded elsewhere, help researchers to interpret documents and reports, and secure an insight into party internal affairs. Johnson and Reynolds (2008) noted that elite interviewing provides a more comprehensive and complicated understanding of political phenomena than other forms of data collection.

Elite interviews are particularly useful in studying political behaviour as they primarily

‘provide information on understandings, opinions, what people remember doing, attitudes, feelings and the like’ (Vromen 2010, p.258). As such, this method of data collection is therefore highly suited for this research due to its focus on parties and their leaders’

attitudes towards the EU.

However, elite interviewing also has a number of weaknesses. The first issue is the extent to which politicians’ stances can be seen as a reflection of party stances. Benoit and Laver (2006) thus argued that politicians’ perceptions of party policy positions can be intensely coloured by their private political perspectives. Szczerbiak and Taggart (2001) similarly asserted that although useful in tracing factional conflict and explaining backbench dissent, data gathered through elite interviewing does not necessarily accurately represent the party’s overall position on European integration. Anticipating this issue, the author conducted as many interviews as possible with senior party officials from the same party (in most cases, between two and four interviewees from individual parties). Moreover, the politicians interviewed have different positions within the party (e.g., party presidents or vice presidents, international secretaries, members of the presidency and political council,

and MPs), which allowed for capturing parties’ wider and underlying positions on this issue. The second potential problem with elite interviews is the reliability of interviewees and the validity of interviewee’s statements. Harrison (2001) noted that information obtained through interviews may be inaccurate and unreliable because interviewees may wish to present themselves in a positive light or due to memory lapse. Jonson and Reynolds (2008) similarly argued that interviewees may give evasive answers. This research project therefore heavily relies on content analysis of party programmatic documents as well as secondary literature to verify the data collected through interviews. In addition, data obtained through personal interviews were corroborated with statements of other interviewees in order to confirm the validity of interviewee’s statements. Moreover, respondents’ statements were also checked for internal consistency in order to arrive at valid conclusions.

Given the nature of the research and the data needed to answer the central research question, the author conducted semi-structured interviews. This type of elite interviews has two key advantages. Unlike structured interviews, semi-structured interviews do not draw on a standardized questionnaire with a formal set of questions asked in the same way and sequence, thus generating quantifiable data. Rather, semi-structured interviews allow respondents to explain their beliefs and feelings, and they primarily generate qualitative data. As Harrison (2001) noted, these interviews permit an understanding of the ‘whys’ and

‘hows’ as well as the ‘whats’ since the interviewer seeks the clarification and elaboration of answers. Thus, semi-structured interviews permit researcher to probe and focus on specific aspects of relevant issues. In other words, the author’s decision to conduct semi-structured interviews was based on the intention to grasp the interviewees’ personal attitudes and interpretation of parties’ attitudes towards Europe without constraining them through a standardised set of formal and fully structured questions.

Moreover, while unstructured interviews assume the form of ‘a free flowing conversation’

(Harrison 2001) due to the lack of any strict set of questions, during semi-structured interviews, interviewers do pose a limited number of more formal and structured questions to each interviewee. As a result, semi-structured interviews may generate comparable information, thus presenting another methodological advantage of this technique. In this study, respondents were asked the same set of key questions about their stances on the EU (see Appendix 2 for questions), allowing for a comparison of their responses. However, the author also asked a number of context-related questions important for understanding the

respondents’ specific positions on the EU. Each interviewee was therefore asked questions tailored to his/her specific role in the party or some peculiar feature of the party analysed.

For example, the researcher asked Serbian politicians questions about the recognition of Kosovo as a precondition for EU accession. Similarly, senior politicians of the Croatian Democratic Union and the Democratic Party of Serbia were probed on their parties’

peculiar relations with European transnational party federations. Finally, the author redesigned or reworded some of the questions in response to new lines of inquiry that emerged during the interviews, which is another advantage of less formalised interviews.

Selecting interviewees constitutes one of the most significant challenges of elite interviewing (Harrison 2001; Burnham at al. 2008; Jonson and Reynolds 2008). In this study, the purpose of the research dictated the list of interviewees (see Appendix 1). As the author sought to understand the motives behind party responses to Europe in Serbia and Croatia, he interviewed: (i) officials of 16 political parties (9 in Serbia and 7 in Croatia,), (ii) experts on Serbian and Croatian party politics, and (iii) officials of the EU and European transnational parties. The author primarily sought to interview senior party officials that possessed information vital for understanding party responses to the EU. The respondents were: (i) high-ranking officials that had significant influence on the general decision-making within examined parties, (ii) officials directly involved in the process of formulating (or changing) party policies towards the EU, and (iii) party officials that also served as state officials directly responsible for these countries’ relations with the EU. Thus, respondents included party presidents and vice presidents, international secretaries, spokespersons, presidents of the party parliamentary groups, MPs, and members of the party presidencies and political councils. They proved to be highly relevant informants given that some of them were also senior state officials, such as the president and vice president of the Serbian parliament, a chairman of the Serbian parliament’s European integration committee, a chair and members of the joint parliamentary committee EU-Croatia, a former state secretary of the Croatian ministry of European integration as well as former ministers of justice in Croatia and for Kosovo in Serbia. Furthermore, country experts consisted of established political analysts and authoritative commentators on political events in both countries.

Finally, officials of the EU and European transnational parties were chosen according to their close links with, and direct responsibilities for, these two countries’ EU integration and political parties.

In order to identify the most relevant interviewees, the author primarily used ‘snowball’ or

‘referral’ sampling techniques (Burnham at al. 2008). Key informants were first interviewed (such as foreign policy advisors to the Croatian president and the vice presidents of a number of Serbian parties). These key informants then provided the names of potential interviewees; the majority of these individuals volunteered this information, but others were directly asked for referrals. This technique was particularly effective in Croatia due to the relatively small number of people who constituted the political elite and the density of their interconnections. However, there was no evidence of similar interconnections within

‘referral’ sampling techniques (Burnham at al. 2008). Key informants were first interviewed (such as foreign policy advisors to the Croatian president and the vice presidents of a number of Serbian parties). These key informants then provided the names of potential interviewees; the majority of these individuals volunteered this information, but others were directly asked for referrals. This technique was particularly effective in Croatia due to the relatively small number of people who constituted the political elite and the density of their interconnections. However, there was no evidence of similar interconnections within