Measuring influence and the research process

Several individuals and organisations participate in policy-making and legislative processes, at different stages, leading to policy outcomes. They exert influence through various channels, directly or indirectly. The policy

‘matter’, first articulated by any one ‘player’ or proposed by any one civil society organisation, may take a completely different shape along the long and most of the time an arduous process and the outcome may not even look like what was originally intended. It may also take some time even before an ‘idea’ makes a political agenda item.

It should not be assumed that the most visible civil society organisations are the most influential ones in the political decision-making and legislative processes. Their relevance varies at different stages of the policy cycle:

some are more active and effective in shaping the initial agenda, some others in contributing to policy development at later stages. This is largely determined by their financial resources, the number and quality of their staff, their connections to policymakers—advisors and key bureaucrats as well as politicians—their access to media, hence their ability to reach multiple audiences.

The major premise of this research related to measurement is that; none of the indicators can provide definitive data on the amount of exposure civil society generates and how much influence they actually have in shaping public opinion—indirect influence on policy process—and the preferences and choices of policymakers—direct influence—and eventually on the policy outcomes. In this respect, it benefits from Abelson (2009), Abelson and Lindquist (2009), and McGann and Weaver (2009).

Unless policymakers themselves express that their decisions were based on recommendations from particular organisations, it is difficult to

determine the degree of such an influence, even if it does exist. On the other hand, since completely isolating the views of certain civil society

organisations from many other individuals and organisations that actively seek a role and an influence on public policy is very difficult, it is virtually impossible to assign a numerical value to the amount of influence they

wield. Examining the organisations that focus on particular policy issues—

security sector and judicial reforms, selected for this research—is a useful point of departure for studying the interaction. For assessing their

effectiveness, this focus would include their strategic goals, areas of expertise, audiences they target and the time frame in which they hope to achieve a policy influence. Since the political system constrains or facilitates access to policymakers and relevant processes, the nature and dynamics of it are also of utmost importance.

It may also be more appropriate to discuss the “relevance of think-tanks in the policy-making process than to speculate about how much policy influence they wield. In other words, […] if, when, and under what conditions they [could] have contributed to specific public policy

discussions and to the broader policy-making environment”. (Abelson 2009:

170-71) Abelson (2009) also suggests that; “scholars should also pay more attention to what policymakers think about contributions think-tanks have made at different stages of the policy making process. […] either through interviews with or through surveys distributed to policymakers throughout government” (179). This research does the former with satisfactory results.

The latter—as it has turned out—if attempted, would have been faced with and crippled by significant practical obstacles.

My unit of analysis⁶¹ is ‘participation of the civil society in the policy-making process’, a social artefact, with a view to, first describing and then explaining its influence on the policy outcomes. I examined the political decision-making, policy formulation, law-drafting, law-making/legislative processes and the participation of civil society at each step of these

intermingled series of interactions of policy-making actors. At each step, the research necessarily attempted to focus on the composition and actions of

61 The unit of analysis is clearest in the case of nomothetic, quantitative studies, but not so in qualitative research. It can be a group, a formal organization or a social artifact. Social artifacts are the products of social beings or their behavior. Social interactions form a class of social artifacts. Each social artifact implies a population of all such ‘objects’—in the case of social interactions, a cluster of them. (Babbie 2009: 101-09)

the group, the dynamics involved and the role and influence of ‘civil society’ in relation to the ultimate decision unit in that specific context.

The data required for process tracing is necessarily and overwhelmingly qualitative in nature. The main method for data collection used to answer research questions and operationalise variables was ‘intensive interview’.

The data was mainly collected from the individuals involved in the

processes under investigation, in various capacities, at each level, and then was aggregated to explain, generalise and characterise. For selecting

interviewees, a combination of various approaches to sampling proved to be the optimal method. Therefore non-probability sampling methods—actually a combination of purposive sampling and chain-referral or snowball

sampling—were utilized, based, secondarily, on ‘reputational’, ‘expertise’

and ‘availability’ criteria, focusing on individuals who were active and willing to talk and reflect. Also they were to be representative of a range of points of view—political, intellectual, institutional, cultural—in terms of their professional qualifications, experience, political and organisational affiliation and seniority.

Data collection spread over a period from September 2011 to December 2012 and developed in stages. The first stage covered primarily the four originally selected civil society organisations; TESEV, TÜSIAD, Heinrich Böll Stiftung-Turkey and CESS, i.e. their senior staff. Then, as a probe, a mixed group of individuals, who were thought to be fairly representative of the decision-making and legislative spectra as well as the judiciary, were interviewed. These included former MPs, former judges (some of whom served at the Supreme Court), selected advisors to MPs, senior staff who served in Prime Minister’s Office in the past, senior staff from central organisations of political parties—with primary responsibility for civil society, National Security Council staff and senior Parliamentary staff. This list, based on the interview results snowballed and also some other

individuals were cherry-picked as necessary, gradually expanding the emerging mental picture of the research domain, facilitating a properly focused research.

These initial stages were followed by a series of steps tracing the processes of political decision-making and legislation: relevant ministries, prime minister’s office/cabinet, political parties—party headquarters, parliamentary/committee staffers and advisors to MPs, individual MPs/members of the Parliamentary committees, leadership cadre of the political parties. Those who were from the ministries included civil-military bureaucracy—active and retired, particularly from the Ministry of Justice and the Ministry of National Defence, the General Secretariat for the National Security Council, as well as the Turkish Armed Forces. The process at the Prime Minister’s Office/Cabinet level was traced through mainly retired senior bureaucrats and former ministers. Staffers and MPs who participated in this research were mainly—but not exclusively—from the Committee on the Constitution, Justice Committee, National Defence Committee, EU Harmonization Committee and Foreign Affairs Committee as well as senior posts in the General Secretariat of the Turkish Grand National Assembly. Other individuals involved in other parliamentary committees were also consulted for getting a better grasp of the context.

In addition to the above listed range of primary group of interviewees, others from various civil society organisations (not necessarily explicitly working in the area of democratisation), foreign/transnational civil society organisations that are active in supporting civil society in Turkey, offices of international organisations and resident foreign embassies/consulates (providing financial, technical and/or political support to civil society in Turkey) were consulted.

The selection and pursuit of interviewees, at each stratum, continued until the saturation point, in other words, until I started receiving the same or very similar inputs. An approximate total of some seventy individuals were interviewed in this research. These include eight senior staff in civil society organisations, twelve senior bureaucrats (judges, military, TGNA), twelve advisors to MPs and experts at Parliamentary Committees, four senior staff in political parties, twelve MPs, five senior staff at Prime Minister’s Office and five senior staff in foreign entities that are resident in

Turkey. Some of them were revisited either for purposes of clarification and confirmation or for cross-checking some interim or provisional results.

Also some focus group discussions—involving a range of individuals from staffers to MPs—were organised to discuss some certain issues or particular aspects of the research subject which were common to more than one interviewee. These were mainly eight; four with small groups of staffers currently working in the TGNA, two with former MPs, one with former Supreme Court judges and one with the military serving in NSC Staff.⁶² Such group discussions proved extremely useful because they gave the participants the opportunity to put their own views and perceptions to test, complement other’s views, and eventually corroborate. However, while focus groups generally created a sense of confidence and encouraged free expression of facts and views thereby substantially improving both the reliability and validity of the research, they also suffered from an inherent risk—mental and/or psychological withdrawal, some participants ceasing cooperation or applying self-censorship (as one can guess, particularly in discussions with TGNA staff). Perhaps, given the current state of extreme political polarisation in all walks of life in Turkey, this was an inevitable trap due to exposure to others. Fortunately, the effect of this situation on the openness and honesty of the participants was negligible, thanks to

‘participatory’ approach, explained below at the end of this section.

Intensive interviews were based on semi-structured and/or open-ended questions⁶³, and “active interpretation” (Rudestam & Newton 1992: 34) fusing the perspective of the phenomena and that of the interpreter. I aspired to hear what interviewees “[had] to say in their own words” and see the phenomenon “as they [saw] it” (Chambliss & Schutt 2003: 30). Applying the ‘rolling interview’ technique, I relied on questions, follow-up probes and active listening, each subject leading to the next subject, to develop an

62 In the course of research in the TGNA, some unintended, spontaneous ‘group discussions’ also occurred which were extremely useful to ‘observe’ and to take note.

63 Appendix A: Questionnaire for Directors of Democratisation Programmes, Appendix B:

Questionnaire for Those Who are active in the Policy-making and/or Legislative Processes.

understanding of the interviewee’s background, attitudes and actions—and reactions to—developments and others’ attitudes and actions. In such in-depth interviews, specific content and order of questions would naturally vary from one interviewee to another. I also asked counterfactual questions to compare if the outcome would be the same, absent intervention by the civil society.

I then compared words with deeds, corroborating the earlier findings obtained from other sources/interviews, examining if, how and to what extent ‘democratic’ messages of the civil society has been received, adopted and transformed into behavioural change, and if this change in behaviour was reflective of a change in attitudes (i.e. if ‘democratic’ values were internalized) or it was the result of other factors.

In order to strengthen measurement validity and for improving reliability, i.e. for getting consistent results, I used methodological triangulation—a logical combination of interviews, observations and review of relevant documentation.

Observations included participation in parliamentary committee meetings; observation of interactions between the committee Chairs,

member MPs and staffers during and, perhaps more importantly, also out of meeting times; working habits during off-duty hours; institutional culture;

participation of civil society organisations in parliamentary committee meetings; relations between media, civil society and member MPs in committees; meetings involving legislative staffers, parliamentary advisors, bureaucrats and civil society; civil society conferences and meetings directly or indirectly related to democratisation. The observation scheme also

benefitted from a coincidental working trip—not an inherent part of this research—to the Netherlands to visit both chambers of the Dutch Parliament and Ministry of Defence. This gave the opportunity to observe

parliamentary debate and to attend one specially organised parliamentary committee session and to discuss ‘participation’ of the civil society in legislative process in this country. This visit also included meetings and discussions with a select group of civil society organisations (think-tanks)

which were actively participating in political decision-making and legislative processes in the Netherlands and with a specific interest in Turkish civil society and democratisation in Turkey in general.⁶⁴

The most salient documents used for triangulation included texts of draft bills and proposals of law, reports of parliamentary committees and sub-committees on draft texts forwarded to the Office of the Speaker of the

Parliament and of course their final versions—laws—passed by the Plenary.

I used an electronic tape recorder and notes—as ‘memory joggers’—and also maintained a daily log/summary of oral interviews. Then based on the notes taken and the views heard, I developed comprehensive notes which established the basis for the ‘findings’ of this dissertation. Sources of

information, views and documents provided directly for this research will be withhold.