SELECTING RESEARCH METHODS 1 QUANTITATIVE & QUALITATIVE RESEARCH

As explained in the previous chapter the BIS problem is too complex and ambiguous to be examined using a predefined method. Researchers describing methodological issues distinguish between qualitative and quantitative research methods. This classification“can

be a helpful umbrella for a range of issues concerned with the practice of business research

[74].” Quantitative research employs variables and measurements, whereas qualitative research does not. According to some researchers the differences go much deeper than the superficial issue of the application of quantification [75]. According to some writers quantitative and qualitative research also differs with respect to epistemological foundations [76]. Within quantitative research the principal approach is deductive in nature i.e. testing a theory with the help of quantitative data collection methods. Within qualitative research the principal orientation is inductive in nature. The objective is to generate theories. Within quantitative research the ontological orientation is objective. It has the view of social reality as an external objective reality, referred to as objectivism [76]. Objectivism is an ontological position that claims that social entities (e.g. organisations) exist in a reality that is external to, and independent of, social actors. Within qualitative research the ontological orientation is that of constructing the situation based on the details, and attempting to understand the reality behind it. This is often associated with the term constructionism or social constructionism [76]. “Constructionism follows from the epistemological orientation

of the interpretivist position to explore the subjective meanings motivating the actions of social actors in order for the researcher to be able to understand these actions” [76].

Interpretivism is the epistemology that sees the role of the researcher as part of a ‘social subject’.” The researcher observes, analyses and interprets phenomena which he or she is part of. Positivists believe in applying methods from the natural sciences to study social reality. The epistemological orientation behind quantitative research is a particular form of positivism and it involves applying quantitative methods from natural sciences models to research the subject at hand. It would be wrong to suggest that several research methods cannot be combined. On the contrary, most of the research done in Information System Science or in Information Security involves a combination of qualitative and quantitative methods. Qualitative characteristics such as explorability or complexity can be combined with ‘strong’ quantitative characteristics, such as generalizability and deductibility [77]. Figure 8 visualises the various qualitative and quantitative methodologies.

Lebek et al. [7] reveal that most academic research (including 55% percent of IS research papers) is empirical and based on quantitative methods (see Figure 8). Non-academic based research projects and publications also largely depend on quantitative data sets. One of the best known practical-oriented research institutes, Ponemon, uses large quantitative data sets to derive qualitative statements [78]. Gartner and IDC also use numerous methods to gather quantitative data to generate qualitative theoretical assumptions. To a certain extent these institutes indicate the limitations of their research outcomes and the conclusions that

they draw. For example, Ponemon raises an important limitation in the sense that they “decided to omit other important variables from analyses such as leading trends and

organisational characteristics.” Many of the ontological and epistemological issues within

the information security area arise precisely from these organisational characteristics. For example, culture, attitudes, perceptions, etc. (relational mechanisms) are hard to capture without making use of qualitative methods (e.g. interviews, case studies and expert panel research). A paper by Lebek et al. reveals that scientific researchers still tend to focus on quantitative research methods when examining non-quantitative topics, such as awareness and behaviour. Figure 9 is the result of an extensive literature study that Lebek et al. performed from 2000 to 2012 on 144 publications dealing with employees’ security awareness and behaviour. The researchers encourage the application of qualitative and interpretivist studies to explore more deeply factors such as user misbehaviour and a lack of user awareness. There is a need for more qualitative and interpretive studies in the BIS research field, as Workman et al. also found [6]. This is also acknowledged by Dhillon, who stated: “There has been little research in information systems security that can be termed

as interpretivist in nature. Generally functionalists do not even acknowledge the existence of such research efforts (ibid.). For them such approaches are ‘abstract’ and ‘too general’”

[79]. However, because of increasing dissatisfaction with the prevalent security approaches, there is a growing body of researchers who have begun to consider alternative philosophical viewpoints in their efforts to develop secure information systems.

Figure 8: Qualitative and quantitative methodologies, taken from Recker [77].

Simulation Field Experiment

Lab Experiment Longitudinal survey

Cross-sectional survey

Multiple case study Grounded theory

Phenomenology Single case study

Ethnography

Qualitative

Another relevant contribution was made by Abraham [80]. She did literature research on publications that examined intangible factors that influence user security behaviour, including the behaviour of senior management and decision-making skills [80]. Her study defined three major themes:

− Management and peer influences − Deterrence efforts or sanctions

− Rewards and the level of employee participation in security efforts within the organisation. Management and peer influence relates to the extent to which employees follow guidelines

set by management (e.g. compliance regulations) through leading by “good example” or setting the “tone at the top.” If managers do not act in accordance with their own predefined guidelines, it is likely employees will follow that behaviour [81], causing IS programmes to fail. In her study Abraham noted a lack of studies that empirically evaluated the effects of management’s use of security practices on end users’ security behaviour.

Deterrence effects or sanctions relates to the effects these measures have on IS behaviour

and IS adoption by employees. Rewarding employees can act as a motivator, creating commitment to IS. This was also found by Spurling in 1995 [82].

Rewards and the level of employee participation in security efforts in the organisation relate

to the degree of positive influence user participation can have on IS strategy formulation and implementation. If users are involved at an early stage of the IS planning process (i.e. maturing towards a desired state) [82], [83], indicate that user participation contributes to “improving security control performance through greater awareness, greater alignment

between IS security risk management and the business environment, and improved control development” [84].

Abraham examined 52 studies, studying individuals’ IS behaviour. She refers to a lack of qualitative studies that examine group interaction and behaviour using qualitative methods; “information security is a complex phenomenon and its repercussions extend

beyond individuals to groups and teams in organisations. While numerous studies have addressed end-user security behaviour, we lack studies that examine security group behaviour. Individuals can act differently in group environments [85] especially when groups are responsible for ensuring security. We identify the need for studies that examine the dynamics of security behaviour in group and team settings in organisations.” This confirms the

importance of qualitative research in this domain.

Three major studies over the last 15 years by Abraham [80], Lebek et al. [7] and Siponen [86] have examined the literature on intangible factors of MBIS success such as user awareness, management commitment, peer influences and behaviour. According to Workman, the limited amount of research in this area is restricting the IS field. Zooming

deeper into a study by Lebek et al. [7], she states that only five of the 144 studies include >500 respondents. The authors argue that “An empirical sample is relevant as long as it is

representative and generalizable. Samples consisting of students and/or IS professionals do not reflect the population of interest. With reference to internal, external and construct validities, surveying students and IS professionals is seen more critically than having a smaller sample size, as long as it represents reality.” Four publications; Siponen et al [87], Al-Omari

et al [29], Pahnila et al. [88], Hovav and D’Arcy [89] interviewed >500 respondents, who were employees, i.e. valid representatives. The remaining studies involved professionals or students as respondents. This clearly indicated the importance of qualitative research on relevant stakeholder groups in order to formulate qualitative statements.

2.2.2 ONTOLOGICAL ASPECTS OF RESEARCH

The philosophy of ontology is about the nature of reality. Ontology raises questions about the assumptions researchers have about the way the world works and the commitment to hold particular views [76]. From a research perspective it raises the main question: What is out

there to know? And how do we conduct our research to capture what we need to know. As Jan

Dietz stated in his book Enterprise Ontology “Ontology requires us to make a strict distinction

between the observing subject and the observed object.” [90]. Dietz continues: “This also puts the researcher into another obligation that of clarifying the philosophical stance taken

Grounded Theory 1% 3% Emperical Research 55% Quantitative 50% Qualitative 5% Deductive Analysis 8% Modeling 10% Experiment 11% Action Research/ Case Study 12%

with respect to this subject-object dichotomy.” The philosophy of epistemology examines

what constitutes acceptable knowledge criteria in the field of study, knowledge perspectives such as adoption, capturing, transformation and presentation. Epistemology deals with the question: “How will we know what we need to know? It also deals with the subject-object dichotomy of the researchers’ position. The position of the researcher can be viewed from an objectivist, subjectivist or constructivist standpoint. “Somewhere between the objectivist and

the subjectivist is the constructivist. Constructivists agree with the subjectivist that there is no absolute objective reality but a form of semi-objectivity reality that they call intersubjective reality.” [90] This reality is created through the process of detailing the factors of influence

(scope and context) and also the factors working behind them – intangible factors such as leadership or culture. In this research project we take the constructivist position, believing there is no absolute reality. Therefore the ontological ‘reality’ of the object is built through a continuous process of observing, analysing, negotiating and achieving social consensus among subjects.

So it’s necessary to examine and explicate the ontological aspects in order to get a better understanding of phenomena. In this case the context (e.g. technology or business opportunities) influences the organisation as well as the ‘construct’ of the organisation (a certain view of the organisation). The scope and the context of the organisation determine why an organisation requires a certain level of BIS maturity, i.e. due to regulations, stakeholder demands [91] or the need for compliance with reporting guidelines [92]. The purpose of the organisation also determines the business objectives and therefore the Information Function (IF) of the organisation. This results in a certain need to protect critical information assets. The scope of the construct therefore also determines the scope of the object. So everything that is out there to know about the organisation and its characteristics needs to be examined in order to determine the level of knowledge required in order to understand “what is out there to know?”

As mentioned earlier in this chapter, much of the successful adoption of BIS phenomena is grounded in intangible factors [93], [87]. Studying tangible methods leading to successful information security (such as ISO27001, ITIL and COBIT) is common [94], [3], but studying intangible factors (e.g. knowledge and culture) alongside tangible factors and making both explicit is not.

2.2.3 EPISTEMOLOGICAL ASPECTS OF RESEARCH

Epistemology deals with the science of knowledge. It explores aspects of knowledge management and deals with research questions such as what level of knowledge is relevant to understanding and forming opinions on a certain subject [76]. To get a better understanding of “knowing what we need to know” at boardroom level, we highlight the important epistemological aspects of BIS.

“Knowledge has been described by Davenport and Prusak as a mixture of experience, values, contextual information, and expert insight that supports an individual to evaluate and incorporate new experience and information. An individual that is able to efficiently handle both new experience and information, and apply it in different scenarios, is often described as a “knowledgeable individual.” Human knowledge, data and information altogether defines organisational knowledge, and when properly shared among organisational members, is a valuable asset which can be used to aid decision-making, improve efficiency, reduce training cost, and reduce risks due to uncertainty [9].

2.2.4 TACIT KNOWLEDGE

One concept of knowledge is tacit knowledge; this is the knowledge which is implicit in the heads of people within the organisation. It is hard to capture on a systematic way and hard to transfer via speech or in writing. Nevertheless, this form of knowledge can be very valuable during periods of change, e.g. knowledge of certain business processes or procedures, or knowledge of certain beliefs or behaviours that are typical of an organisation. It is hard to observe and pinpoint them, but these aspects clearly influence the “way people do things.” Action learning is an effective method that can be used to generate and capture tacit knowledge.

2.2.5 EXPLICIT KNOWLEDGE

Explicit knowledge is knowledge that can be articulated, coded, accessed and verbalised. It can also be transferred to others. Most forms of explicit knowledge can be stored in data stores. Included in this type of knowledge for example are methods (NIST2_{, SANS),} frameworks (COBIT3_{, ISO), and other prescribed forms of guidance. The opposite is implicit} knowledge. Implicit knowledge refers to a lack of awareness of certain knowledge [95]. Another aspect of knowledge is the way we generate knowledge among individuals and transfer it to others. This is what we refer to as knowledge sharing. Effective knowledge sharing mechanisms can help individuals to effectively share both implicit and tacit knowledge [9]. Nonaka [96] refers to “a continuous dialogue between implicit and tacit

knowledge via patters of interaction, socialisation, combination, internationalisation and externalisation”. In line with Nonaka’s dynamic theory of knowledge creation between groups

and individuals and vice versa [97], Cook and Brown suggest that organisational knowledge is created via balancing knowledge and knowing [98].

In this research project we aimed to examine aspects of the knowledge management process that are involved in generating, capturing, recording, codifying, selecting, presenting and transferring knowledge. And the aspects of knowledge management content, i.e. what is it

what we need to know in order to master a problem. In 2015 Flores published “Information

2 National Institute of Standards and Technology, NIST is an agency of the U.S. Department of Commerce. 3 Control Objectives for Information and related Technology (Source; ISACA)

security knowledge sharing in organisations: Investigating the effect of behavioural information security governance and national culture” In it he states an important

aspect of knowledge sharing within the Information Security domain: “establishment of

knowledge sharing is beneficial as the individual knowledge possessed by information security professionals is transformed into organisational knowledge and transferred to end users and other stakeholders. Flores refers to organisational learning in order to “prevent security-related information and tacit knowledge from being laid scattered throughout the organisation or preserved by information security personnel as their personal property.”

In Figure 10 the research methodology proposed by Fayolle et al. [99] is visualised from an ontological and epistemological perspective. What is visualised is that the process from ontological and epistemological knowledge is captured, analysed and presented using numerous methods and thus forms the methodology of this research project (to gain a more qualitative view of boardroom parameters).