Case studies

A case study is used to understand phenomena in detail and involves collecting a great deal of information about a specific subject in context using multiple sources of evidence, especially when the boundaries between phenomena and context are not clear (Yin, 2009; Saunders, Lewis, & Thornhill, 2009). The use of case studies is most favourable in situations where the researcher needs to answer the how and why questions without controlling behavioural events, while focusing on contemporary events (Yin, 2009). The phenomena are investigated in their real life context using multiple sources of evidence such as direct observation of events, interviews of individuals participating in the events, documents and artefacts (Yin, 2009).

Case study research is the frequently-used qualitative research method in information systems research, and is appropriate for understanding the interactions between information technology-related innovations and organizational contexts. It is suited for the study of information system implementation, development and use within organisations (Myers & Avison, 2002).

Four types of case study strategies can be chosen using two separate dimensions: 1. Single case vs. multiple case, and

2. Holistic case vs. embedded case (Yin, 2009).

A single case study focuses on a unique, extreme or critical case. On the contrary, a multiple case study strategy investigates phenomena in more than one case. A multiple case study strategy is usually preferable as it allows for the generalisation of findings (Yin, 2009).

The second dimension talks about the unit of analysis. A holistic case is used when the research focuses on one unit as a whole (e.g. an organisation as a whole). In the embedded case, the researcher explores sub-units within one unit (e.g. centres and departments of one organisation).

Dooley (2002) says that case studies can be methodology or strategy. As methodology, they are used to expand and generalise theories analytically rather than to generalise theories statistically (building and testing). As strategy, they hold together multiple methods for the purpose of fulfilling all the phases of research outlined below. However, as methodology they are usually not recommended for studies owing to the following reasons:

1. There is a lack of scientific rigour.

2. They provide little basis for scientific generalisations - multiple cases can be used for generalisations.

3. They are time-consuming and laborious.

4. They are non-experimental; hence, they cannot be used to generate causal relationships (however; they can complement experiments).

Case study researchers such as Yin, (2009) have suggested techniques for organizing and conducting the research successfully, proposing six steps that should be used in order to attain methodological rigour, validity and reliability, namely:

1. Determining and defining the research questions;

2. Selecting the cases and determining data gathering and analysis techniques; 3. Preparing to collect the data;

4. Collecting data in the field;

5. Evaluating and analysing the data; 6. Preparing the report.

On discussing the generalizability from the perspective of interpretive case study research, Walshman (1995) identifies four possible types of generalization: development of concepts, generation of theory, drawing of specific implications, and contribution of rich insight. These allow explanations of particular phenomena derived from empirical interpretive research

Research Question 1 is focused on an awareness of the real world problem; case studies

with survey strategies were used. Critical literature review was conducted in the different disciplines of the study area, namely InfoSec, HCI security and UX. Once a problem was identified, it became necessary to explore the extent of the problem. Semi-structured interviews gathered preliminary data on the case site. Based on the findings of the pilot semi- structured interviews, a self-administered Internet-mediated questionnaire was developed and deployed using eSurvey pro tool. Questionnaires are good for descriptive or exploratory empirical studies as they allow the researcher to gather large amounts of information that would have been very difficult to achieve with interviews. An exploratory case study was used to evaluate the authenticity of the problem as it allows researchers to gather realistic data of the phenomenon being investigated (Creswell, 2007; Yin, 2009), and this is in line with Stage 1 and 2 of the design science method. According to Bhattacherjee, (2012), case research is a detailed inquiry of an issue in a case site over a period of time. This method is implemented in social and behavioural scientific research, to gain a detailed contextualised analysis of a social phenomenon within a site (Crinson & Leontowitsch, 2011). Data collection is done using interviews, surveys, literature reviews and heuristic evaluations. This data triangulation ensures that the data is validated. Analysed data was used for artefact building; in this case it is a framework.

In order to understand the end users’ perception/attitudes of security, their behaviour and experiences, the survey gathered information on the user’s knowledge of security threats to which they are exposed ; their awareness of security policies; their usage of security technologies; their feelings about, experiences with and behaviour towards embedded security features in their application programs.

Both open and closed questions were used as the open questions allow for capturing the feelings or attitudes underlying behaviour, while closed questions allowed respondents to choose an option closely describing them. Closed questions are usually: lists, categories, ranking, rating, quantity and matrix, depending on the type of information required. The questionnaire was pretested prior to full deployment in order to allow for redesign and convergence testing using the initial data. A cover letter explaining the purpose of the survey was broadcast to the population, together with a link to the online survey.

The survey deployment can be:

2. Direct emailing that violates the anonymity of the respondents and presents difficulty in capturing and organising the data, or

3. An anonymous online survey using tools such as Survey monkey, eSurvey Pro. The choice was an anonymous online survey. The advantages associated with this are:

1. It is fast to deploy; 2. Easy to analyse, and

3. Maintains the anonymity of the participants.

Research questions 2, 3 and 4 involve a critical analysis of usable security and user

experience components through literature surveys. USec and UX evaluation criteria for end user program security features were developed by evaluating respective metrics. The outcome was used to come up with a suggestion of a model for evaluating secure UX, which is an output of Stage 3 of design research DSR process.

The Main research question deals with the development of the artefact/framework for secure UX based on the theoretical framework of question 1 and a secure UX evaluation model from question 2.