Data Analysis

Data analysis is the process of taking raw data and making sense out of them as evidence-based

interpretations of the phenomenon studied (Merriam, 1998; Rubin and Rubin, 2005). It is a complex

process that includes classification, comparisons, and aggregation of material to extract meanings

and implications that reveal patterns (Rubin and Rubin, 2005). Merriam (1998) identified three

different levels:

1. Descriptive - the researcher’s narrative that compresses and links the data to convey the

interpretation of what has been studied.

2. Category construction - the construction of categories or themes that represent recurring

patterns across the data.

3. Theory development - this level occurs when categories are reduced, refined and linked

Data triangulation, which is the use of multiple data sources in a study (Patton, 2002; Silverman,

2006), was carried out by triangulating the data collection methods indicated above. Data analysis

began as soon as data collection started so that emerging findings could be further explored and

themes refined. Following a data analysis plan (see Appendix 7 - Data Analysis Plan), a number of

analytical techniques were employed to robustly and rigorously interrogate the data - thematic

analysis, constant comparative analysis, cross-case synthesis, template analysis, and data display.

These are described below with explanations and examples of how they were used in this study.

1. Thematic analysis - to identify, analyse and report patterns within the data (Braun and

Clarke, 2006).

2. Constant comparative analysis - comparing one piece of data (one interview, one

statement, one theme) with others to develop conceptualizations of possible relations

between them (Silverman, 2006; Thorne, 2000).

3. Cross-case synthesis - aggregation of the findings across individual cases (Yin, 2003),

in this study, across units.

The three analytical techniques were used concomitantly while developing the units (chapters 4 to

6), which have been presented in three sections/themes:

1. Perceptions and Interpretations of ROI

2. Applications of the ROI approach

3. Challenges to applying the ROI approach

For example, ‘data collection’ was a sub-category under the theme ‘challenges in applying the ROI

approach’. Within this sub-category quotations from participants were compared within and across

the three contexts to reveal the challenges faced: access to data, quality of data and the timeliness

of data (see Appendix 8 - Sample of Quotations from Interviews with Participants (Data Collection)).

Template - a list of codes and categories representing the themes revealed from the data, which

can be pre-determined, added to and amended as data collection and analysis progresses (King,

1998; Saunders et al., 2003). Two examples are depicted in the excerpts from the QSR NVivo Software; see Figure 7 and Figure 8.

A. M O (a p p lic a tio n o f th e idea and co n ce p t o f R O I)

\ Name ^ Sources References

O ROt

Q ROI I a4i*1 it being > ! > ♦ » » Money or T«m*) ₂ ft

Q ROI M a < on* opt l getting return* VfM) 11 21

ROI as a methodology or fran‘« *o rti sb> s 14 43

Q ROI as a metric (calculating) 6 ₁₂

Q ROI links business reeds to initiative 3 6

Source. Researcher/Author

F ig u re 7: Q S R N V ivo E x c e rp t (M ain q u e stio n - a p p lica tio n o f RO I)

B. SQ1 (application of abdi's approach)

Name 3 Scwrces References

£ Q Costs 21 57 Q Caa 0 0 Q Baseline 12 20 Q Benchmark 2 2 Data Analysis 9 23 Data Cdlectjcn 30 166 Q Data Source* 20 47 Q Types of Data Source. Researcfier/Autnor 8 11

F igure 8: Q S R N V ivo E xc erp t (D ata)

Data display - matrix display (a table where the collected data is arranged for easy viewing in one

place, illustrates the contrasts and ranges of observations, and allowing for detailed and cross-case analysis (Miles and Huberman, 1994; Saunders et al., 2003)) and mapping (focus on the connection and relationship between the categories or themes, with the arrows indicating directions of influence (Henderson and Segal, 2013; Merriam, 1998)). Examples of these analytical tools are - matrix display (see Table 17: Argyris and Kaplan's Processes (All Contexts - Process I (Stage 1)) on page 239) and mapping (Figure 12: Learning and Applying the ROI Approach on page 241).

interim report was prepared for abdi Ltd that focused on the workshops observed and provided

feedback on how participants were coping with learning and applying the approach, as well as

recommendations for improvements to the approach. This was submitted to abdi in June 2013.

Research Quality

Empirical social research studies commonly use four tests to determine their quality (Saunders et

al., 2003; Yin, 2003). These are (Merriam, 1998; Yin, 2003):

1. Construct validity - establish correct operational procedures for the concepts being

studied.

2. Internal validity - establish a causal relationship, showing where certain conditions lead

to other conditions.

3. External validity - establish domain where the findings of the study can be generalised.

4. Reliability - demonstrate that the operations of the study can be repeated with the same

results.

Table 9 shows the actions taken to meet these requirements.

Table 9: Research Quality

Tests Actions Taken

Construct Validity 1. Multiple sources of evidence used (direct observation, interviews and document analyses) - triangulation

2. Established a chain of evidence (there is a clear link from the research questions, to case study template/protocol, case studies reports (with audit trail) and findings) 3. All key informants reviewed drafts of their case studies

Internal Validity 1. More than one participant from each organisation 2. Cross-case analysis completed within and across contexts

External Validity 1. Analytic generalisation in single and multiple cases

2. Case studies provide a rich, thick description so that readers can ascertain the transferability of the findings

Reliability 1. Clearly articulated documented procedures covering data collection and analytical processes

2. Case study template/protocol used for each case study 3. Interview guidelines were prepared for each interviewee

4. A case study database was created in NVivo, which was used to perform coding

3.3 Chapter Summary

Using Return on Investment (ROI) as a case in point, this study seeks to answer the research

question: How appropriate are financial metrics for evaluating human capital investments?

Constructivism/Interpretivism was employed as the research paradigm because participants’

accounts of how they applied an ROI approach were being sought. These would provide the richest

practical sources of data to allow judgements of appropriateness. Case study research was selected

as the strategy, by way of an exploratory embedded case study (i.e. a single case with sub-units) to

investigate the phenomenon (i.e. the case of the diffusion of ROI to human capital investments). The

unit of analysis is the attempt to learn, introduce and apply a ROI approach in practice in particular

sectors/contexts, with the abdi recommended ROI approach chosen as the approach. This was

executed through a qualitative study, where the methods used for data collection were observations,

interviews and document analyses. A selection of appropriate analytical tools was used to rigorously

interrogate the data and to maintain research quality. These included thematic analysis, constant