Using computer software to analyze qualitative data:
Accounting versus other disciplines
Tjerk Budding*
VU University Amsterdam, the Netherlands
Martine Cools
Lessius - KU Leuven, Belgium
& Rotterdam School of Management, the Netherlands
DRAFT November 2008
Please do not circulate or quote without the permission of the authors
*Corresponding author:
Tjerk Budding, VU University Amsterdam, Faculty of Economics and Business Administration, ARCA, De Boelelaan 1105, room 2E-43, 1081 HV Amsterdam, The Netherlands, Tel +31/20/598.60.40, Fax +31/20/598.98.70, email: [email protected]
Martine Cools, Lessius – KU Leuven, Korte Nieuwstraat 33, 2000, Antwerpen, Belgium, Tel: +32/3/201.18.75, Fax: +32/3/201.18.42, email: [email protected]
The authors like to thank Pursey Heugens, the participants of the Accounting research seminar at Rotterdam School of Management, the EIASM conference on new directions in management accounting in Brussels, the EAA Conference in Lisbon and the ENROAC-MACORG Conference in Paris for their comments on earlier drafts of this paper.
Using computer software to analyze qualitative data:
Accounting versus other disciplines
Abstract
The aim of this paper is to investigate the use of Computer Assisted Qualitative Data Analysis Software (CAQDAS). In order to get insights in its current use and further potential for accounting researchers, we review the accounting literature in comparison to other disciplines until December 2007. Our analysis framework takes into account the diverse functions CAQDAS offers, as well as the extent of analysis activities undertaken through CAQDAS. Related to the CAQDAS functions, researchers can use CAQDAS just for organizing their data (‘code and retrieve’), or for undertaking a ‘gentle’ or ‘far-reaching’ analysis within the software package (Lee & Fielding 1995, Seale 2005). Related to analysis quality, researchers can give an indication of whether they find CAQDAS important to enhance the ‘validity’, ‘reliability’ and/or ‘theoretical sophistication’ of their work (Silverman 2005). A thorough search through the contents of ten accounting journals identified 26 published articles using CAQDAS before January 2008. Atals/ti, NUDIST-Nvivo and The Ethnograph are used most. A search for academic papers using these packages in three online databases (from their existence until December 2007) led us to identify about 1150 published papers in journals with a JCR impact score. Our initial analysis compares accounting to sociology. We find that accounting researchers make use of CAQDAS to an extensive degree, in a comparable way to what sociology researchers do. Accounting researchers especially mention the advantages of reliability and validity, appearing more apologetic about their use of qualitative data than sociology researchers. Further, the sociology journals seem to streamline more the way in which authors report about their analysis process based on CAQDAS. We conclude that readers could get better insights in the research process when authors clearly and concisely indicate what they actually did with CADQDAS. Apart from the lessons learned by comparing CAQDAS use in accounting to other disciplines, we want to make sure future CAQDAS users embark on software analysis projects with reasonable expectations: how
far-reaching (or not) the options offered, CAQDAS cannot provide a substitute for continuing to think critically about the meaning of data.
Keywords: Computer Assisted Qualitative Data Analysis Software (CAQDAS), research method, qualitative data analysis
1. Introduction
This paper investigates the use of Computer Assisted Qualitative Data Analysis Software (CAQDAS) in the accounting domain. In order to get insights in its current use and further potential, we review the accounting literature in comparison to other disciplines until December 2007 to identify the papers supported by CAQDAS. Over recent years, an increasing number of published accounting articles have used CAQDAS. Table 1 provides a chronological overview of the 26 relevant accounting publications. The first accounting article using CAQDAS was published in 1999. The main software packages used in the accounting literature are Atlas/ti and NUDIST – later Nvivo. The Ethnograph was used twice, while one study used the less known package OCP and another one TACT.
Table 1: The use of CAQDAS in the extant accounting literature1
1999 2000 2001 2002 2004 2005 2006 2007 Widener & Selto (JMAR Atlas/ti Smith & Taffler (AAAJ, OCP) Anderson- Gough, Grey & Robson (AOS Ethnograph) Malina & Selto (JMAR Atlas/ti) Lillis (AOS NUDIST) Willman, Fenton-O’Creevy, Nicholson & Soane (AOS Nvivo) Budding (MAR NUDIST) Empson (AOS NUDIST) Malina & Selto (MAR Atlas/ti) Sweeney & Pierce (AAAJ NUDIST) Abernethy, Horne, Lillis, Malina & Selto (MAR Atlas/ti) Anderson-Gough, Grey & Robson (AOS Ethnograph) Fogarty & Rogers (AOS NUDIST & Diction 5.0) Breton & Côté (AAAJ TACT) Dixon, Ritchie & Siwale (AAAJ Nvivo) Donada & Nogatchewsky (MAR Nvivo) Irvine & Gaffikin (AAAJ NUDIST) Neu, Ocampo Gomez, Graham & Heincke (AOS Atlas/ti) Sian (AOS Atlas/ti) Sweeney & Pierce (AAAJ NUDIST) Cuganesan, Boedker & Guthrie (AAAJ Nvivo) Dambrin, Lambert & Sponem (MAR Nvivo) Durocher, Fortin & Côté (AOS Nvivo) Ezzamel, Robson, Stapleton & McLean (MAR Ethnograph) Neu & Ocampo (CPA Atlas/ti) Rahaman, Everett & Neu (AAAJ Atlas/ti)
1 1 2 2 4 3 7 6
1
The articles were published in Accounting, Auditing, Accountability Journal (AAAJ), Accounting, Organizations and Society (AOS), Critical Perspectives on Accounting (CPA), Journal of Accounting Research (JAR), Journal of Management Accounting Research (JMAR) and Management Accounting Research (MAR).
The increasing popularity of CAQDAS in the accounting literature is not an isolated phenomenon. As will become clear from our review further on in this paper, CAQDAS has been used more frequently in the broader field of ‘Business and Management’, as well as in ‘Education’, ‘Nursing and Medicine’, ‘Psychology’ and ‘Social and Political Sciences’. Our aim here is to discuss the use of CAQDAS by accounting researchers by comparing it to the applications of Atlas/ti, NUDIST – Nvivo and The Ethnograph in other research disciplines. To undertake the literature review, we set up a framework focusing on the CAQDAS functions used and the research quality considerations mentioned in the publications. In terms of the CAQDAS functions, we analyze whether the authors use CAQDAS just for organizing their data (‘code and retrieve’), or whether they undertake a ‘gentle’ or even a ‘far-reaching’ analysis within the software package (Lee & Fielding 1995, Seale 2005). In terms of analysis quality, we investigate whether CAQDAS was indicated as being important for the researchers to enhance the ‘validity’, ‘reliability’ and/or ‘theoretical sophistication’ of their work (Silverman 2005).
In a recent call for modeling management accounting phenomena with greater real-world complexity, Luft & Shields (2003) emphasize the need for rigorously undertaken qualitative research. Also Scapens and Bromwich (2001) call for greater refinement in research methods used in case studies and fieldwork with more explicit attention to the validity of evidence used. This paper aims at contributing to the debate on the problems versus benefits of undertaking qualitative studies (Eisenhardt 1989, Gephart 2004, Yin 1994). Authors like Lillis & Mundy (2005) and Modell (2005) tried to find an answer in calling for multi-method approaches. Our paper aims at promoting the use of CAQDAS with realistic expectations to improve the quality of analysis when qualitative data come in, although the advantages and disadvantages of a purely manual versus a CAQDAS-sustained analysis need to be evaluated for every research study separately. CAQDAS packages offer useful tools and complement the traditional manual analysis but can never substitute for continuing to think critically about the meaning of data.
The remainder of this paper is organized as follows. Section 2 provides an overview of the analysis functions of CAQDAS, which are incorporated as the first part of our analysis framework.
Section 3 discusses how the use of CAQDAS can enhance the quality of qualitative data analysis and will also mention potential pitfalls of relying on the software. These quality issues are important for the second part of our analysis framework. Section 4 details the set-up of our literature review and analysis protocol. Section 5 discusses the use of CAQDAS in the accounting versus other disciplines. At the current stage of our analysis, we focus on the results for the sociology literature. Section 6 concludes with a discussion and advice for future research.
2. Analysis functions of CAQDAS
The use of computers for basic content analysis of text became popular in the 1960s. In literature studies, academics started to use large mainframe computers for counting the number of times particular words occurred in a text. The introduction of the personal computer in the 1980s enhanced the possibilities, leading to a steady increase of computer use for qualitative research in the social sciences from that time onwards (Fielding & Lee 1998, Seale 2000). Appendix A provides an overview of the different functions CAQDAS packages can undertake (Miles & Huberman 1994, Miles & Weitzman 1994). Appendix A also provides an overview of the different software families reflecting the functions offered by the different packages (Miles & Weitzman, 1994). While activities like writing and editing field notes, memos and reports at various stages, and searching and retrieving can be undertaken with a simple word processor, more complex qualitative data handling functions will become slow or even impossible without specific CAQDAS support. Especially sophisticated organization of the data, data linking and display, content analysis, conclusion drawing and verification, theory building and graphic mapping can be handled rather easily using the right CAQDAS packages (Miles & Huberman 1994).
The first aim of our analysis framework is to provide insights in the degree to which different research disciplines make use of the various functions of CAQDAS and to situate the current state of management accounting studies in this context. We therefore distinguish between the following three categories of CAQDAS use:
o to store information
o and for simple coding, searching along codes and retrieval of coded data. - The use of CAQDAS as a gentle analysis tool refers to
o its functions for data linking, developing categories, clusters and networks of information, preparing the road for further analysis and theory development through the investigation of the relationships between the concepts.
o its possibilities for memoing and annotation, inviting the researcher to write down his reflective commentaries and think thoroughly about the data.
o the possibilities to count the frequencies with which words, phrases or codes occur, hereby providing indications of the relative importance of concepts or the importance of certain verbal expressions by the interviewees.
o its more advanced possibilities for coding, in the sense that autocoding can be instructed, and that codes can be flexibly rearranged in the coding hierarchy. The advanced use of code and retrieve functions also facilitates data display.
- The use of CAQDAS as a far-reaching analysis tool refers to
o its use for developing conceptual maps, often aided by the graphic mapping functions of the packages
o its use for theory building by investigating relationships among concepts within the CAQDAS package.
o theory testing using the data, being the most advanced use of the code and retrieve functions. Specific queries can be inserted and the results can be nicely displayed. We also wonder which advantages and disadvantages are mentioned by the authors when describing their experience with CAQDAS. The main advantages of CAQDAS versus working with the familiar word processor for qualitative field research can be summarized in the following way:
- CAQDAS packages can assist in carrying out the mechanical tasks involved in interpretative analysis, especially in terms of the management of data material. Qualitative research processes are known for generating huge amounts of unstructured textual data, such as
interview transcripts, protocols, field notes and personal documents. Proper data management
is required to avoid ‘data overload’ (Kelle, 1995).
- CAQDAS increases the speed of handling large quantities of data. Coding will go faster, better and can be done with more precision when using CAQDAS. Also storing and retrieving relevant chunks of text will be much quicker and easier. Especially in the initial stages of analysis, the rapidity with which CAQDAS can identify patterns in large volumes of text can be useful. In this way, computer-aided analysis can reduce analysis time and consequently cut out much drudgery (Miles & Huberman 1994, Seale 2005, Tesch 1989).
- CAQDAS helps researchers demonstrate that their conclusions are based on rigorous analysis
and enhances the trust placed in research texts by readers. Helpful activities, easily executed using CAQDAS, are counting the number of times things occur as well as demonstrating that the researchers have searched for negative instances by examining the whole corpus of data rather than selecting only anecdotes supporting their interpretation. Further, CAQDAS encourages researchers to maximize the consistency of their coding and is also found helpful in relation to sampling issues, serving representativeness or theory development. The rapid retrieval, enabled by CAQDAS, can help in dealing with larger samples, so enhancing confidence with which empirical generalizations are made (Seale 2005). Still, one should not overlook that theoretical sampling in qualitative research has a different purpose from random sampling: the aim is not so much to create empirical generalizations through large representative samples, but to develop theory (Kelle & Laurie in Kelle 1995).
- Furthermore, the software can help in keeping track of emerging ideas, arguments and theoretical concepts. This is important in qualitative research, because here data analysis and theory construction are more closely interlinked than in other kinds of enquiry (Kelle 1995). - CAQDAS also facilitates team research. In collaborative projects, researchers need to agree
on the meaning of codes and assess inter-rater reliability. This matter is easier to ignore without the discipline imposed by the use of computers (Seale 2005).
Although CAQDAS packages can be of great help for researchers, they also have a number of disadvantages.
- The most important concern is that researchers risk taking a narrowly exclusive approach if they stick too much to the computer analysis when interpreting the data. In this way, they could become alienated from the data (Cannon 1998, Lee & Fielding 1995, Seale 2005). - A more practical consideration is whether the advantages of using CAQDAS outweigh the
time invested in learning to use the package, especially if the researcher would mainly use the functions of CAQDAS packages that might as well be carried out by a good word processor. Seale (2005) discusses the use of CAQDAS in the context of small data extracts: there is no point in spending time entering data if things can be worked out more quickly by hand.
In 1995, Lee & Fielding noticed that most researchers using CAQDAS confined their use to coding and retrieval of text segments, using the computer as an electronic filing cabinet. Theory building is generally done in the mind or with the aid of paper (Seale 2005). Seale (2005) discusses the attraction of applying the new analytic possibilities of several (revised) CAQDAS packages for researchers to come close to what has traditionally been the domain of quantitative research: counting, theory development and theory testing. Several CAQDAS packages support theorizing by offering the capacity to map out ideas in diagrams or conceptual networks (e.g. Atlas/ti, NVivo, or specific conceptual mapping software). ‘Links between concepts can be visually represented in a variety of forms, so that one type of link can mean ‘X causes Y’, another ‘X is associated with Y’, or ‘X loves Y’, ‘X depends on Y’, ‘X is a property of Y’ and so on. Since the network is linked to coded segments of data, instances of X and instances of Y, or instances of X where Y also occurs, or instances of Y where X is not present (and so on) can be generated by the application of Boolean search statements’ (Seale 2005, 202). For theory-testing, Seale (2005) recommends to distinguish between two broad categories of codes: heuristic and factual codes. Heuristic codes usually reflect the researcher’s theoretical concepts (indicating: this chunk of text is ‘about research method’, or ‘about validity’). They are used to retrieve segments of data which can then be further analyzed, sub-categorized etc. Factual codes represent facts (e.g. this is a quote from Seale, or this is a paper from the management
accounting literature). Rigorous hypothesis testing supported by CAQDAS needs factual coding, because it allows the researcher to test whether something occurs related to certain antecedent conditions (Seale 2005).
Coffey et al. (1996) make use of the hypertext link. It allows the analyst of data or the reader of a report to click on a highlighted word or icon, which brings them instantly to some link which has been previously made2. Instead of retrieving the relevant segment, hypertext links present the segment in its original context and further allow the analyst to attach explanations, interpretations and memos to particular links3. Certain CAQDAS packages promote hypertext links as a way to avoid the decontextualization of data, caused by simple code-and-retrieve approaches. The sophisticated options offered by these CAQDAS packages fuelled the methodological debate on the dominant grounded theory approach, its limitations, and ‘post-modern’ alternatives (Kelle 1997).
Opponents of CAQDAS fear that computer technology will be used uncritically for data analysis. Still, experienced qualitative, as well as quantitative, researchers stay aware of the need to treat computers as instruments for pursuing arguments about data, rather than limiting thought to what the computer can offer (Seale 2005).
3. Using computer software to enhance field study quality
‘Deciding to do qualitative research is not a soft option. Such research demands theoretical
sophistication and methodological rigour’ (Silverman 2005, 209).
The requirement of methodological rigor involves that qualitative researchers need to demonstrate their methodological awareness to the reader: researchers should be committed to show the procedures and evidence that have led to particular conclusions, and stay open to the possibility that conclusions may need to be revised in the light of new evidence (Seale 1999). CAQDAS can play an important role in this process, and our discussion will focus on validity and reliability, two criteria
2
cf. the use of Internet
3
The reader of the research report will then interact with the computer/CD-ROM rather than with a research report published on paper (Seale 2005).
traditionally used to judge the quality of empirical research. In addition, we will pay attention to theoretical sophistication, linking it to creativeness during the field study process.
3.1. Validity
Validity is the extent to which the data are in some sense a ‘true’ reflection of the real world (Silverman 2005). It is ‘interpreted as the extent to which an account accurately represents the social phenomena to which it refers’ (Hammersley 1990, 57). Validity deals with the way in which field researchers manage to analyze the often large volumes of qualitative data in order to reach credible results (Eisenhardt 1989). Lukka & Kasanen (1995) state that high quality case studies may produce credibly generalizable results. By paying attention to the real (business) context and to the deeper general structural relationships, a way is provided to move from isolated observations to results of a more general status. Miles and Huberman (1998) use the concepts ‘internal validity’, ‘credibility’ and ‘authenticity’ as synonyms. They emphasize that qualitative researchers should critically question whether their findings make sense and whether they are credible to the people studied as well as the readers. The assumption of the existence of an ‘objective reality’ has to be judged in the right context: reality is socially constructed, which means that some researchers find the idea of an external objective reality hard to sustain (Ryan et al. 2002). But the main message is that it is important to handle upcoming conclusions with care by engaging in a continuous process of checking, questioning and theorizing (Miles & Huberman 1998).
The validity of an explanation can become doubtful when the researcher makes no attempt to deal with contrary cases (Silverman 2005). Mehan (1979) commented on the anecdotalism he observed in ethnographic field studies. Some research reports seemed to include just a few exemplary instances of the behavior under study. Further, a majority of researchers did not mention the criteria or grounds for including certain instances and not others, which makes it hard to judge the representativeness of the instances and findings generated from them. Another observation by Mehan (1979) was that the presentation of summarized research results in tabular form did not preserve the original materials upon which the analysis was conducted. It becomes consequently impossible to enable alternative interpretations of the same materials.
Triangulation4 has been a common response to anecdotalism, and is formulated by Yin (1994, p. 92) in the following way: ‘Any finding or conclusion in a case study is likely to be much more convincing and accurate if it is based on several different sources of information, following a corroboratory mode’. Working with CAQDAS facilitates different kinds of triangulation, especially data triangulation and researcher triangulation. CAQDAS allows data triangulation by providing tools to create and manage different sorts of data, like interview transcripts and archival documents. Further, CAQDAS packages are usually also designed to facilitate researcher triangulation (Weston et al. 2001). The manual of NUDIST N5, for example, describes how NUDIST supports teamwork by three instruments (Richards 2000a). One is called ‘within-project co-operation’. It means that two or more researchers can work together on the same database (they can easily identify different bodies of data, or different categories for thinking about phenomena by having a node for each researcher and coding what they contribute at their node, storing their ideas in a memo at their node, etc.). A second sort of collaboration allows ‘merging projects’, whereby two or more NUDIST projects can be easily merged. The third option is referred to as ‘viewing a project in no-save mode’. It implies that one researcher can ‘save a project in no-save mode’, so that it can be opened, explored and discussed by colleagues working with a no-save viewer.
Silverman (2005) calls for prudence in terms of overly relying on triangulation in order to increase validity. He points at the difficulties for novice researchers and the analytical limitations. Instead, also in the tradition of Yin, he promotes a number of interrelated techniques that press researchers for high-quality research and force them to continue to think critically about their qualitative data:
- The data analysis should be demonstrated to be comprehensive, relying on all relevant evidence. An analytic strategy to deal with this issue is constantly comparing data: attempting
4
Most authors (Eisenhardt 1989, Ferreira and Merchant 1992, Miles and Huberman 1998, Ryan et al. 2002) use Denzin’s (1978) terminology of the different kinds of triangulation: ‘data triangulation’ refers to collecting multiple sources of evidence on a particular issue; ‘method triangulation’ means that different research methods (e.g. observations, interviewing) can be used to collect data; ‘researcher triangulation’ refers to the use of several researchers. Working in teams can prevent an individual researcher’s bias from being brought into the study; ‘theory triangulation’ means that alternative theories are used to study a specific case.
to find another case through which to test out a provisional hypothesis, until all parts of the data are inspected and analyzed (Yin 1994, Silverman 2005, Tesch 1989).
- This involves that the analysis should include all major rival interpretations. The researcher should seek to refute his initial assumptions about his data in order to achieve objectivity, and should actively seek out and address anomalies or deviant cases (Yin 1994, Silverman 2005). - Where appropriate, qualitative researchers can use tabulations: ‘Simple counting techniques,
theoretically derived and ideally based on members’ own categories, can offer a means to survey the whole corpus of data ordinarily lost in intensive, qualitative research. Instead of taking the researcher’s word for it, the reader has a chance to gain a sense of the flavor of the data as a whole. In turn, researchers are able to test and revise their generalizations, removing nagging doubts about the accuracy of their impressions about the data’ (Silverman 2005 p. 220).
While we acknowledge that validity ultimately requires researchers to expose their findings to the confirmation of outsiders, CAQDAS offers a number of ways to improve the ‘content’ aspect of validity. The data management functions of CAQDAS packages enable researchers to take into account all case material in the analysis. CAQDAS provides instruments to search in the complete data set for specific words, codes, phrases etc. and makes it easy to go back to the complete database when a specific research step needs to be carried out or repeated. It means that the software provides evidence that no occurrence has been overlooked. In addition, CAQDAS can help to handle large volumes of data while it still allows the researcher to stay focused. It provides facilities to search systematically and thoroughly for occurrences of codes (Kelle & Laurie 1995). The easily manageable search and retrieve instruments allow a complete analysis that can always be refined (Tesch 1989). Also counting and the presentation of the counted numbers in tables become easy tasks using CAQDAS.
The newest options offered by CAQDAS packages, such as hypertext links, can clearly address the problem of anecdotalism: presenting a research report to the reader in a CD-ROM format will convey a lot of information and will allow the ‘readers’ to fully examine the corpus of data on
which conclusions are based. One should however not ignore that this leads to an exhaustive task for the readers of the report: ‘it is likely that the single-paragraph abstract of findings will retain a greater appeal for readers concerned to survey a range of studies in order to extract the major research findings of a particular field’ (Seale 2005, 206).
The possibilities CAQDAS offers for complete and refined research on large volumes of different sorts of data, has disadvantages. Kelle & Laurie (1995) observe a paradox concerning computer use in qualitative analysis: while the technology facilitates the clerical handling of voluminous textual material, the analysis process does not necessarily become quicker or easier. The sheer volume of information that can be taken into account when using a CAQDAS package can become overwhelming for researchers. An important aspect is that the potential benefits of a larger sample size may be outweighed by the extra costs in time and effort required for data preparation and data entry. In the context of technically easy data retrieval and coding instruments, researchers should be careful not to develop a ‘fascination with volume’. Instead, the decisions about which and how much data to use should depend on the research aims (Seidel 1991).
3.2. Reliability
Reliability is the extent to which evidence is independent of a person using it. This expression implies that the investigator has to adopt an independent and impersonal point of view (Yin 1994). Reliability refers to consistency of the research process and to its reasonable stability over time and across researchers and methods. Miles & Huberman (1998) refer to reliability as a synonym for dependability or auditability. They stress the importance of ‘quality control’ by investigating whether the analysis process has been executed with reasonable care. Some researchers, as Ryan et al. (2002), remark that the focus on reliability is difficult to realize and even undesirable for field studies, because the interpretations of the researcher and his/her relation to the subject matter are essential elements of the explanations of the case. They therefore prefer the term ‘procedural reliability’ (Ryan et al. 2002), whereby they insist on having all evidence recorded in a coherent set of field notes and fully documenting the case analysis. In other words, the researcher is asked ‘to leave an audit trail’ (cf. Miles and Huberman 1998).
In the context of improving the reliability of qualitative studies, a number of researchers are calling for ‘low-inference descriptors’: detailed data presentations which make minimal inferences are always preferable to researchers’ presentation of their own (high-inference) summaries of their data. This means researchers are stimulated to include verbatim accounts of what people say rather than present researchers’ reconstructions of the general sense of what a person said (Seale 1999, Silverman 2005). This request is closely related to Mehan’s (1979) unease with presentations of summarized research results in tabular form – although his stress was on the problematic consequences in terms of validity.
Using CAQDAS allows researchers to demonstrate that their conclusions are based on consistent analysis:
- CAQDAS packages enable a thorough and full documentation of the analysis process. They encourage systematization and largely are self-documenting (Fielding & Lee 1998). NUDIST, for instance, registers whether codes have been merged with other nodes, have been moved within the hierarchical tree of nodes, etc. (Richards 2000a). In addition, CAQDAS packages stimulate researchers to keep track of emerging ideas, arguments and theoretical concepts, by offering electronic memoing facilities.
- A related fact is that CAQDAS forces users to make explicit choices during the analysis process. In manual analyses, such decisions often remain implicit. The most obvious example is that the computerized system requires the researcher to make clear from the start which items he is interested in (Fielding & Lee 1998, Tesch 1989). Miles & Huberman (998) stress the importance of undertaking research that is relatively neutral and reasonably free from unacknowledged researcher biases. By asking for clear instructions, CAQDAS forces researchers to be explicit about inevitable biases.
- By making analysis choices explicit and by thoroughly documenting the research process, CAQDAS allows researchers to follow a consistent and systematic research path, and to increase the stability of the research over time, across researchers and methods (Fielding & Lee 1998, Tesch 1989).
The prospect of being able to work with CAQDAS also has a major disadvantage. In general, field studies should be sufficiently prepared in terms of the field study design and the analysis protocol (Yin 1994, Miles & Huberman 1998, Ryan et al. 2002). Although CAQDAS encourages researchers to pay attention to these issues, a number of CAQDAS researchers warn for the following potential danger. The prospect of using the software could tempt researchers to prepare insufficiently, because they quickly want to start working with the package. Insufficient preparation could lead them to base the analysis on a kind of ‘trial and error’ procedures when going through the case material, instead of basing the analysis on a thoroughly prepared research design (Russell & Gregory 1993, Fielding & Lee 1998). These potential pitfalls of case study research cannot be prevented by the CAQDAS package itself. Instead, avoiding theme remains to be the researchers’ own responsibility.
3.3. Theoretical sophistication and creativeness
Apart from discussing validity and reliability, theoretical sophistication deserves to be discussed when evaluating the potential of CAQDAS for management accounting field research. ‘Good-quality research thinks theoretically through and with the data’ (Silverman 2005, 242), and therefore the theorizing process and the creativeness this requires, are important topics.
Working with CAQDAS has the following advantages:
- As the presentation of the analysis functions offered by CAQDAS indicates, it is clear that CAQDAS can take over a lot of – often dull – routine tasks. CAQDAS allows, for example, a quick and easy search through the complete database. Consequently, the researcher can concentrate on essential design and analysis tasks (Tesch 1989). An example of traceable and confirmable research procedures are the Node Searches in NUDIST (Richards 2000b). The facility ‘Node Search’ consists of a set of tools for automatically finding relationships between the coding of two or more nodes. Examples are ‘union’ (creating a new node merging the coding under two previously separate nodes), ‘overlap’, ‘intersection’ and ‘followed by’. In the context of such facilities, CAQDAS packages make it easy to re-ask the same question
about something different and allow storing the answer to a question as coding at a new node, which can be used for further investigation.
- CAQDAS packages keep track of emerging ideas, arguments and theoretical concepts. Depending on the specific software package used, the researcher can add his comments in separate memos that are easily retrievable. Such options stimulate the researcher to keep on critically evaluating his data during the theorizing process.
- CAQDAS at the same time enables analytic reassessment of the research, because it permits a far-reaching flexibility and revision of analysis procedures (Tesch 1989).
- The newest features offered by CAQDAS packages include hypertext links. Apart from dealing with the traditional validity and related reliability concerns, since they make the link between the original data and the final presentation extremely explicit, working with hypertext links can also lead to useful and creative theorizing approaches: researchers can benefit from the possibility to intimately link data, analysis and interpretation (Seale 2005).
A disadvantage of using CAQDAS is that researchers – busy with the CAQDAS package and learning about all possible/new options - could be tempted to stick to their initial coding scheme, and forget to relate other interesting phenomena to the analysis. It means that they could take a narrow approach to the data (Eisenhardt 1989). While CAQDAS has to potential to aggravate this potential pitfall, the software packages all offer instruments to overcome this problem. The memoing instruments have been discussed, and there is also the possibility to add ‘free nodes’. These nodes, indicating previously unmentioned issues, can be added after the initial hierarchy of codes has been set up. Free nodes are useful to capture new themes that emerge during the analysis process and can help the researcher to develop a new perspective on the research topic. The opposite danger is that case study researchers end up with a theory that is overly complex (Eisenhardt 1989). This is a problem that cannot be avoided by the use of a software package, and of which all qualitative researchers should be aware at any stage of their analysis. To conclude, the discussion of the three quality criteria is summarized in Table 2.
Table 2: The impact of using CAQDAS for the quality of qualitative research
Quality criteria
VALIDITY RELIABILITY CREATIVENESS CAQDAS advantages Triangulation (data, researcher) Completeness and refinement Counting techniques
Ease to retrieve low-inference descriptors
Thorough and full documentation of the research process
Analytic choices made explicit Stable research path over time, across researchers and methods
Concentration on essential design and analysis tasks
Keeping track of emerging ideas, arguments, theoretical concepts, enabling analytic reassessment of the research
Flexibility and revising options Intimate link between data, analysis and interpretation (e.g. hyperlinks) CAQDAS
disadvantages
Danger for researchers to become overwhelmed by large sample size: need for careful selection of data and careful examination of extra costs in terms of time/effort required for data preparation/entry
Researchers potentially tempted to prepare insufficiently in their eagerness to embark on using the CAQDAS package
Danger that researchers have a narrow view on the data
Too many nodes might end up in an overly complex theory
4. Approach
The previous two sections reviewed the research method literature related to our analysis framework: we will pay attention to the functions offered by CAQDAS and crucial quality issues to review the CAQDAS-based qualitative literature. This framework guides our analysis of the extent to which CAQDAS packages are actually used in different research disciplines. In this section, we explain the set-up of the literature review and the analysis protocol.
The identification of the accounting articles using CAQDAS was different from the selection of the publications in other disciplines. To identify the relevant accounting publications, we searched through the following journals for the period January 1994 till December 2007: 1. Accounting, Auditing, Accountability Journal (AAAJ), 2. Journal of Accounting Research (JAR), 3. Journal of Accounting & Economics (JAE), 4. The Accounting Review (TAR), 5. Contemporary Accounting Research (CAR), 6. Journal of Management Accounting Research (JMAR), 7) Accounting, Organizations and Society (AOS), 8. Management Accounting Research (MAR), 9. The European Accounting Review (EAR) and 10. Critical Perspectives on Accounting (CPA). Table 1 provides an overview of the articles actually using CAQDAS to perform their analysis and indicates the packages involved. We exclude articles referring to CAQDAS in general without applying it to a specific
research project5. As Table 1 shows, the number of papers relying on CAQDAS is overall growing. Only in five of the ten selected journals (9 in AOS, 8 in AAAJ, 1 in CPA, 2 in JMAR and 6 in MAR) papers have used CAQDAS, which is not a coincidence given that these are the journals publishing most of the field research in accounting.
Given the large amount of hits encountered after a first search in the databases, we used a stricter procedure to select the published articles from other disciplines. First, we confined our search to three databases that have an extensive content of scientific papers: 1) Wiley Interscience, 2) ABI Inform and 3) Sciencedirect. We searched all scientific publications in these databases from their existence until December 2007 for their use of Atlas/ti, NUDIST – Nvivo and The Ethnograph, the packages most frequently used in accounting. Our search was therefore directed by the following keywords:
- CAQDAS; Computer assisted qualitative data analysis software - NUDIST software; nudist
- N-Vivo; N-VIVO; NVivo - Atlas/ti; atlas.ti; atlasti - Ethnograph
Searching the full document text produced approximately 1.700 papers. In order to focus on the highest quality publications, we selected only those papers that were published in journals mentioned in the Journal Citation Report (JCR) of the ISI Web of Science. To determine the disciplines in which the CAQDAS papers were published as well as to know the ranking of the journals, we used the categorization of disciplines and the ranking of journals as indicated by the JCR Social Science Edition (SSE) and the JCR Science Edition (SCE). Both the SCE and the SSE distinguish a rather great number of categories that express the (sub)discipline of journals. In order to get a better overview, we constructed 31 groups of disciplines as indicated in the table in Appendix B. For 21 of these groups we found papers that reported the use of CAQDAS.
5
The accounting article by Irvine and Gaffiking (2006) was included because it discusses methodological issues based on a concrete research project.
Several checks were undertaken in order to make sure that only relevant publications were added to our database, e.g. by removing the papers that were dealing with ‘nudists’ or referred to the medical term ‘nvivo’. Further, we checked to make sure that every paper was only included once, so more hits per paper were excluded. Given that we only analyzed the papers in the journals that were ranked in the JCR list, our final research sample contained 1150 publications. However, some journals belong to several discipline groups.
We are currently analyzing the social science papers that will be used for more in-depth analysis. In order to determine an appropriate research sample, we use a stratified sampling method. Step one consists of including only the discipline groups providing ten or more relevant publications. Step two consists of randomly selecting a number of papers. This number depends on the total number of papers found per group: for the groups having less than 50 papers we selected 10 papers randomly, for the groups with 50 to 100 papers we selected 20 papers, and for the groups with more than 100 papers we keep 30 papers in our sample.
The selected papers are categorized (along discipline group, CAQDAS package used etc.) in SPSS. For the literature review and analysis, we use NVivo7. This CAQDAS package allows us to manage the large number of papers involved, fastens our coding and retrieval processes and facilitates the display of the results. CAQDAS ensures our analysis takes into account the complete coding scheme. Further, it facilitates the researcher triangulation we are able to benefit from. Both researchers compared and discussed the coding of the first three accounting papers, before the rest of the accounting papers were analyzed. After this was done, both researchers again compared their coding and made sure they came to an agreement about every aspect of the coding scheme. Based on this experience, coding of the papers from other disciplines can be divided among both researchers.
During several coding rounds the original coding scheme, heavily based on the research methods literature discussed above, has been refined. The resulting coding scheme is presented in Appendix C.
5. Analysis of field research using CAQDAS
This section presents the preliminary outcomes of the analysis. In the first place, we give an overview of the use of CAQDAS in the different research disciplines excluding the accounting papers reviewed. Second, we compare Accounting and Sociology in terms of our framework on the functions of CAQDAS used and the quality items.
5.1. A descriptive analysis of the use of CAQDAS in the academic literature
Table 3 provides a chronological overview of the academic articles found through the selection procedure described above. The table reveals a remarkable increase in the use of CAQDAS based on the packages Atlas/ti, NUDIST and Nvivo, and the Ethnograph over the last decade. The discipline groups that stand out in terms of published papers using these packages are Nursing and Medicine (38% of the total number of CAQDAS papers), Social and Political Sciences (19%), Business and Management excluding Accounting (10%), Psychology (9%), Computer science (5%), Education (4%) and Information Science and Library Science (3%).
Table 4 describes the use of the different CAQDAS packages in the literature. Nvivo is the most widely used package (used in 47% of the selected papers), closely followed by Atlas/ti (38%). While Accounting papers have only recently shifted towards using Nvivo instead of NUDIST, the use of NUDIST in the other disciplines is relatively low (8%). The Ethnograph is only used in 7% of the selected papers. The dominance of Atlas/ti and Nvivo seems to be present in most of the science domains.
Table 3: The use of Atlas/ti, NUDIST – Nvivo and The Ethnograph over time
1991 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Total %
Anthropology 2 1 2 1 2 1 4 13 1.13%
Business & Man. 1 2 2 3 1 7 12 12 10 12 12 18 26 119 10.35%
Education 1 2 1 1 1 3 1 7 7 12 13 49 4.26%
Nursing and Medicine 1 2 1 1 2 2 8 5 6 7 13 21 45 71 104 147 436 37.91%
Public Administration 1 1 1 2 5 10 0.87%
Psychology 5 1 4 1 6 3 10 23 14 32 99 8.61%
Social and political
sciences 2 1 9 1 6 5 6 14 18 35 50 74 221 19.22% Geography related studies 3 1 8 12 1.04% Communication 1 3 1 5 0.43% Law 1 1 1 3 0.26% Environmental studies 1 1 1 2 2 2 3 4 9 25 2.17% Ethics 1 1 1 3 4 10 0.87% History 1 1 0.09%
Information Science &
Library science 1 1 2 2 5 8 3 15 37 3.22% Planning & development 1 3 1 2 4 5 16 1.39% Biology 1 3 4 1 5 14 1.22% Engineering 1 3 1 4 9 0.78% Computer Science 1 1 2 1 3 7 12 8 25 60 5.22% Agricultural 1 1 0.09% Geology / Physical Geography 4 1 5 0.43% Miscellaneous 1 3 1 5 0.43% Total 1 3 9 4 3 4 29 9 32 30 50 61 113 187 234 380 1150 100.00%
Table 4: The use of Atlas/ti, NUDIST – Nvivo and The Ethnograph in the academic literature
Atlas/ti Ethnograph NUDIST Nvivo Total
Anthropology 6 3 0 4 13
Business & Man. 32 15 27 42 116
Education 13 4 4 27 48 Nursing and Medicine 180 28 17 209 434 Public Administration 3 3 1 3 10 Psychology 46 7 5 40 98 Social and political sciences 90 11 21 98 220 Geography related studies 4 0 0 8 12 Communication 1 2 0 2 5 Law 1 0 1 1 3 Environmental studies 9 0 4 12 25 Ethics 2 1 3 4 10 History 0 0 1 0 1 Information Science & Library science 10 1 4 22 37 Planning & development 4 0 1 11 16 Biology 3 0 0 11 14 Engineering 4 0 0 5 9 Computer Science 20 0 3 35 58 Agricultural 0 0 0 0 0 Geology / Physical Geography 4 0 0 1 5 Miscellaneous 2 0 1 2 5 Total 434 75 93 537 1139 Percentages 38% 7% 8% 47% 100%
5.2. A first look at Accounting versus Sociology
As the tables above reveal, our analysis is not only aiming at comparing Accounting with Sociology, but our goal is also to place Accounting CAQDAS use in the context of other research disciplines. The analysis is still in full process. In order to avoid providing an incomplete picture by comparing all disciplines at this stage, we here present our first insights about the use of CAQDAS in Accounting as compared to its use in Sociology, one of the fields known for a high level of qualitative
research. Appendix D lists all Accounting papers indicating the use of CAQDAS. Appendix E gives an overview of the Sociology articles included so far in this analysis. Note that the sociology articles in Appendix E have been found by searching the databases until April 2005 only, for the following key words: CAQDAS; Computer assisted qualitative data analysis software; NUDIST software; nudist; N-Vivo - N-VIVO; Atlas/ti. The analysis of the more recent papers and of the papers using ‘The Ethnograph’ is in full process at this moment.
Table 5 gives an overview of the number of papers discussed. As indicated in the section above, the selection of the sociology papers (only articles published in journals with a JCR impact score) has been more severe than for the accounting papers. In addition, the Sociology papers included date until April 2005.
Table 5: Articles using CAQDAS in Accounting and Sociology along year of publication
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Total
Accounting 0 1 1 2 2 0 4 3 7 6 26
Sociology 1 0 1 3 5 4 7 9 0 0 30
Table 6 details the different packages used in Accounting and Sociology along our original search criteria. All Accounting papers are included in this analysis. The Sociology papers brought in here do not reflect the complete sample, since they include only the Atlas/ti and NUDIST –Nvivo papers until April 2005.
Table 6: Use of different packages in Accounting versus Sociology according to the original search criteria
Ethnograph
Nudist Nvivo Atlas.ti TACT
NUDIST and
Diction 5 OCP
Accounting 3 6 7 8 1 1 1
Sociology Not yet included 6 0 24 0 0 0
The first part of our analysis framework guides us towards the extent of CAQDAS use in the different disciplines. Table 7 summarizes the use of CAQDAS, as it has been reported by the authors in the published article. Three categories are distinguished:
- the use of CAQDAS as code and retrieve (a data storage) tool - the use of CAQDAS as a ‘gentle analysis tool’
- the use of CAQDAS as a far-reaching analysis tool: it means CAQDAS is used for developing conceptual maps, and for building and testing theory using the data.
Table 7: Degree of CAQDAS use in Accounting versus Sociology
code and retrieve gentle analysis far reaching analysis
Accounting 19 19 7
Sociology 4 17 4
In contrast to what we had expected beforehand, the number of Accounting papers making advanced use of the theorizing tools of CAQDAS does not seem to have to underdo for the Sociology papers. Most far-reaching in their use of CAQDAS are Malina & Selto (JMAR 2001), Malina & Selto (MAR 2004) and Abernethy et al. (MAR 2005). These authors rely a lot more on CAQDAS than any of the Sociology papers involved in the preliminary analysis.
Table 8: Different techniques used in CAQDAS in Accounting versus Sociology
Accounting Sociology
autocoding in CAQDAS 0 1
constructing conceptual maps 2 1
content analysis 6 4
frequency counts 9 3
creating categories for further analysis and investigating inter
relationships 4 10
memoing in CAQDAS 0 3
purposive coding in CAQDAS 0 1
theory development 3 2
theory testing 2 2
Table 8 provides an overview of the different techniques used within CAQDAS. Both fields have a different emphasis. Accounting papers relatively make more use CAQDAS for content analysis and frequency counts, while the Sociology papers indicate more often to use CAQDAS to create categories for further analysis and to investigate the inter-relationships between these categories. A large number of these papers indicate the use of CAQDAS in the context of their grounded theory approach. In future research, we will link the use of CAQDAS to the intended goal of the paper in terms of description, theory development or theory testing.
Table 9: Advantages of using CAQDAS, mentioned in Accounting and Sociology
Accounting Sociology
ease to go back to context 2 3
facilitate teamwork 7 9
increase speed 2 1
use CAQDAS retrievals for data display 8 10
ascertain prevalence and substance of a
theme 1 2
measuring intercoder reliability 3 1
rigor of analysis 6 3
Table 9 presents the major advantages mentioned in the two fields when using CAQDAS. The retrieval of data for display and the facilitation of teamwork are indicated as appreciated features when using CAQDAS. In addition, the Accounting authors seem more concerned about the reliability of their analysis, expressed by using CAQDAS. They aim at proving that their work is undertaken in a strict way. When reading the CAQDAS-based papers, we observe that the Sociology papers account for the use of CAQDAS in a different way than the Accounting papers. The Sociology papers seem to follow a more standardized, predictable way to discuss their use of the CAQDAS functions. The references to CAQDAS in the Accounting papers are made in a much more loose and diverse way. Accounting papers go from the extreme of providing detailed information on the use of CAQDAS in Journals like AAAJ and MAR to the other extreme of only mentioning the use of CAQDAS in a footnote (e.g. Sian 2006 in AOS). When further refining our analysis in the future, we want to incorporate how much attention is paid in each paper to explaining the use of CAQDAS.
5.3. Detailed analysis of the Accounting papers
In this paragraph we analyze the Accounting papers in more detail along the coding scheme (Appendix C). The following tables represent how the Accounting papers report about the degree of analysis they undertake using CAQDAS packages.
Table 10: Degree of analysis
Code and retrieve Gentle analysis Far-reaching analysis
Most papers stick to the basic code and retrieve functions, and recur to gentle analysis methods. Almost 3% of the Accounting papers use CAQDAS in a more advanced way. The table below shows the ‘gentle’ or soft use of CAQDAS these Accounting papers are actually mentioning.
Table 11: Different techniques used in CAQDAS in Accounting – Gentle analysis
Creating categories Frequency counts Autocoding in CAQDAS Memoing in CAQDAS Query functions used Accounting 6 9 0 0 9
Most papers make use of CAQDAS for frequency counts. While in some cases words and phrases are counted, other papers count the number of times a certain node has appeared (as we actually apply in this paper). The reference to query functions indicates that quite some authors use CADQAS’ query functions and use the resulting retrieval to fill in data matrices, like the thematic conceptual matrices suggested by Miles & Huberman (1998). Further, the use of CAQDAS to refine conceptual categories has been mentioned in several papers. We now look at what authors indicate in their papers when they use CAQDAS in an advanced way.
Table 12: Different techniques used in CAQDAS in Accounting – Far reaching analysis
Theory development Investigating interrelationships Theory testing calculation of probabilities that a word collocates with another CAQDAS identifying semantic tonalities of passages of text Accounting 5 6 2 1 1
Five papers indicated the use of CAQDAS for theory development, and all of them referred to the far-reaching use of the computer to investigate interrelationships between concepts. Only two papers indicated the use of CAQDAS for theory testing. Interestingly, one paper indicated the use of the package to calculate the probability that a word collocates with another. And one paper let the package DICTION automatically identify the semantic tonalities of passages of text.
Table 13: Advantages of using CAQDAS, mentioned in Accounting
Flexibility 4
Ease to go back to context 2
Facilitate teamwork 3
Measure intercoder
reliability 3
Use CAQDAS retrievals for data
display 2
Justification
Ascertain prevalence and
substance of a theme 1
Negative/contrary cases 1
Prove rigor of analysis 6
Prove validity 4
Leave audit trail 3
Increase speed 3
Table 13 displays the advantages the authors of the papers mentioned about the use of CAQDAS. It shows that the researchers especially appreciate CAQDAS for its flexibility and to prove its rigor as well as the validity of the analysis. The table below indicates that few accounting papers give information about the experienced disadvantages when using CAQDAS for their analysis.
Table 14: Disadvantages of using CAQDAS, mentioned in Accounting
Time spent Narrow analysis Too many categories
Accounting 0 1 0
Only one paper (Abernethy et al 2005) mentioned the possibility to end up with a narrow analysis when recurring too much to the software and forgetting to analyze the data yourself.
Table 15: Information processed in CAQDAS accounting papers
Interviews Other documents Newspaper articles
Field notes
Accounting 22 7 1 2
Table 15 gives an overview of the type of information processed in CAQDAS accounting papers. About 85% of the papers use CAQDAS with interview transcripts. Other documents (like annual reports, chairmen’s statements etc.) are mentioned, and one paper analyses newspaper articles
using CAQDAS. Table 16 displays the amount of detail provided in the paper about the use of CAQDAS.
Table 16: CAQDAS detail
Accounting
In footnote only 1
In research method section only 16
In analysis section only 1
In appendix only 0
In results section only 0
In discussion section only 0
In methods section and conclusion 0
In research method and footnote 3
In research method, results, analysis, discussion section, and foodnote
1
In research method and discussion 1
In research method, footnotes,
appendix, and results 1
In research method, footnotes, and
results 1
In research method, discussion,
and footnotes 1
It shows that most papers report the use of CAQDAS in the research method section, 24 out of the 26 papers explain the way they used CAQDAS in this section. 8 of these 22 papers also pay attention to CAQDAS in another part of the paper. In one paper, the author only reports the use of CAQDAS in a footnote. In another paper, the use of CAQDAS is discussed in the analysis section.
6. Preliminary discussion and conclusions
The purpose of this paper is to provide an overview of the use of CAQDAS to analyze qualitative accounting data: are we in accounting benefiting from CAQDAS to its full potential? In order to find an answer we are reviewing the accounting papers referring to CAQDAS use in comparison to the use of CAQDAS in other research disciplines. Our analysis is guided by a framework focusing on the CAQDAS functions used and the research quality considerations mentioned in the publications. Related to the CAQDAS functions, researchers can use CAQDAS just for organizing their data (‘code and retrieve’), or for undertaking a ‘gentle’ or ‘far-reaching’ analysis
within the software package (Lee & Fielding 1995, Seale 2005). Related to analysis quality, researchers can give an indication of whether they find CAQDAS important to enhance the ‘validity’, ‘reliability’ and/or ‘theoretical sophistication’ of their work (Silverman 2005). A thorough search through the contents of ten accounting journals identified 26 published articles using CAQDAS before January 2008. Atals/ti, NUDIST-Nvivo and The Ethnograph are used most. A search for academic papers using these packages in three online databases (from their existence until December 2007) led us to identify about 1150 published papers in journals with a JCR impact score. A sample of each discipline containing more than ten CAQDAS papers is currently being analyzed based on our coding protocol.
What can we conclude from our preliminary analysis comparing all Accounting papers using CAQDAS with a limited set of Sociology CAQDAS papers? Accounting – and especially management accounting - researchers use a wide variety of CAQDAS functions, so Accounting does not seem to underdo for Sociology. Authors like Abernethy et al. (2005) and Malina and Selto (2001, 2004) make use of the most advanced functions of ATLAS/ti for theory development and testing, while most researchers stick to the code-and-retrieve and the more gentle analysis options offered by CAQDAS. The advantages of using CAQDAS at all degrees are clear: researcher as well as data triangulation becomes easier. Further, ‘low-inference descriptors’ (Seale 1999, Silverman 2005) can be easily traced back. Since CAQDAS requires researchers to make explicit choices and in this way stimulates researchers towards an explicitly systematic and rigorous analysis, researchers can enhance the trust of readers in the validity and reliability of the final conclusions.
That CAQDAS use proves the rigor of the analysis is indicated more often by Accounting than by Sociology researchers. It seems that Accounting researchers are even more apologetic about their use of qualitative data, which might be the main reason why they use CAQDAS to a more extreme degree than Sociology researchers (by going over to theory testing, visual mapping etc.). While Accounting researchers focus on the use of CAQDAS to enhance the validity and reliability of their study, the sociologists indicate to use CAQDAS often for its possibilities in terms of creativity and theoretical sophistication. A lot of Sociology researchers describe how CAQDAS is used for creating
categories for theory development, aided by the memoing functions provided by CAQDAS. In contrast, qualitative Accounting researchers often seem to prefer to rather avoid this option because it makes them feel alienated from the interpretive freedom they have in the qualitative research process.
We observe that the extent of reporting and explanation of how CAQDAS has been used is more diverse in the Accounting than in the Sociology literature. Accounts on CAQDAS use appeared more streamlined and uniform in the Sociology papers. One striking aspect was the place of mentioning CAQDAS in the article. A extreme is an Accounting paper only mentioning the use of CAQDAS in a footnote, while the paper does not reveal in any other way how CAQDAS was used. Another aspect is that some Accounting papers do not report what CAQDAS is used for. Just mentioning that ‘we analyzed our interview material using CAQDAS’ does not provide satisfactory information about the analysis undertaken. We conclude that readers could get better insights in the research process when authors clearly and concisely indicate what they actually did with CADQDAS.
In terms of the reports on CAQDAS use in both disciplines, we find that a lot depends on the journal. Two extremes are the Journal of Social Science and Medicine (providing a highly streamlined account of the research process) versus Accounting, Organizations and Society (characterized by a high diversity in reporting on CAQDAS use). At a first glance, AOS seems to publish more articles that mention marginally that CAQDAS was used, without providing particular details or proving the value for validity and reliability in the paper. In contrast, MAR and AAAJ articles sometimes go so far in their explanation of how CAQDAS improved the rigor of analysis, that they become quite apologetic about the choice for qualitative data.
One avenue for future research would be to link these observations back to the methods textbooks that have been used in both fields. Further, the current analysis only captures how researchers mention the use of CAQDAS in their published article. Another idea for future research is to involve these authors in a survey about their use of CAQDAS. This approach would enable us to get better insights into the degree to which the various CAQDAS options were tested out and actually used.
To conclude, apart from the lessons learned from CAQDAS publications in other disciplines, we want to make sure future CAQDAS users embark on software analysis projects with reasonable expectations: how far-reaching (or not) the options offered, CAQDAS cannot provide a substitute for continuing to think critically about the meaning of data.
Bibliography
Brown, D., 2002. Going Digital and Staying Qualitative: Some Alternative Strategies for Digitizing the Qualitative Research Process. Forum Qualitative Social Research 3 (2).
Buston, K., 1997. NUD*IST in Action: its use and its usefulness in a study of chronic illness in young people. Sociological Research Online 2 (3).
Cannon, J., 1998. Making sense of the interview material: thematising, NUD*IST, and 10meg of transcripts. Presented at the annual meeting of the Australian Association for Research in Education, Adelaide.
Coffey A, B. Holbrook, & P. Atkinson (1996), ‘Qualitative data analysis: technologies and representations’, Sociological Research Online, 1 (1),
http://www.soc.surrey.ac.uk/socresonline/1/1/4.html
Denzin, N.K., 1978. Sociological methods: A Source Book (2nd ed.). McGraw-Hill, New York. Dolan, A., Ayland, C., 2001. Analysis on Trial. International Journal of Market Research 43 (4), 377-389.
Eisenhardt, K.M., 1989. Building Theories from Case Study Research, Academy of Management Review 14 (4), 532-550.
Ferreira, L.D., Merchant, K.A., 1992. Field research in management accounting and control: a review and evaluation. Accounting, Auditing and Accountability Journal, 5 (4), 3-34.
Fielding, N.G., Lee, R.M., 1991. Using Computers in Qualitative Research. Sage Publications, London, Newbury Park, New Delhi.
Fielding, N.G., Lee, R.M., 1998. Computer Analysis and Qualitative Research. Sage Publications, London, Thousand Oaks, New Delhi.
Gephart, R. (2004). From the editors: Qualitative Research and the Academy of Management Journal.
Academy of Management Journal 47 (4), 454-462.
Kelle, U., Laurie, H., 1995. Computer Use in Qualitative Research and Issues of Validity. In: Kelle, U. (ed.), Computer-Aided Qualitative Data Analysis; Theory, Methods and Practice. Sage Publications, London, Thousand Oaks, New Delhi, 19-28.
Kelle, U., 1995. Computer-aided Qualitative Data Analysis; Theory, Methods and Practice. Sage Publications, London, Thousand Oaks, New Delhi.
Lee, R.M., Fielding, N.G., 1995. User's Experiences of Qualitative Data Analysis Software. In: Kelle, U. , Computer-aided Qualitative Data Analysis; Theory, Methods and Practice. Sage Publications, London, Thousand Oaks, New Delhi, 29-40.
Lillis, A.M., 1999. A framework for the analysis of interview data from multiple field research sites. Accounting and Finance 39 (1), 79-105.
Lillis A.M, Mundy J., 2005. Cross-sectional field studies in management accounting research - closing the gaps between surveys and case studies. Journal of Management Accounting Research 17, 119-141. Luft, J., Shields, M.D., 2003. Mapping management accounting: graphics and guidelines for theory-consistent empirical research. Accounting, Organizations and Society 28, 169-249.
Lukka K., Kasanen, E., 1995. Methodological themes. The problem of generalizability: anecdotes and evidence in accounting research. Accounting, Auditing & Accountability Journal 8 (5), 71-90.
Mehan, H., 1979, Learning lessons: Social Organization in the Classroom, Cambridge, MA: Harvard University Press
Miles, M.B., Huberman, A.M., 1994. Qualitative Data Analysis. Sage Publications, Thousand Oaks. Miles M.B., Huberman, A.M., 1998. Qualitative data analysis, an expanded sourcebook. Sage Publications, London, Second ed.
Miles, M.B., Weitzman, E.A., 1994. Choosing computer programs for qualitative data analysis. In: Miles, M.B., Huberman, A.M., 1994. Qualitative Data Analysis. Sage Publications, Thousand Oaks.
Modell S., 2005. Triangulation between case study and survey methods in management accounting research: an assessment of validity implications. Management Accounting Research 16(2), 231-254. Morison, M., Moir, J., 1998. The role of computer software in the analysis of qualitative data: efficient clerk, research assistant or Trojan horse?. Journal of Advanced Nursing 28 (1), 106-116.
Prein, G., Kelle, U., Richards, L., Richards, T., 1995. Introduction: Using Linkages and Networks for Theory Building. In: Kelle, U. (Ed.), Computer-aided qualitative data analysis. Sage Publications, London, Thousand Oaks, New Delhi.
Richards, L., Richards, T., 1991. The Transformation of Qualitative Method: Computational Paradigms and Research Processes. In: Fielding, N.G., Lee, R.M. (eds), Using Computers in Qualitative Research. Sage Publications, London, Newbury Park, New Delhi.
Richards L., 2000a. Using N5 in Qualitative Research. QSR Ltd, Australia. Richards T., 2000b. N5 Reference Guide, QSR Ltd, Australia.
Russell, C.K., Gregory, D.M., 1993. Issues for consideration when choosing a qualitative data management system. Journal of Advanced Nursing 18 (11), 1806-1816.
Ryan, B., Scapens, R., Theobald, M., 2002. Research method and methodology in finance and accounting. Thomson,London.
Scapens, R., 1990. Researching management accounting practice: The role of case study methods. British Accounting Review 22 (3), 259-281.
Scapens, R, Bromwich, M., (2001). Management Accounting Research: the first decade. Management Accounting Research 12 (2), 245-254.
Seale C., 1999. The Quality of Qualitative Research, London:Sage
Seale, C., 2000. Using Computers to Analyse Qualitative Data. In:Silverman, D. (Ed.), Doing Qualitative Research. Sage Publications, London.
Seidel, J., 1991. Method and Madness in the Application of Computer Technology to Qualitative Data Analysis. In: Fielding, N.G., Lee, R.M. (eds), Using Computers in Qualitative Research. Sage Publications, London, Newbury Park, New Delhi.
Tesch, R., 1989. Computer software and qualitative analysis: a reassessment. In G. Blank et al (Eds), New technology in Sociology: Practical applications in research and work pp.141-154, New Brunswick, NJ: Transaction books
Ticehurst, G.W., A.J. Veal, 2000. Business Research Methods: a managerial approach. Pearson Education, French Forest, Australia.
Webb, C., 1999. Analysing qualitative data: computerized and other approaches. Journal of Advanced Nursing 29 (2), 323-330.
Weitzman, E.A., 1999. Analyzing Qualitative Data with Computer Software. Health Services Research 34 (5), 1241-1263.
Weston C., Gandell, T., Beauchamp, J., McAlpine, L., Wiseman, C., Beauchamp, C., 2001. Analysing Interview Data: the development and evolution of a coding system. Qualitative Sociology 24 (3), 381-400.
Yin, R.K., 1994. Case Study Research: Design and Methods. Sage Publications, Applied Business Research Series London.
Yin, R.K., 2003. Case Study Research: Design and Methods. Sage Publications, Applied Business Research Series, London.