Formulating Research Methods for Information Systems

(1)

(2)

Formulating Research Methods for Information Systems

(3)

This page intentionally left blank

(4)

Formulating Research

Methods for Information Systems

Volume 1

Edited by

Leslie P. Willcocks

London School of Economics and Political Science, UK

Chris Sauer

University of Oxford, UK

and

Mary C. Lacity

University of Missouri, USA

(5)

No portion of this publication may be reproduced, copied or transmitted save with written permission or in accordance with the provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any licence permitting limited copying issued by the Copyright Licensing Agency, Saffron House, 6–10 Kirby Street, London EC1N 8TS.

Any person who does any unauthorized act in relation to this publication may be liable to criminal prosecution and civil claims for damages.

The authors have asserted their rights to be identified as the authors of this work in accordance with the Copyright, Designs and Patents Act 1988.

First published 2015 by PALGRAVE MACMILLAN

Palgrave Macmillan in the UK is an imprint of Macmillan Publishers Limited, registered in England, company number 785998, of Houndmills, Basingstoke, Hampshire RG21 6XS.

Palgrave Macmillan in the US is a division of St Martin’s Press LLC, 175 Fifth Avenue, New York, NY 10010.

Palgrave Macmillan is the global academic imprint of the above companies and has companies and representatives throughout the world.

Palgrave® and Macmillan® are registered trademarks in the United States, the United Kingdom, Europe and other countries.

ISBN 978-1-349-56112-4 ISBN 978-1-137-50985-7 (eBook) DOI 10.1057/9781137509857

This book is printed on paper suitable for recycling and made from fully managed and sustained forest sources. Logging, pulping and manufacturing processes are expected to conform to the environmental regulations of the country of origin.

A catalogue record for this book is available from the British Library.

Library of Congress Cataloging-in-Publication Data

Formulating research methods for information systems / [edited by]

Leslie P. Willcocks, London School of Economics, UK, Chris Sauer, University of Oxford, UK, Mary C. Lacity, University of Missouri, USA.

pages cm

Includes bibliographical references.

1. Information storage and retrieval systems – Research. 2. Database searching – Research. I. Willcocks, Leslie. II. Sauer, Chris. III. Lacity, Mary Cecelia.

ZA3075.F67 2015

025.0407291—dc23 2015013406

Individual chapters © Journal of Information Technology (JIT) 2015

(6)

v

I Information Systems Research:

Retrospect and Prospect

1 Retrospect and prospect: information systems research in the

last and next 25 years 19

Allen S. Lee

2 Commentaries on retrospect and prospects for IS research 48 2.1 On some extensions to Lee’s retrospect and

prospect IS paper 48

Mats Lundeberg

2.2 Knowledge lost and found: a commentary on

Allen Lee’s ‘retrospect and prospect’ 50 Richard L. Baskerville

2.3 Retrospect and prospect: information systems in the

last and next 25 years: response and extension 54 Robert M. Davison

2.4 Comments on Allen Lee’s article ‘Retrospect and prospect’: information systems research in the last and next 25 years’ from a socio-theoretical IS research

perspective 61

Chrisanthi Avgerou

3 Towards dissolution of the IS research debate: from

polarization to polarity 66

Brian Fitzgerald and Debra Howcroft

(7)

vi Contents

II Metholodological Practice in Information Systems Research

4 Methodological practice and policy for organisationally and socially relevant IS research: an inclusive-exclusive

perspective 97

Robert M. Davison and Maris G. Martinsons

5 Commentaries on methodological practice 112

5.1 Is there a methodological crisis? 112

Michael D. Myers

5.2 Commentary on Davison and Martinsons:

methodological practice and policy for organisationally and socially relevant IS research: an inclusive – exclusive

perspective 116

Allen S. Lee

5.3 In celebration of diversity in information

systems research 121

Robert D. Galliers

5.4 Open sources? A commentary on ‘IS research methods:

inclusive or exclusive?’ 126

Joe Nandhakumar and Harry Scarbrough

5.5 Pluralism is not about me, it is about us 130 Matt Germonprez

5.6 Research methods and the relevance of the IS discipline: a critical analysis of the role of

methodological pluralism 133

Rajeev Sharma

III Generalizability in Information Systems Research

6 Classifying generalization: paradigm war or abuse of

terminology? 151

John N. Williams and Eric W. K. Tsang

7 Generalization in IS research: a critique of the conflicting

positions of Lee & Baskerville and Tsang & Williams 179 Peter B. Seddon and Rens Scheepers

(8)

Contents vii

IV The Role of Theory in Information Systems Research

8 Is theory king?: questioning the theory fetish in

information systems 213

David Avison and Julien Malaurent

9 Commentaries on the role of theory 238

9.1 Theory is king? But first, what is theory? 238 Allen S. Lee

9.2 Maybe not the king, but an invaluable subordinate:

a commentary on Avison and Malaurent’s advocacy of

“theory light” IS research 245

M. Lynne Markus

9.3 The power of an intellectual account: developing

stories of the digital age 257

Ola Henfridsson

9.4 From “theory light” to theorizing: a reaction to

Avison & Malaurent 262

Deborah R. Compeau and Fernando Olivera

9.5 Taking theory too far? A commentary on Avison and

Malaurent 272

David Silverman

9.6 Theory – still king, but needing a revolution 280 Shirley Gregor

9.7 Is theory king?: a rejoinder 289

David Avison and Julien Malaurent

Correction to: Formulating Research Methods

for Information Systems C1

(9)

viii

List of Figures

1.1 A way of mapping an information system 28

1.2 Another way of mapping an information system 28 7.1 A copy of the Generalizability Framework from L&B03

(their Figure 5) 181

7.2 Our interpretation of L&B03’s generalization framework 185 7.3 Types of Induction, Tsang and Williams “types of

generalization” (T&W12, Figure 1, p.740) 192 9.5.1 The missing phenomenon in quantitative research 276 9.5.2 The missing phenomenon in (some) qualitative research 276

9.5.3 The phenomenon reappears 276

(10)

ix

List of Tables

1.1 Professions and their road to relevance as sciences of the

artificial 44

3.1 Colonicity analysis 69

3.2 Analysis of polysyllabicity and esotericity 70 3.3 Summary of ‘soft’ v. ‘hard’ research dichotomies 78 7.1 Key concepts in Tsang and Williams’ (2012, 2014)

conceptualization of generalization 190

9.1.1 Four Science Categories 240

The original version of this book was revised.

The correction to this book can be found at DOI 10.1057/9781137509857_11.

(11)

1

Overview

The field of Information Systems (IS) has long concerned itself with the process of enquiry, that is, what should be researched, the methods that can be properly utilized, and the assessment of the validity of outcomes.

Like almost all social scientists, IS scholars have been greatly influenced by methods adopted by the natural sciences, and by the power of quantitative techniques. At the same time IS scholars have come from varied backgrounds, and the research role of qualitative enquiry, what the IS field has come to call ‘interpretive’ and ‘critical’ approaches, have been frequently juxtaposed against ‘positivist’ approaches. There is a case to be made that these are no longer helpful distinctions, if they ever were.

Lee and Hubona (2009), for example, show common issues across seem- ingly different research approaches, namely common scientific basis, the fallacy of affirming the consequent and the issue of summative validity. A strong case has also been made for multi-methods and mixed methodologies (Mingers, 2001), and this approach has been increasingly adopted in recent years.

For the reader, getting involved in these and many other issues, and the debates that surround them, will be one of the pleasures emerging from engaging with the 17 chapters of these two volumes. For intro- ductory purposes, our intent here is to point to the key role in research of the process, logic and rigour of enquiry, and provide a meta-perspective across all IS research perspectives. With this purpose in view, Dewey’s (2004a; 2004b) work on pragmatic enquiry and method can provide overarching shape, points and direction. Knowledge as a model of how something works is provisional and contextual. A scientific enquiry will proceed through identifying a problem, challenge, or lack

Introduction

Leslie P. Willcocks , Chris Sauer and Mary C. Lacity

The original version of this chapter was revised.

The correction to this chapter can be found at DOI 10.1057/9781137509857_11.

(12)

2 Leslie P. Willcocks, Chris Sauer and Mary C. Lacity

of understanding that needs to be addressed to further meaning and/or practice. The search creates hypotheses, is experimental, collects evidence and searches with validating controls to discover a provisional, useable truth, in Dewey’s words a warranted assertability, that ‘fits’ (corresponds as a key in a lock, suggests de Waal (2005)) with the evidence and works in practice. The process of enquiry can also be supported by the Peircean notion of scientific method, which he saw as involving three phases or stages (Peirce 1931–1958, 5.590). These were abduction – making conjectures or creating hypotheses for the purposes of testing; deduc- tion – inferring what should be the case, that is, drawing of conclusions as to what observable phenomena should be expected if the hypotheses are correct; and induction – the testing of hypotheses, that is, the entire process of experimentation and interpretation performed in the service of hypothesis testing.

It is useful at this point to warn researchers of the fragility of the research process, and the elusiveness of knowledge, before we launch into the rich and detailed chapters on formulating research methods for IS. John Law is particularly interested in questioning the certainties that so often inhabit research processes, ‘ The problem is not so much the standard research methods themselves, but the normativities that are attached to them in discourse about method. ... Methods, their rules and even more – methods’ practices, not only describe but also help to produce the reality they understand ... (Moreover) the research methods passed down to us tend to work on the assumption that the world is properly understood as a set of fairly specific, determinate, and more or less identifiable processes. ’ For the unwary, Law (2004) points to the inherent messiness in social science research.

But it would seem that this messiness extends also into, and is shared with, the natural sciences. Thus according to Karl Popper (1959), ‘ There is no such thing as a logical method of having new ideas or a logical reconstruction of this process. ’ Meanwhile Michael Polanyi as a working scientist and philosopher has commented: ‘ Upon examining the grounds on which science is pursued I saw that its progress is determined at every stage by indefinable powers of thought. No rules can account for the way a good idea is found for starting an inquiry, and there are no firm rules either for the verification or the refutation of the proposed solu- tion of a problem. Rules widely current may be plausible enough, but scientific enquiry often proceeds and triumphs by contradicting them. ’

These views alert us to the softer, less-explored, less-documented sides of the research process. Indeed, as researchers, we are deeply aware of the essential crafting that makes research possible. This involves employing

(13)

Introduction 3

capabilities that cannot be captured in any one method or approach.

Consider, for example, the roles in your own research of experience, insight, improvisation, your other life experiences, thinking (logically, laterally, deeply, differently) skill and judgement, feeling, commitment and integrity (in design, research and reporting outcomes). Sociologist C.

Wright Mills captured this in his 1959 book The Sociological Imagination :

‘ To the individual social scientist who feels himself part of the classic tradition (of sociology), social science is the practice of a craft. A person at work on problems of substance, he/she is among those who are quickly made impatient and weary by elaborate discussions of method- and-theory-in-general; so much of it interrupts ... proper studies.

Ultimately the most essential question we can ask ourselves in any research process is – How certain can we be? The question might be – ironically – how much faith can we put in our methods and processes of enquiry? In framing the IS research process in this Introduction, it is useful to draw on two major philosophers to tease this issue out a little more.

The first philosopher is John Dewey, who, in his 1929 Gifford lectures on The Quest for Certainty , critiqued the idea of perfection in knowledge.

Instead of drawing a hard line between the knower and the known, knowledge is experimental (Dewey, 1960). When we know things, we do not know them in themselves; rather we know our interaction with them. Knowledge in itself (of the sort Plato countenanced) or gained through a passive empiricism (consider Hume and Locke) is at best ‘thin’

knowledge and cannot substitute for active engagement with the world through experimentation. This leads Dewey on to a theory of action, arguing that we interact with reality to gain new knowledge and control over it. And the major purpose of knowledge is as a tool for further interaction with reality. For Dewey, in the quest for absolute certainty, philosophers (and we might add, researchers) have set up a separation between theory and practice, knowledge and action, which makes fundamental and wrong assumptions about how knowledge is generated, the nature of reality and our interactions with it, what mind is and how it works, and why we seek and how we use knowledge. For IS researchers, such propositions would seem to put new vigour into the debates in the IS field about the roles of theory and practice, of rigour and relevance, of instrumental versus ‘for itself’ knowledge.

Ludwig Wittgenstein is the second major philosopher we draw upon.

Wittgenstein is salutary on the limits to what we can know. First there is the question of what has to be assumed in order for any enquiry to proceed. For him, a world-picture is the substratum of all your enquiring

(14)

and asserting. The propositions describing it are not all equally subject to testing. In On Certainty (1969, 105), he suggests that all testing and all confirmation and disconfirmation of a hypothesis take place already within a system: ‘ ... the questions that we raise and our doubts depend on the fact that some propositions are exempt from doubt, are as it were like hinges on which those turn ’ (Wittgenstein, 1969, 341). All enquiries on our part is set so as to exempt certain propositions from doubt, if they are ever even formulated: ‘ They lie apart from the route travelled by enquiry’ (Wittgenstein, 1969, 88). Connectedly, the truths and beliefs that emerge from the enquiry belong to our frame of reference. When we first begin to believe anything, what we believe is not a single prop- osition but a whole system of propositions (Wittgenstein, 1969, 141).

Wittgenstein suggests there is always something misleading about the use of the expression ‘true or false’. For him it is like saying ‘it tallies with the facts or it doesn’t’, when the very thing that is in question is what ‘tallying’ is here. In practice, what men and women consider reasonable alters. What is considered reasonable at certain periods is considered unreasonable at others, and vice versa.

The difficulty, for Wittgenstein, is to realize the groundlessness of our believing, proposing that ‘knowledge, in the end, is based on acknowl- edgement’ (Wittgenstein, 1969, 378). In other words, it is a social agree- ment. Clearly Wittgenstein has profound doubts about how much we can know as a result of any process of enquiry. He suggests a thought experiment that as researchers we might take on board: ‘Suppose it were forbidden to say “I know” and only allowed to say “I believe I know”?’

(Wittgensein, 1969, 366). In On Certainty (Wittgenstein, 1969), and even more extensively in the Philosophical Investigations (2009), Wittgenstein underscores the therapeutic role of philosophy in reminding us of how terms function within language games, and in helping us to come to terms with the realization that there are no all-embracing criteria for the assessment of things in the world, to which we could appeal, with absolute certainty, with ‘complete conviction, the total absence of doubt’

outside the ‘nest of propositions’ (1969, 225) in which we dwell.

We do not, of course, have to believe these philosophers. However, given their high and enduring status in their field, their views at least provide an opportunity for a deeper thinking through of the issues for the prospective IS researcher. This would be particularly productive for IS researchers, who, despite calls for more theory from many (see Chapters 1 and 2), typically do not engage systematically in establishing philosophical foundations for their research, or dealing with philosophical issues in their work (Mingers and Willcocks, 2004).

(15)

Introduction 5

Having, as it were, ‘framed’ these two volumes, let us now look at the content of Volume 1. The first volume of this Formulating Research Methods in Information Systems series includes interesting and compel- ling articles from the Journal of Information Technology (JIT) pertaining to the past and future of IS research, methodological practice, the role of general- izability and the role of theory.

Introduction to Section I – Information Systems Research:

Retrospect and Prospect

In Section I we have three chapters that give an overview of major research issues in the past and for the future of the IS field. Chapters 1 and 2 give the views of a range of senior scholars on the development of IS research over the past and next 25 years. Chapter 3 seeks to move from polarization to polarity in the debates over research approaches and methods.

In 2010, Allen Lee, a senior scholar, was asked to reflect on the past and future of the IS in commemoration of the first 25 years of the JIT . He does just that in Chapter 1, ‘Retrospect and Prospect: Information Systems Research in the Last and Next 25 Years’, which provides a debating plat- form for the commentaries in Chapter 2. Lee poses a challenge – that our development and knowledge should be more constructivist and design science orientated, but from within our typical locus in universities, which is in business schools and management departments. Unlike, say organizational behaviour, the field has not been stable for long enough for all its knowledge to be built cumulatively by hypothetic–deductive research. But this is a challenge to the established disciplines, and also to the IS journals. Can we achieve and retain sufficient respectability in the eyes of our peers while operating under different epistemic norms?

Allen Lee shows that taken-for-granted concepts such as ‘information’,

‘theory’, ‘system’, ‘organization’ and ‘relevance’ need to be rethought, and poses the challenge that the future development of the IS field may be better modelled on the research disciplines found in the professions, including medicine, law, architecture and engineering.

Chapter 2 consists of four commentaries on Allen Lee’s chapter. In

‘On some extensions to Lee’s retrospect and prospect IS paper’, Mats Lundeberg argues that, as an IS ‘native’, he is sympathetic to Lee’s perspective and focuses on the ramifications for research approaches. In particular, he comments on problematizing theories-in-use and espoused theories, balancing prescriptions and general direction, and working with different levels of abstraction. Richard Baskerville contributes

(16)

‘Knowledge lost and found: a commentary on Allen Lee’s “retrospect and prospect”’. Here he not only welcomes Lee’s revisiting of systems theory but also argues that, in Lee’s perspective, technology seems more separated from organizational systems than reality will allow, and invites a further extension of Lee’s concern about ‘organization’. Baskerville also offers a nuanced deliberation on the role of theory in IS, and points to the challenges inherent in the anxiety to ‘scienc-ify’ the IS field, and the gulf between what and how IS is researched, and how it is taught and the knowledge used, agreeing the need to anchor future studies in problems of practice – echoes of our Dewey discussion above.

In an assessment of Allen Lee’s paper, Robert Davison has written

‘Retrospect and prospect: information systems in the last and next 25 years: response and extension’. Davison argues that in some respects the changes need to be more radical than Lee suggests, including in the way PhD students are trained, in the selection criteria for new teachers and how leading journals support different types of research. Davison goes on to argue that emerging markets, particularly India and China, may also change assumptions and approaches in the IS field, though he worries that at the moment Western intellectual hegemony is being perpetuated in these countries rather than being reshaped. Sensitized to the culture and politics of the IS field, Davison suggests more context- sensitive problem shaping and research is necessary, and more questioning of beliefs and taken-for-granted assumptions that are rife in the IS field – echoes here of Wittgenstein discussed above.

In ‘Comments on Allen Lee’s article “retrospect and prospect”: information systems research in the last and next 25 years’ from a socio-theoretical IS research perspective’, Chrisanthi Avgerou extends Davison’s concerns, arguing for richness and diversity in the IS field. She questions Lee’s suggested programme, wondering about the validity of the detec- tion of common theories-in-use for a few fundamental concepts across the whole IS field, and is also wary of its consequences. She argues that, in fact, the co-existence of alternative theories for an observed phenomenon is the norm rather than the exception, both in the natural and social sciences. Moreover, within IS, there are institutional obstacles to publishing some types of research, for example socio-theoretical research and design research. These obstacles should not be mis-diagnosed as across-the-field epistemological weaknesses. Avgerou also argues that Lee probably underplays the historical, social and intellectual evolution of the five fundamental IS concepts he addresses, and worries that choosing between becoming a science of the natural and science of the artificial in systems theory terms is a dilemma dangerously narrowing

(17)

Introduction 7

IS research routes. Historically, IS has developed a broader epistemological scope, including science of the social, the economic and the socio- technical.

Chapter 3 is ‘Towards Dissolution of the IS Research Debate: From Polarization to Polarity’. Here Brian Fitzgerald and Debra Howcroft point to the debate between ‘hard’ and ‘soft’ research approaches that continues in the IS field, but with little prospect of resolution . The debate is typically characterized by tendentious arguments as advocates from each approach offer a somewhat one-sided condemnation of the counterpart from the inimical research tradition. This chapter begins by relating two fictitious tales which serve to highlight the futility of research conducted at the extremity of each research approach.

The dichotomies which characterize these rival factions are also summarized. The debate is then framed in terms of the polarization problem whereby IS researchers are divided geographically and paradig- matically into ‘hard’ and ‘soft’ camps. A variety of different strategies have been proposed for resolving the debate and these are discussed in detail. They are grouped into four categories, referred to as suprema- cism, isolationism, integration and pluralism . Finally, the authors contend that the debate cannot be resolved, and offer the metaphor of magnetic polarity as a means of reflecting this. The chapter concludes by arguing that it would be more appropriate to recast the debate at a macro-level in order to accommodate different research agendas and recognize the strengths within each tradition.

Introduction to Section II – Methodological Practice in Information Systems Research

Section II includes two chapters that deal with major debates around the subject of methodological practice. This section begins with Chapter 4 which argues that IS as a research field is notable for its methodological one-sidedness in favour of work in the positivist tradition. Chapter 5 provides six assessments of this contention, and proposes, with Chapter 4, a variety of ways forward.

Chapter 4 is ‘Methodological Practice and Policy for Organisationally and Socially Relevant IS Research: An Inclusive-Exclusive Perspective’.

In this chapter, Robert Davison and Maris Martinsons argue that the tendency of IS researchers to work with a rather parsimonious set of research methods, notably those that follow the positivist tradition, demonstrates a significant degree of methodological exclusiveness. They contend that such an exclusive practice is both counterproductive to

(18)

good IS research and unethical. In addition, it severely and unreasonably limits the extent to which IS research and researchers can contribute to both pressing organizational problems and the scholarly literature. The authors synthesize their position in a set of four arguments that guide further discussion of the nature and consequences of methodological exclusiveness, as well as of possible solutions. The chapter concludes with an exposition of steps that could be taken to address the current situation.

To address the issues raised and solutions suggested by Davison and Martinsons (DM), Chapter 5 provides six Commentaries on Methodological Practice . In ‘Is there a methodological crisis?’ Michael Myers, a notable proponent elsewhere of interpretive research methods and practices, agrees with many of DM’s recommendations but not with the conclusion that there is a methodological crisis. He agrees that PhD students should be much more trained in multiple research methods – in fact suggests, beyond DM that this should become compulsory – but does not see it as inappropriate or unethical for students to adopt the same methods as their supervisors.

Allen Lee chooses a different perspective. In ‘Commentary on Davison and Martinsons: methodological practice and policy for organization- ally and socially relevant IS research: an inclusive–exclusive perspective’, Lee (Section 5.2) joins DM in their call for change and challenges them to go further, seeking ‘to reframe and extend the truth that they speak’ . He finds the anti-positivist language and arguments unhelpful, and offers an interpretive explanation as to why positivist researchers operate as they do. It reflects their training; they have no nefarious rationales; it reflects their understanding of the world and how to analyse phenomena therein. There have been different groups in IS each with their own ‘language’ and ‘culture’ where up to fairly recently no group had a clear understanding of the language and culture of the other. The problems stem from the historical legacy and the natural misinterpretation of groups of one another. The four remedies DM suggest will help, but Lee sees structural change as necessary, and not just at the individual level.

In the next commentary Robert Galliers writes in ‘In celebration of diversity in Information systems research’, arguing that closed systems tend towards entropy. He supports, with reference to evidence, the claims of limited PhD methods in training and the over-emphasis on positivist, quantitative methods in the IS field in both the training and in the overall number of journal publications. Citing the Kolb learning

(19)

Introduction 9

cycle, Galliers points to the value of an all-round experience of multiple methods, but recognizes that ‘old habits die hard’, for example what he calls the tyranny of the ‘or’ in the ‘rigour or relevance’ debate so often readdressed in the IS field.

Joe Nandhakumar and Harry Scarbrough offer ‘Open sources? a commentary on IS research methods: inclusive or exclusive?’ They see a greater plurality of methods as one strand in the discipline’s ability to overcome the challenges identified by Davison and Martinsons.

However, they argue that, without changes in the wider discipline itself, such plurality risks becoming a token of diversity and not the driver for the more engaged and questioning scholarship that Davison and Martinsons advocate. There has been some convergence between the theoretical and practical concerns of the IS discipline with a wider set of globalizing changes within and across organizations. For this shift to be sustained, however, ‘it is crucial that IS remains an open not a closed discipline; a field of enquiry less rigidly organized than others but one which, as a result, is better equipped to address emerging societal and theoretical challenges.’

The commentary by Matt Germonprez argues that ‘Pluralism is not about me, it is about us’. By this he means that the pluralism which DM advocate should be a communal activity and not a burden to be borne by any one individual. Pluralism is about how we as an IS community understand how to recognize and value pluralism among our colleagues working on similar IS research enquiries. This community of pluralism is not achieved through providing equity but through revealing values.

Germonprez also argues that the role of the academic researcher is clearly changing to reflect the nature of the digitally enabled world around us, and pluralism is particularly useful and necessary to reveal, identify and frame newly emergent phenomena.

In the final commentary Rajeev Sharma provides ‘Research methods and the relevance of the IS discipline: a critical analysis of the role of methodological pluralism’. Sharma argues that DM’s call for more pluralism in research methods is underpinned by the assumption that this will make the discipline more relevant. He finds the call for methodological pluralism commendable, and recommends that it be promoted at the level of the body of IS research rather than at the level of the individual or research programme. However, he suggests that in itself this is unlikely to have an impact on the relevance of IS, or solve any crisis it might have. We need to look beyond methods to explain the current state of IS, and to solve any crisis it is facing.

(20)

Introduction to Section III – Generalizability in Information Systems Research

Section III includes two chapters that extend – but also try to conclude – a long-standing, and important, debate in IS research on generalization and generalizability. The two chapters discuss the implications of the robust exchange of views about the meaning of the term ‘generalization’

in three previous papers: Lee and Baskerville (2003), Tsang and Williams (2012) and Lee and Baskerville (2012). The first paper by Lee and Baskerville was, in many ways for the IS community, groundbreaking when it appeared in 2003, but subsequently it has ignited a considerable – and healthy – debate on this key topic amongst scholars.

Chapter 6 concerns ‘Classifying Generalization: Paradigm War or Abuse of Terminology?’ Here John Williams and Eric Tsang provide some history to a debate they have been having with Allen Lee and Richard Baskerville, whose 2003 paper attempted to clarify the concept of generalization, and classify generalization into four types. Tsang and Williams (2012) objected to the Lee and Baskerville account of generalization as well as their classification and offered repairs. Tsang and Williams then proposed a classification of induction, within which they distinguished five types of generalization.

In Chapter 6, John Williams and Eric Tsang (WT) object that in the Lee and Baskerville (2012) rejoinder most of the TW propositions and recommendations were not actually responded to. Instead, Lee and Baskerville (LB) argued that the two classification systems were in practice compatible, but that they themselves were offering a ‘new language’. In this chapter Williams and Tsang suggest that in so far as they resist this ‘new language’ and insofar as LB think that WT’s position is committed to positivism and the rejection of interpretivism, WT stand accused of ‘paradigmatic domination’. To further the debate, in Chapter 6, Williams and Tsang first clarify the terms ‘theoretical statement’ and ‘empirical statement’. Then they seek to show that the WT and LB classifications of generalizing are incompatible. For Williams and Tsang, the LB classification remains flawed and should not be relied upon. Furthermore, Williams and Tsang seek to show that they are inno- cent of ‘paradigmatic domination’.

Chapter 7 is ‘Generalization in IS Research: A Critique of the Conflicting Positions of Lee & Baskerville and Tsang & Williams’. Here Peter Seddon and Rens Scheepers provide a companion to, and assessment of, the arguments made by Williams and Tsang in Chapter 6.

Their objectives are, first, to help the reader understand the issues by

(21)

Introduction 11

summarizing the arguments in the four previous papers, and second, to assess the implications of the debate for future IS research. Seddon and Scheepers conclude that when the chapters are interpreted from the perspectives of the respective pairs of authors, most of what they say is sound. However, because their perspectives are so different, their differ- ences of opinion are also very real. As a way of showing that neither pair of authors’ conception of generalization is the ‘last word’ on this topic, the chapter also compares key concepts from both pairs of authors with those from Seddon and Scheepers (2012). It is argued that although the Seddon and Scheepers’ framework is also not the ‘last word’, it may prove more useful than either of the two preceding (LB and WT) frameworks.

Introduction to Section IV – The Role of Theory in Information Systems Research

Section IV includes two chapters that address what has become the much contested role of theory in IS research. Chapter 8 is a response to its authors’ sense that many leading IS journals – and scholars – seem over- insistent on the necessity for, and amount of, theory in research papers.

Chapter 9 reveals a very rich, not always supportive, set of responses and proposals to this suggestion, and to the idea of IS needing and encour- aging many more ‘theory-light’ papers.

Chapter 8, by David Avison and Julien Malaurent, is entitled ‘ Is Theory King? Questioning the Theory Fetish in Information Systems’. Here the authors argue that there is too much emphasis on the requirement for theory use and theory building in qualitative research published in leading IS journals. The authors discuss six concerns they have that relate to this high status of theory in such papers. Avison and Malaurent argue for what they refer to as ‘theory-light’ papers where theory plays no significant part in the paper and the contribution lies elsewhere, for example new arguments, facts, patterns or relationships. Some examples of theory-light papers (and research) are provided from other disciplines and one exemplar information systems paper is studied in depth.

Avison and Malaurent see these papers as equally worthy as those which demonstrate the applicability and predictive qualities of theory use as well as the potential of theory building. They propose a list of ten questions that authors and reviewers might ask themselves when writing or reviewing such theory-light papers. The more demanding role for the reader is also discussed along with the requirement for editorial teams to adapt. They suggest that the requirement for a contribution to theory would be replaced with the requirement that any journal paper has a

(22)

high potential for stimulating research that will impact on information systems theory and/or practice.

Chapter 9 is entitled ‘Commentaries on the Role of Theory’. Allen Lee addresses the question ‘Theory is king? But first, what is theory?’

He points to the lack of precision with which the word ‘theory’ is used in the IS field. Lee argues that what theory is cannot be separated from what science is and develops four science categories to suggest a nuanced view of theory. He invites us to entertain the possibility that

‘theory’ is not the same across the entire terrain of the four categories of science. This leads us to question the presumption that ‘theory’

necessarily is, or should be , the same across all sciences – a presump- tion for which there is no a priori justification. The problem is that theory as applied in the natural sciences (science category I in Lee’s schema) has tended to generate hegemonic sway over how theories are considered in the other three categories. But IS does not study the physical world but rather the world of people and their institutions which fall into Lee’s science categories II and IV. In fact as a ‘science of the artificial’ that seeks to describe and create what does not, or has not yet existed, it probably best fits category IV, along with business, social work, law, education, public policy and clinical psychology.

From this analysis, Lee suggest that science category I theory should not be king in a field such as IS, and that even what AM describe in their examples as theory-light papers may well be quite rich in theory of a different kind.

Lynne Markus heads her response as ‘Maybe not the king, but an invaluable subordinate: a commentary on Avison and Malaurent’s advocacy of

“theory light” IS research’. Here she first explains why she is predisposed to agree with Avison and Malaurent’s call for the publication of high- quality ‘theory light’ qualitative IS research articles as a complement to theory-contributing papers. Second, she offers an additional rationale for ‘theory light’ qualitative research: ‘Theory light’ quantitative research (sometimes called ‘big data’ research) is already a prominent feature of top IS journals. Third, she proposes an alternative theory of the problem that Avison and Malaurent identified: outcomes such as trivial and uninteresting findings in qualitative IS research might result not only from overemphasis on theory but also from conflicting or overly narrow definitions of theory and theoretical contribution. Consequently, Markus proposes an alternative (or complement) to Avison and Malarent’s solu- tion: she advocates qualitative IS research that develops theories of substantive human and societal and/or IT-related problems and theories of solutions to those problems.

(23)

Introduction 13

In ‘The power of an intellectual account: developing stories of the digital age’, Olaf Henfridsson proposes a position that he feels is not too far from that of AM. Maybe, he suggests, the problem lies with the awkward term ‘theory-light’. Theory may not be king, but theorizing certainly is, and no powerful intellectual account can be without theorizing. If IS has compelling stories to tell, then we increase our chances to develop compelling case stories if we (a) make sequences of events meaningful, (b) build ties to cumulative tradition, (c) name and frame, and (d) stress-test the intellectual account.

In ‘From “theory light” to theorizing: a reaction to Avison &

Malaurent’, Deborah Compeau and Fernando Olivera agree with Avison and Malaurent (AM) about the concerns they raise, but view their solu- tion of promoting ‘theory light’ papers to be problematic, and ultimately unnecessary. They find the term ‘theory-light’ too ambiguous.

They also argue that theory is essential to any research process. They question whether the examples of theory-light articles cited by AM are in fact so. They then argue that the real problem is the IS field’s overly narrow emphasis on a particular type of theoretical contribution and turn to a discussion of the various ways of thinking about theory, and to their belief that it is theorizing rather than theory that matters most.

David Silverman’s commentary ‘Taking theory too far? A commentary on Avison and Malaurent’ comes from a highly regarded sociologist and qualitative researcher who finds considerable sympathy with AM’s position. He suggests that IS is not alone; over-valorizing theory can occur even in subjects with a single, coherent theoretical base. However, we have to be cautious not to throw the baby out with the bath water. Data do not speak for themselves and so without theory there is no data. So it is always a question of balance between theories and data. In qualitative research, in particular, theoretical window dressing does little to conceal a largely pre-theoretical, commonsensical way of interpreting data.

Silverman is very insightful on the strengths of qualitative research, for example, showing how to extend AM’s points about rigour: ‘This is usually best achieved by generating hypotheses from the intensive analysis of a small amount of data. These hypotheses can then be tested through extensive analysis of one’s whole data set to obtain deviant cases. Such cases are then analysed in order to modify one’s hypotheses.’ For Silverman theorizing, conceived as relating a set of concepts, is usually best done in the course of data analysis. He stresses the impor- tance of thinking with/through data. So, instead of advocating ‘theory light’ papers, it might be better to talk about work which treats theory as emergent.

(24)

In ‘Theory – still king, but needing a revolution’, Shirley Gregor agrees that Avison and Malaurent raise valid concerns about journal norms in respect to acceptance of novelty, tolerance for triviality, possibly unethical behaviour, and implicitly accepted epistemological practices for theorizing and theory development. Although she shares many of their concerns about journal practice, she has serious concerns that readers will view their chapter as attacking theory-in-itself despite the authors’ protestations that they are not doing so. Gregor suggests that the guidelines they advance could promote the idea that theory-free articles are acceptable, which she sees as a dangerous path for the IS field. We should be focussing more on the theorizing process and under- lying epistemological issues. In this way, we can view and justify new and surprising ideas and interesting empirical findings in single articles against a backdrop of ongoing theory development in the field as a whole.

In their rejoinder to these commentaries, Avison and Malaurent keep the debate very much alive by first detailing what has not been contested – the criticism of leading journals for not being original, chal- lenging and exciting; the fact that IS does have compelling stories to tell;

the ten attributes AM put forward for good ‘theory-light’ papers. They refute any charges of theory bashing or down-playing the role of theory.

Responding to the call to broaden theory, they suggest that they are not against this; but even more important in papers that stress theory is a greater deepening – stronger and better founded. The term ‘theory-light’

is defended, and AM point out that the term never meant that they were for ‘theory-free’. They pick up on Markus’s observation that in IS there are already numerous published ‘theory-light’ quantitative papers. They agree with her conclusion that if high-quality theory-light quantitative papers are acceptable in leading IS journals then that should be the case for the equivalent ‘theory-light’ qualitative papers as well.

Conclusion

The carefully selected JIT articles in this volume capture the living debates surrounding the development of the IS field and the application of research methods therein. Using a time horizon of the past and the next 25 years, the chapters examine whether there is a methodological crisis in the field, the role of diversity, the nature of inter-disciplinary discourse, the relevance and applicability of generalization, and whether theory is king, or something else altogether.

(25)

Introduction 15

What surprised us as editors in looking across 30 years of research and publication represented just in one journal is the richness and diversity, and robust health of research and the debates within the IS field, and its clear evolution towards more methodological pluralism. Willcocks and Lee experienced a similar surprise several years ago when they were asked to edit a volume which subsequently came out as Major Currents in the Information Systems Field (Willcocks and Lee, 2008). On that occasion the two editors were startled to find that they had to classify the IS field into six areas of endeavour and uncovered so much high-quality mate- rial that they had to appoint six sub-editors to produce a six-volume set of representative articles. On this occasion, we thought that one volume would encapsulate the best that has been written in the JIT on the subject of research methods; instead we are producing a two-volume set, with the second volume additionally covering critical analyses and literature review techniques, and the craft of case study research.

There will then be a further three volumes entitled ‘Enacting Research Methods in Information Systems’. This would suggest that the ‘crisis’

alluded to often in the chapters in this present volume may be the birth pangs of a vibrant field, that, looking ahead, is set to play an increasingly central role in understanding how information and communica- tion technologies are designed, shape the contemporary world and with what consequences.

Note

The titles and academic institutions for the authors as listed at the end of chapters were correct on the date of original publication. These details may have changed subsequently, and readers need to bear this in mind if trying to contact any author.

References

de Waal, C. (2005) On Pragmatism . Belmont: Wadsworth.

Dewey, J. (2004a) Essays in Experimental Logic . New York: Dover Press.

Dewey, J. (2004b) Reconstruction in Philosophy . New York: Dover Press.

Dewey, J. (1960) The Quest for Certainty: A Study of the Relation of Knowledge to Action. Capricorn, London: Capricorn.

Lee, A. and Baskerville, R. (2003) Generalizing Generalizability in Information Systems Research. Information Systems Research 14(3): 221–243.

Lee, A. and Baskerville, R. (2012) Conceptualizing Generalizability: New Contributions and a Reply. MIS Quarterly 36(3): 749–761.

Lee, A., and Hubona, G. (2009) A Scientific Basis for Rigor in IS Research. MIS Quarterly 33: 237–262.

(26)

Law, D. (2004) After Method: Mess in Social Science Research . London: Routledge.

Mills, C. Wright (1959) The Sociological Imagination . Oxford: Oxford University Press.

Mingers, J. (2001) Combining Research Methods in IS: Towards a Pluralist Methodology. Information Systems Research 12(3): 240–259.

Mingers, J. and Willcocks, L. (eds) (2004) Social Theory and Philosophy For Information Systems . Chichester: Wiley.

Peirce, C. (1931–1958) Collected Papers of Charles Sanders Peirce (8 Volumes) . Cambridge: Harvard University Press.

Polanyi, M. (1966) The Logic of Tacit Inference. Philosophy 41(155): 1–18 Popper, K. (1959) The Logic of Scientific Discovery . London: Hutchinson.

Seddon, P. and Scheepers, R. (2012) Drawing General Conclusions from Samples:

Towards the Improved Treatment of Generalization of Knowledge Claims in IS Research. European Journal of Information Systems 21: 6–21.

Tsang, E. and Williams, J. (2012) Generalization and Induction: Misconceptions, Clarifications, and a Classification of Induction. MIS Quarterly 36(3): 729–748.

Willcocks, L. and Lee. A. (2008) Major Currents in the Information Systems Field.

Sage, London, six volumes.

Wittgenstein, L. (1969) On Certainty . Oxford: Basil Blackwell.

Wittgenstein, L. (2009) Philosophical Investigations . Chichester: Wiley, 4th edition.

(27)

I

Information Systems Research:

Retrospect and Prospect

(28)

19

To its detriment, past research in information systems (IS) has taken for granted many of its own key concepts, including ‘information ,’ ‘theory ,’ ‘system ,’ ‘organization ,’ and

‘relevance.’ This essay examines these concepts, shows how they have been neglected, and offers the prospect in which research in IS no longer models itself on the research disciplines found in the natural and social sciences, but instead charts a course for its future development by modeling itself on the research disciplines found in the profes- sions, such as medicine, engineering, architecture, and law.

Keywords: information; theory; system; organization; relevance; professions

Introduction

There are doubtless numerous, equally insightful perspectives with which one can, first, take stock of what the academic discipline of information systems (IS) has achieved in the last 25 years and, second, chart out worth- while paths for its development in the next 25 years. Certainly, a prescription of what remains to be achieved should account for a diagnosis of what has, or has not, already been done. And in performing a diagnosis and formulating a prescription, one could proceed as a ‘native’ (as a member of the community of IS researchers who is addressing other members of the community) or as an ‘outside observer’ (as a researcher doing research on this community, like an anthropologist doing research on natives).

1

Retrospect and prospect:

information systems research in the last and next 25 years

Allen S. Lee

School of Business, Virginia Commonwealth University, USA

Reprinted from Journal of Information Technology, 25, 336–348, 2010, doi:10.1057/

The original version of this chapter was revised.

The correction to this chapter can be found at DOI 10.1057/9781137509857_11.

(29)

20 Allen S. Lee

Members of the IS research community have already provided compila- tions of their discipline’s past achievements, along with thickly detailed reflections on them. For instance, there is the six-volume compila- tion, Major Currents in Information Systems (Willcocks and Lee, 2008), showcasing papers representative of six areas of IS research: IS research infrastructure, IS development, design science theories and research practices, management of IS, social and organizational IS research, and IS, globalization and developing countries. Insightful reflections, high- lighting and diagnosing the state of the art in each area, were provided by the respective volume’s editors: Debra Howcroft and Frank Land;

David Avison and Richard Baskerville; Alan R Hevner; Mary C Lacity;

Jonathan Liebenau and Nathalie Mitev; and Chrisanthi Avgerou. The perspective taken was one of members of the research community who are examining the work of other members of the research community – natives speaking to and about other natives, as it were.

Regarding this essay’s theme on retrospect and prospect in IS research, I will proceed from the inventory of past achievements already taken by other members of our research community. In proceeding, I will be mindful of the distinction drawn by Argyris and Schön (1978) between what they call ‘theories-in-use’ and ‘espoused theories’ (pp. 6–7):

‘When someone is asked how he would behave under certain circum- stances, the answer he gives is his espoused theory of action for that situation. This is the theory of action to which he gives allegiance, and which, upon request, he communicates to others. However, the theory that actually governs his actions is his theory-in-use, which may or may not be compatible with his espoused theory; furthermore, the individual may or may not be aware of the incompatibility of the two theories.’

I will be using the term ‘theories-in-use’ not to refer to theories found in the IS research literature, but to refer to theories about how to do research – for instance, an IS researcher’s own personally held beliefs that have guided her research activities in the past and that could be supplemented, further developed, or otherwise changed so as to advance IS research in the future. The perspective I will take is one of an outside observer interested in examining the taken-for-granted beliefs held by natives who happen to call themselves ‘IS researchers.’ Apart from their activity of observing other natives (to whom they refer as ‘organizational members,’ ‘actors,’ ‘managers,’ ‘executives,’ ‘customers,’ ‘survey respondents,’ ‘experimental subjects,’ and so forth), IS researchers are no less natives in their own right and are no less suitable to have their own taken-for-granted theories-in-use opened up and identified.

(30)

Retrospect and prospect 21

Detrimental consequences can follow from the variance between a native’s ‘theory-in-use’ and the native’s ‘espoused theory.’ A senior manager may publicly espouse and truly believe, for instance, that he subscribes to racial and gender equality even if, in practice, his hiring and promotion decisions consistently favor one race and gender over another. Similarly, editors may publicly profess and truly believe that qualitative research and design research are just as valid as statistical hypothesis-testing research even if, in practice, their editorial decisions consistently favor statistical hypothesis-testing research over qualitative research and design research.

The detrimental consequences of the variance between a theory-in-use and an espoused theory include not only direct, problematic outcomes (e.g., racial and gender inequality, or non-publication of deserving qualitative and design research), but also the resulting invisibility of the problematic outcomes to the person whose professed belief is different from the outcomes. A self-sealing dynamic, in which the functioning of the theory-in-use is perpetuated, emerges from the fact that theories-in-use are taken for granted and not noticed by the people holding them. ‘Like fish who are presumably unaware of the water in which they swim’ (Van Maanen, 1979: 546), people are typically unaware of the theories-in-use that they use to navigate through their day-to-day activities, including their work. IS researchers can be considered to operate under a number of theories-in-use which are at variance with their espoused theories.

Consider, for example, many IS researchers’ espoused belief in scientific rigor; it calls for them to render key terms into what they call ‘scientific constructs.’ These researchers have not, however, done this with the key term, ‘technology.’ Instead, ‘technology’ remains an ever-present but unreflected-upon idea that IS researchers take for granted when they do their research, much like the water that fish take for granted even as it surrounds them when them swim.

Another illuminating case-in-point is the term ‘social networking’ or

‘social network analysis.’ Though not an expert in this field of study, I was asked to participate in a workshop on it at the 2010 meeting of the Hawaii International Conference on Systems Sciences. In preparing for the workshop, I learned that social network analysis has its origins in sociology and anthropology, where the objectively existing entity being studied is not individuals or even a collection of individuals, but a larger whole – namely, the social structure, which is not the sum of its parts, but greater than the sum of its parts. A basic feature of the idea of social structure is that it has an existence independent of the individuals who happen to be occupying it at the moment. This is a defining feature