érvalua+ion
Metkods
for their friendship and scholarship, and their commitments to methodological eclecticism, interdisciplinary inquixy, integration of theory and practice, valuing both reflection and action, scholarship that is socially relevant and meaningful, individualized professional and personal development, lifelong learning/ social justice and equity,
human diversíty and global community; a scholarly community governed by principies and processes rather than rules and regulations, and iimovations in learning-centered, nontraditional doctoral education, including faculty meetings that are interesting
EDITION
F A C U L D A D E DE ní-wc» DESPORTO t DE E D Ü Ca Ç A í í UNIVERSIDADE DO PORTC B Í B L Í O T H C .
Michael Quinn Patton
H P O R T O
O FACULDADE DE DESPORTO UNIVERSIDADE DO PORTO
«015774»
F A C U L D A D E P E C I Ê N C I A S D O DESPORTO E Dí- EDUCAÇÃO FÍSiCA
UNfVERSfDARE 00 PORTO
I
| / Q ^
4 SCTy/fTâM
/ÍÈvSage Publications
I X I International Educational and ProfessionaI Publisher Thousand Oaks • London • New Delhi
Ali rights reserved. No part of this book may be reproduced or utilize d in any form or by any means, electronic or mechanical, including photocopying, recording, orby any infor-ma tion storage and retrieval system, without permission in writing from the publisher.
For information:
Sage Publications, Inc. 2455 Teller Road
Thousand Oaks, Califórnia 91320 E-mail: [email protected] Sage Publications Ltd. 6 Bonhill Street London EC2A 4PU United Kingdom
Sage Publications índia Pvt. Ltd. M-32 Market
Greater Kailash I New Delhi 110 048 índia Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Patton, Michael Quinn.
Qualitative research and evaluation methods / by Michael Quinn Patton.— 3rd ed.
p. cm. „— Rev. ed. of: Qualitative evaluation and research methods. 2nd ed. 1990. )
Includes bibliographical references and index. ISBN 0-7619-1971-6
1. Social sciences—Methodology. 2. Evaluation research (Social action programs). I. Patton, Michael Quinn. Qualitative evaluation and research methods. II. Title.
H62.P3218 2001 300'.7'23—dc21 2001005181 04 05 06 10 9 8 7 6 5 4 Acquiring Editor: Editorial Assistant: Production Editor: Editorial Assistant: Copy Editor: Typesetter/Desigtier: Cover Designer: C. Deborah Laughton Verônica Novak Diana E. Axelsen Kathryn Joumey Kate Peterson Janelle LeMaster Michelle Lee
BHef (Confervfs
Preface xxi
PjA"RTT 1 . Conceptual Issues in Qualitative Inquiry 1
1. The Nature of Qualitative Inquiry 3 2. Strategic Themes in Qualitative Inquiry 37 3. Varie ty in Qualitative Inquiry: Theoretical Orientations 75
4. Particularly Appropriate Qualitative Applications 143
P A R T 2 . Qualitative Designs and Data Collection 207
5. Designing Qualitative Studies 209 6. Fieldwork Strategies and Observation Methods 259
7. Qualitative Interviewing (^339^
P j A R T 3 . Analysis, Interpretation, and Reporting 429
8. Qualitative Analysis and Interpretation (431^) 9. Enhancing the Quality and Credibility of Qualitative Analysis
References RI Author Index 11 Subject Index 113 About the Author Al
Defailed (Zov\ie.rús
Preface xxi
P . A R T 1, Conceptual Issues in Qualitative Inquiry 1
1. The Nature of Qualitative Inquiry 3 The Fruit of Qualitative Methods 3 | g g S j | | i Three Kinds of Qualitative Data 4
Recognizing Qualitative Data 4 Qualitative Findirigs: Themes, Patterns/ Concepts, Insights, Understandings 5
E H 1 H Women's Ways of Knowing: An Example of
Qualitative Frndings 7 m H n m B n
Coming-of-Age Paradigms 9 Different Purposes of and Audiences for Qualitative Studies:
Research, Evaluation, Dissertations, and Personal Inquiry 9
Making Methods Decisions 12 Methods Choices: Contrasting Qualitative and Quantitative Emphases 12
n s m n n H n
m U l i ^ l U B i Some Guiding Questions and Options for Methods Decisions 13
Comparing Two Kinds of Data: An Example 14
The Power of Qualitative Data 17 Face Validity and Credibility 20
Inquiry by Observation 21 The Raw Data of Qualitative Inquiry 26
People-Oriented Inquiry 27 The Fruit of Qualitative Methods Revisited 28
^ ^ [ f f l g Internet E-mail Discussion Groups (listservs)
on Qualitative Methods 29 Between-Chapters Interlude:
Top Ten Pieces of Advice to a Graduate Student
Considering a Qualitative Dissertation 31 2. Strategic Themes in Qualitative Inquiry 37
General Principies 37 The Purpose of a Strategic Framework 38
Design Strategies for Qualitative Inquiry 39
Naturalistic Inquiry 39 |X]J|||]||9I Themes of Qualitative Inquiry 40
Emergent Design Flexibility 43 Purposeful Sampling 45 Data Collection and Fieldwork: Strategies for Qualitative Inquiry 47
Qualitative Data 47 Direct Personal Experience and Engagement: Going Into the Field 47
Empathic Neutrality 49 Empathy and Insight 51 A Dynamic, Developmental Perspective 54
Analysis Strategies for Qualitative Inquiry 55
Unique Case Orientation 55 Inductive Analysis and Creative Synthesis 55
Holistic Perspective 58 Context Sensitivity 61 Voice and Perspective: Reflexivity 63
Ü i i Reflexive Questions: Triangulated Inquiry 66
From Strategic Ideais to Practical Choices 66 Beyond Competing Inquiry Paradigms 68
Pragmatism 71 Ideal Conditions for Research: A Cautionary Tale 72
Special Gifts 75 From Core Strategies to Rich Diversity 76
Which Approach Is Right? 77 Alternative Ways of Distinguishing Qualitative Traditions 79
Foundational Questions 80 Theoretical Traditions and Orientations 81
Ethnography 81 Í B J M I Culture, Culture Everywhere: Sample of Media Headlines 83
Autoethnography and Evocative Forms of Inquiry 84 l52jm[|lBS9 Varieties of Autoethnography: A Partial Lexicology 85
Truth and Reality-Oriented Correspondence Theory:
Positivist, Realist, and Analytic Induction Approaches 91
Social Construction and Constructivism 91 Constructivism Versus Constructionism 97
Phenomenology 104 Heuristic Inquiry 107 Qualitative Heuristics: A German Alternative Tradition 109
Ethnomethodology 110 Symbolic Interaction 112 Hermeneutics 113 Narra tology or Narrative Analysis 115
Ecological Psychology 118 A Systems Perspective and Systems Theory 119
Chãos and Complexity Theory: Nonlinear Dynamics 123
Grounded Theory 124 — • Complexity (Chãos) Theory Precepts and Qualitative
Inquixy Implications 126 Orientational Qualitative Inquiry: Feminist Inquiry Criticai Theory
and Queer Theory as Examples 129 Variety in Qualitative Inquiry: Different Answers to Core Questions 131
• X O ^ K K I Variety in Qualitative Inquiry: Theoretical Traditions 132
Pragmatism 135 B B Sample Internet E-mail Discussion Groups (listservs) and
Sites Relevant to Qualitative Inquiry and Theory 136
The Apple of Your Eye 137 APPENDIX 3.1. Example of Autoethnographic Writing 138
Apprenticeship in Pragmatism 143 Practical Purposes and Concrete Questions 145
A Focus on Quality 145 Quality Assurance and Program Evaluation 147
^ ^ ^ Q Q O m Comparing Program Evaluation and Quality Assurance 149
Evaluation Applications 151 Outcomes Evaluation 151 Evaluating Individualized Outcomes 152
I I B e h i n d the Numbers of an Employment Program:
The Story of Li 155
Process Studies 159 Implementation Evaluation 161
Logic Models and Theories of Action 162
Evaluability Assessments 163 Comparing Programs: Focus on Diversity 163
155181131111 Types of Teacher Centers 166
Prevention Evaluation 166 Documenting Development Over Time and Investigating
System Changes 167 From Evaluation Issues to Evaluation Models 168
Evaluation Models 169 Goal-Free Evaluation 169 Transaction Models: Responsive and Illuminative Evaluation 171
Connoisseurship Studies 172 Utilization-Focused Evaluation 173 Interactive and Participatory Applications 175
Personalizing and Humanizing Evaluation 175 Harmonizing Program and Evaluation Values 176 Developmental Applications: Action Research, Action Learning,
Reflective Practice, and Learning Organizations 177 j^QQJ^yjQQI Common Principies Undergirding Qualitative Inquiry
and Humanistic Values 177 Matching Program Philosophy and Evaluation Approach:
An Illustration 178
Appreciative Inquiry 181 Participatory Research and Evaluation: Valuing and Facilitating
Collaborative Inquiry 185 Supporting Democratic Dialogue and Deliberation 185
Supporting Democracy Through Process Use: Helping the
Citizenry Weigh Evidence and Think Evaluatively 187
Special Applications 191 The Need for Unobtrusive Measures 191
State-of-the-Art Considerations: Lack of Proven Quantitative
Instrumentation 192 Confirmatory and Elucidating Research: Adding Depth, Detail,
and Meaning to Quantitative Analyses 193
Rapid Reconnaissance 194 Capturing and Communicating Stories 195
Example of a "Most Significant Change" Story 197
Legislative Auditing and Monitoring 198 Futuring Applications: Anticipatory Research and Prospective
Policy Analysis 200 Breaking the Routine: Generating New Insights 202
Summary: A Vision of the Utility of Qualitative Methods 202 Qualitative Inquiry Applications: Summary Checklist
of Particularly Appropriate Uses of Qualitative Methods 204 Sample Internet E-mail Discussion Groups (listservs)
and Sites Relevant to Qualitative Applications and Practice 205
EXHIBIT 4.9
P A R T 2 . Qualitative Designs and Data Collection 207
5. Designing Qualitative Studies 209
The First Evaluation 209 A Meta-Evaluation 211 Clarity About Purpose: A Typology 213
Basic Research 215
EXHIBIT 5.1 Fundamental Disciplinary Questions 216
Applied Research 217 Sample Interdisciplinary Applied Research Questions 218
Evaluation Research: Summative and Formative 218 Action-Oriented, Problem-Solving Research 221 The Purpose of Purpose Distinctions 222
Criticai Trade-Offs in Design 223 A Typology of Research Purposes 224
^QQQQ^IjQQI Family Research Example: Research Questions
Matched to Research Category 225
Breadth Versus Depth 227 Units of Analysis 228 Purposeful Sampling 230
f ã ^ j ^ j j j Q Q Examples of Units of Analysis for Case Studies and
Comparisons 231 Information-Rich Cases 242
Sample Size 242
E&UBBB^^BBBSBÊBBÍ^BBB
— B a Sampling Strategies 243
Emergent Designs and Protection of Human Subjects 246
Methodological Mixes 247 Triangulation 247 Mixing Data, Design, and Analysis Approaches 248
The Case of Operation Reach-Out: Variations in Program
Evaluation Design 249 Altemative Pure and Mixed Strategies 251
E 3 J Measurement, Design, and Analysis: Pure and Mixed Combinations Design and Methods Decisions
Design Issues and Options mSBSBiuSffinSãfiSEmSSHS ^
Choices
6. Fieldwork Strategies and Observation Methods To Understand the World
Folk Wisdom About Human Observation The Value of Direct Observations
Observation-Based Evaluation and Applied Research in a
Political World 265 Variations in Observational Methods 265
Variations in Observer Involvement: Participant or Onlooker or Both? 265 Insider and Outsider Perspectives: Emic Versus Etic Approaches 267 Who Conducts the Inquiry? Solo and Team Versus Participatory
and Collaborative Approaches 269 Overt Versus Covert Observations 273
252 253 254 257 259 260 261 264
Variations in Observational Focus 275 Dimensions Along Which Fieldwork Varies: An Overview 276
What to Observe: A Sensitizing Framework 276 KQ^njnnQn Dimensions Showing Fieldwork Variations 277
Sources of Data 279
E P ^ ^ m Examples of Sensitizing Concepts 280
The Setting 280 • B H J i n Example of Combining Description and Metaphor
to Provide a Sense of Place 282
The Human, Social Environment 283
Historical Perspectives 284 Planned Program Implementation Activities and Formal Interactions 285
Informal Interactions and Unplanned Activities 285
The Native Language of the Program 288
Nonverbal Communication 290 Unobtrusive Observations 291
Documents 293 Observing What Does Not Happen 295
Nested and Layered Case Studies Dnring Fieldwork 297
Observing Oneself 299 Nested, Layered, and Overlapping Mini-Case Studies
During Fieldwork: Example From the Wilderness
Education Program Evaluation 300
Sources of Data Reviewed 301 Creativity in Fieldwork 302 Doing Fieldwork: The Data-Gathering Process 302
Field Notes 302
EXHIBIT 6.5 Fieldnotes Comparisons 304
Procedurally Speaking 305 Observations, Interviews, and Documentation: Bringing Together
Multiple Perspectives 306 The Technology of Fieldwork and Observation 307
Stages of Fieldwork 310 Entry Into the Field 310 What You Say and What You Do 314
Bringing Fieldwork to a Close 322
Evaluation Feedback 324 The Observer and What Is Observed: Unity and Separation 326
The Personal Experience of Fieldwork 329 A Part of and Apart From the World Observed 329
Summary Guidelines for Fieldwork 330 ^ ^ Q H Q Summary Guidelines for Fieldwork 330
Between-Chapters Interlude:
Outside to Inside, Inside to Outside: Shifting Perspectives 333
Preface 335 "Nothing About Us, Without Us" 335
Barbara Lee
Qualitative Interviewing 339 Beyond Silent Observation 339 Rigorous and Skillful Interviewing 340
Inner Perspectives 340 Variations in Qualitative Interviewing 341
The Informal Conversational Interview 342
The Interview Guide 343 The Standardized Open-Ended Interview 344
Q J Q ^ Q Q H Evaluation Interview Guide for Participants in an
Employment Training Program 345
Combining Approaches 347 Summary of Interviewing Strategies 348
Question Options 348 Experience and Behavior Questions 348
Ü M i Variations in Interview Instrumentation 349
Opinion and Values Questions 350
Feeling Questions 350 Knowledge Questions 350 Sensory Questions 350 Background/Demographic Questions 351
Distinguishing Question Types 351 The Time Frame of Questions 351 HxffiHHHKB A Matrix of Question Options 352
Wording Questions 353
Asking Truly Open-Ended Questions 353
The Horns of a Dichotomy 354
Asking Singular Questions 358
Clarity of Questions 361
Why to Take Care Asking "Why?" 363
Rapport and Neutrality 365
Neutral Questions 365
Using Illustrative Examples in Questions 366
Role-Playing and Simulation Questions 367
Presupposition Questions 369
Alternative Question Forma ts 370
Prefatory Statements and Announcements 370
Probes and Follow-Up Questions 371
Process Feedback During the Interview 374
Support and Recognition Responses 375
Maintaining Control and Enhancing the Quality of Responses 375
The One-Shot Question 378
The Final or Closing Question 379
Beyond Technique 379
Mechanics of Gathering Interview Data 380
Recording the Data 380
BSSBFÜFHHHI
Tins fnr Taoe-Rerordine: Interviews: How to KeeoTranscribers Sane 382
Taking Notes During Interviews 383
After the Interview 383
Special Applications and Issues 385
Think-Aloud Protocol Interviewing 385
Focus Group Interviews 385
Group Interviews 390
Cross-Cultural Interviewing 391
Language Differences 392
Differing Norms and Values 393
Beyond Standard Interviewing: Creative Qualitative Modes of Inquiry 394
Participant Interview Chain 396
Data Collection by Program Staff 397
• j M S I U I I I ^ S Training Nonresearchers as Focus Group Interviewers:
Creativity and Data Quality: Qualitative Bricolage 400 Specialized and Targeted Interview Approaches 402 Ethical Challenges in Qualitative Interviewing 405
Informed Consent and Confidentiality 407 i s a y n n i l i s i Ethical Issues Checklist 408 New Directions in Informed Consent: Confidentiality
Versus People Owning Their Own Stories 411 Reciprocity: Should Interviewees Be Compensated? If So, How? 412
How Hard Should Interviewers Push for Sensitive Information? 415
Be Careful. It's Dangerous Out There. 415 Personal Reflections on Interviewing 416
An Interview With the King of the Monkeys 417
Halcolm on Interviewing 418 APPENDIX 7.1. Sample of a Detailed Interview Guide 419
APPENDIX 7.2. Examples of Standardized Open-Ended Interviews 422
P;A~RT
3 . Analysis, Interpretation, and Reporting 4298. Qualitative Analysis and Interpretation 431
The Complete Analysis Isn't 431
The Challenge 432 Purpose as Context 434
When Does Analysis Begin? 436
Thick Description 437 Options for Organizing and Reporting Qualitative Data 439
Organizing the Data 440 Protecting Data 441 Computer-Assisted Qualitative Data Management and Analysis 442
KMHHffifli Examples of Software Programs for Qualitative Analysis 444 I M l i l i Internet Resources and E-mail Discussion Groups (listservs)
on Qualitative Analysis 445
Case Studies 447
UJ2Ü3QE&I C a s e Study: Layers of Possible Analysis 448
From Data to Case Study 449 ü i B i l i i The Process of Constructing Case Studies 450
Inductive and Deductive Qualitative Analyses 453
Indigenous Concepts and Practices 454
Sensitizing Concepts 456 Indigenous Typologies 457 Analyst-Constructed Typologies 458
||||]||||!gj Qualitative Analysis of Ancestry at the U.S. Census 461
The Intellectual and Mechanical Work of Analysis 462 Coding Data, Finding Patterns, Labeling Themes, and Developing
Category Systems 462 I M U f l C T First-Cut Coding Examples: Sample Codes From the
Field Note Margins 464 Convergence and Divergence in Coding and Classifying 465
Determining Substantive Significance 467
Logical Analysis 468 •^TQnnQgl An Empirical Typology of Teacher Roles in Dealing With
High School Dropouts 469
A Process/Outcomes Matrix 471
E B E B I Mapping Stakeholders' Stakes 472
— Conceptual Guide for Data Collection and Analysis:
Utilization of Planmng, Evaluation, and Reporting 473 ^ y O Q Q Q I P Matrix of Linkages Between Program Processes and Impacts 474
An Analysis Example: Recognizing Processes, Outcomes, and
Linkages in Qualitative Data 474
Interpreting Findings 477 Interpreting for Meaning 477 Comparisons, Causes, Consequences, and Relationships 478
Theory-Based Analysis Approaches 481 Phenomenological Analysis 482
Grounded Theory 487 Qualitative Comparative Analysis 492
Analytic Induction 493 Special Analytical Issues and Frameworks 494
Reflexivity and Voice 494 Collaborative and Participatory Analyses 496
The Hermeneutic Circle and Interpretation 497 Analyzing Institutional Documents 498
Finding Nothing 500 Synthesizing Qualitative Studies 500
Reporting Findings 502 Balance Between Description and Interpretation 503
Communicating With Metaphors and Analogies 504
Drawing Conclusions 506 Special Issues in Evaluation Reporting and an Example 506
Feedback and Analysis 506 Evaluative Feedback Using Indigenous Typologies 507
Q J J ^ Q j J Q Distinguishing Observations From Perceived Impacts 509
To Write a Report or Not to Write a Report? 510
Focus 511 The Executive Summary and Research Abstract 511
I B g M i l l Utilization-Focused Evaluation Reporting 512
Carpe Diem Briefings 512 The Creativity of Qualitative Inquiry 512
The Past and the Future: Deciding in Which Direction to Look 515 APPENDIX 8.1. Excerpts From a Codebook for Use by Multiple Coders 516
APPENDIX 8.2. Mike: An Dlustrative Case Study 518 APPENDIX 8.3. Excerpts From an Illustrative Interview Analysis:
Reflections on Outcomes From Participants in a
Wilderness Education Program 525 Between-Chapters Interlude:
Riddles of Qualitative Inquiry: Who Am I? 537
Gary D. Shank
9. Enhancing the Quality and Credibility of Qualitative Analysis 541
Interpreting Truth 541 Alternative Criteria for Judging Quality 542
^^QQQyQQ Alternative Sets of Criteria for Judging the Quality
and Credibility of Qualitative Inquiry 544
Traditional Scientific Research Criteria 544 Social Construction and Constructivist Criteria 546
Artistic and Evocative Criteria 547 Criticai Change Criteria 548 Evaluation Standards and Principies 549
Credibility 552 Rigor: Strategies for Enhancing the Quality of Analysis 553
Integrity in Analysis: Generating and Assessing Rival Conclusions 553
Negative Cases 554 Triangulation 555 I S f f l M K E l A Story of Triangulation: Testing Conclusions With
More Fieldwork 559 Design Checks: Keeping Methods and Data in Context 563
High-Quality Lessons Learned 564 I^Sn^QEQI High-Quality Lessons Leamed 565
The Credibility of the Researcher 566 Considering Investiga tor Effects: Varieties of Reactivity 567
Intellectual Rigor 570 The Paradigms Debate and Credibility 570
Beyond the Numbers Game 572 Beyond Objectivity and Subjectivity: New Concepts, New Language 574
Reflections on Truth and Utility as Criteria of Quality 577 From Generalizations to Extrapolations and Transferability 581 The Credibility Issue in Retrospect: Increased Legitimacy for
Qualitative Methods 584 Beyond the Qualitative-Quantitative Debate 584
Matching Claims and Criteria 587 APPENDIX 9.1. Case Study: A Documenter's Perspective 589
Refere nc es RI Author Index II Subject Index 113 About the Author Al
Preface.
T
he story is told that at the conclusion of a rigorous course in philoso-phy, one of the students lamented: "Professor, you have knocked a hole in everything I've ever believed in, but you have given me nothing to take its place."To which the philosopher replied: "You will recall that among the labors of Hercules he was required to clean out the Augean stables. He was not, let me point out, required to fill them,"
W
hile part of the task of this revi-sion has been to clean out the qualitative Augean stables, the truly Herculean task has been deciding what to add. Unlike the professor who can be content with getting the stables cleaned, the author of a revision bears responsibility for restocking the stables with fresh nutri-ents and feed, a task made especially chal-lenging because of the unprecedented blossoming of qualitative inquiry in recent years.In doing this revision, I reviewed over a thousand new books on qualitative meth-ods, program evaluation, case studies, monographs, and related works published in the last decade, as well as hundreds of ar-ticles scattered through scores of journals covering the full range of disciplines and professions. Two important new qualita-tive journals— Qualitaqualita-tive Inquiry and Field
Methods—began publication, as did spe-cialized qualitative journals m a number of professions (e.gv health, nursing, social
work, organizational development) and some devoted to specific approaches (e.g.,
Grounded Theory Reviezu). The Handbook of
Qualitative Research was published (1994), as was a revision (2000), and the Handbook of
Methods in Cultural Anthropology (1998) made its debut. Sophisticated new software programs have been developed to support qualitative analysis. Internet listservs have emerged to facilitate dialogue. The Hercu-lean challenge has been analyzing this geo-metric growth to determine primary trends, patterns, and themes. The results of that analysis are reflected throughout this new edition.
The first edition of this book (1980), enti-tled Qualitative Evaluation Methods, focused on the variety of ways in which qualitative methods were being applied in the then still-emergent profession of program evalu-ation. (The American Evaluation Associa-tion was not established until 1984.) That edition appeared in the midst of the heated qualitative-quantitative debate about the relative value of different methods and al-ternative paradigms. The second edition (1990), entitled Qualitative Evaluation and
Re-search Methods, was influenced by maturing of the paradigms debate. It included much more attention to the ways in which differ-ent theoretical and philosophical perspec-tives influenced qualitative inquiry, as well as the greater range of applications in eval-uation as that profession blossomed. This latest edition involves yet another change of title, Qualitative Research and Evaluation
Methods, reflecting the degree to which de-velopments in qualitative inquiry during the la st decade have been driven by a diver-sifying research agenda and scholarly dia-logue, much of which has found its way into evaluation, to be sure.
The classic qualitative-quantitative de-bate has been largely resolved with recogni-tion that a variety of methodological
ap-proaches are needed and credible, that mixed methods can be especially valuable, and that the challenge is to appropriately match methods to questions rather than ad-hering to some narrow methodological or-thodoxy. With less need to establish the value of qualitative inquiry by debating those of quantitative and experimental per-suasion, qualitative inquirers have turned their attention to eacli other, noticing that they are engaging in different kinds of quali-tative inquiry from widely different per-spectives. Qualitative methodologists and theorists have thus taken to debating each other. The upshot of ali the developmental work in qualitative methods is that there is now as much variation among qualitative researchers as there is between qualitatively and quantitatively oriented scholars and evaluators. A primary purpose of this new edition is to sort out the major perspectives in that debate, portray the diversity of quali-tative approaches now available, and exam-ine the influences of this diversity on appli-cations, especially but not exclusively in program evaluation, which has experienced a parallel flowering of diversity and atten-dant controversies about new directions.
!=l Organization of This Edition
Chapter 1 provides a range of examples of qualitative findings. I begin by presenting a number of significant illustrations of the fruit of qualitative inquiry, in order to give a taste of what results from qualitative stud-ies and help those new to such inquiry know where they are headed and what they are trying to produce. Chapter 2 reviews and adds to the primary strategic themes that define qualitative inquiry. Chapter 3 exam-ines different qualitative approaches, in-cluding several that have emerged
dis-tinctly in the last decade. Chapter 4 presents a wide range of qualitative applications, many of them new, in evaluation, action re-search, and organizational, community, and international development. Chapters 5, 6, and 7 cover design and data gathering, of-fering guidance in purposeful sampling, mixed methods, fieldwork, observational approaches, and interviewing, with special attention directed to the skills and compe-tencies needed to gather high-quality data. Chapter 8 provides direction and processes for analyzing qualitative data, always the most challenging aspect of this work. Finally, Chapter 9 de ais with paradigms, politics, and ways of enhancing the credibil-ity of qualitative inquiry. This chapter also presents what I consider to be the five dis-tinct and competing frameworks for under-taking and judging the quality of qualitative studies: traditional scientific research crite-ria; social construction and constructivist criteria; artistic and evocative criteria; criti-cai change criteria; and pragmatic, utility-oriented evaluation standards and princi-pies. Along the way I've added, as is my wont, hundreds of new stories and exam-ples. I've also created over 50 new exhibits that summarize and illuminate major points.
t£I. Acknowledgments
I began this preface by noting the Herculean task of revision given the enormous growth and increased diversity of qualitative in-quiry. One task proved more than Hercu-lean, and I could not complete it. I began to list the many colleagues and evaluation cli-ents to whom I am indebted and who de-serve acknowledgement for their contribu-tions to my understanding and writing over the years. Now that I have reached this third
edition and traveled many qualitative miles, the list of those to whom I am in-debted is too long and the danger of leaving out important influences too great for me to include such traditional acknowledge-ments here. I can only refer the reader to the references and stories in the book as a start-ing point.
I must, however, acknowledge cartoonist Michael Cochran of Tupper Lake, New York, who drew the many new illustrations in-clude d here to lighten the reader's way along this journey. Our collaboration began at a Union Institute research methods semi-nar he took while pursuing his doctorate in professional psychology. His evaluation of the seminar include d cartoons. I liked his wit and style, so I offered ideas for cartoons on qualitative inquiry and he turned them into art. Doing the cartoons, he told me, was a wonderful distraction from writing his dis-sertation. Fm grateful for his humor and tal-ent.
The editorial and production staff at Sage Publications deserve special mention. Only fellow authors who have struggled with edi-tors of limited vision and understanding can fully appreciate what it means to work with C. Deborah Laughton, an experienced and knowledgeable editor who not only knows qualitative methods and evaluation as deeply as any practitioner of these arts, but also has significantly shaped those fields by conceptualizing works that she saw a need for and then nurtuiing authors, which she does better than any editor I've ever known, to assure that those works came to fruition. That she is also a writer and designer has made our working together a genuine col-laboration. Kate Peterson's skilled copyedit-ing added appreciably to the final product as she made many suggestions for improving clarity and readability, and I came to trust and even rely on her unusual eye for detail Janelle Lemaster's interior design work
con-verted the raw maimscript into the carefully crafted book you now hold. Diana Axelsen pulled it ali together as production editor to get the book launched on schedule. I came to count on not only her great management competence but also her good humor. Finally, in acknowledging the superb Sage production team, I should also reprise the preface to the first edition in which I noted that my initial foray into qualitative writing was due entirely to the persuasive powers of Sara Miller McCune, co-founder of Sage Publications, who had shepherded the first edition of Utilization-Focused Evaluation (1978) into print and, based on the perspective in that book, urged me during a trip to Minne-sota in 1978 to write a qualitative compan-ion. Her vision and follow-through have made Sage Publications the leading pub-lisher of both evaluation and qualitative in-quiry books.
With the reader's indulgence, and by way of further providing a historical context for this third edition, permit me to include an excerpt from that first preface so maxty years ago:
As other authors know, there is no way to re-ally recognize the contribution of one's family to a book like this, the writing of which was a struggle and matter of endurance for both family and author. While Sara Miller McCune was persuading me that the book should be written, Jeanne was persuading me that we could nurture together both a new book and a
newborn child. (Having been left out of that decision, the newborn child subsequently made it clear he didn't always agree.) The
con-tribution of Jeanne to the book exemplifies why the personal and professional sometimes cannot and ought not be separated. Jeanne's reflections on her own evaluation fieldwork and interviewing experiences helped me clar-ify and break through some particularly diffi-cult sections of the book. Her editorial advice was invaluable. Those were her tangible con-tributions; the intangibles she contributed are the things that made the book happen.
Those intangibles and Jeanne's ongoing support have remained the mainstay of my writing. Meanwhile, the newborn child re-ferred to above, Quinn Campbell, has com-pleted a master's degree in engineering, and his younger sister, Charmagne Campbell-Patton, is about to complete college. As this was being completed, their older brother, Braxidon Patton, participated in a two-day workshop I conducted on qualitative meth-ods in preparation for his first evaluation fieldwork, a sideline he has tumed to as a way of supporting his real passion, writing and performing rock music. Thus have the years passed, love maturing and children growing, bringing forth the need to revise the old, celebrate the new, and clean out the qualitative Augean stables while restocking them with fresh nutrients. It is those nutri-ents that follow.
p y v R T 1
Conceptual Issues
in Qualitative Inquiry
• Psychometricians try to measure it Experimentalists try to control it. Interviewers ask questions about it. Observers watch it.
Participant observers do it. Statisticians count it. Evaluators value it.
Qualitative inquirers find meaning in it. • When in doubt, observe and ask questions.
When certain, observe at length and ask many more questions. • Gigo's law of deduction: Garbage in,, garbage out.
Halcolm/s law of induction: No new experience, no new insight. • Qualitative inquiry cultivates the most useful of ali human capacities:
The capacity to learn.
• Innovators are told: "Think outside the box."
Qualitative scholars tell their students: "Study the box. Observe it. Inside. Out-side. From inside to outside, and outside to inOut-side. Where is it? How did it get there? What's around it? Who says it's a 'box'? What do they mean? Why does it matter? Or does it? What is not 'box'? Ask the box questions. Question others about the box. What's the perspective from inside? From outside? Study diagrams of the box. Find documents related to the box. What does thinking have to do with the box anyway? Understand this box. Study another box. And another. Under-stand box. UnderUnder-stand. Then you can think inside and outside the box. Perhaps. For awhile. Until it changes. Until you change. Until outside becomes inside —again. Then start over. Study the box."
• There is no burden of proof. There is only the world to experience and understand. Shed the burden of proof to lighten the load for the journey of experience.
The Nature of Qualitative Inquiry
There once lived a man in a country with no fruit trees. A scholar, he spent a great deal of time reading. He often carne across references to fruit. The descrip-tions enticed him to undertake a journey to experience fruit for himself.
He went to the marketplace and inquired where he could find the land of fruit. After much searching he located someone who knew the way. After a long and arduous journey he came to the end of the directions and found himself at the entrance to a large apple orchard. It was springtime and the apple trees were in blossom.
The scholar entered the orchard and, expectantly, pulled off a blossom and put it in his mouth. He liked neither the texture of the flower nor its taste. He went quickly to another tree and sampled another blossom, and then another, and another. Each blossom, though quite beautiful, was distasteful to him. He left the orchard and returned to his home country, reporting to his fellow villag-ers that fruit was a much overrated food.
Being unable to recognize the difference between the springblossom and the summer fruit, the scholar never realized that he had not experienced what he was looking for.
TThe F ^ u i t o f Q u a l i t a t i v e j M e ^ o d s
—From Halcolm's Inquiry Parables
Three Kinds of Qualitative Data
Interviews
Open-ended questions and probes yield in-depth responses about people's experiences, per-ceptions, opinions, feelings, and knowledge. Data consist of verbatim quotations with sufficient context to be interpretable.
Observatíons
Fieldwork descriptions of activities, behaviors, actions, conversations, interpersonai interac-tions, organizational or community processes, or any other aspect of observable human experi-ence. Data consist of field notes: rich, detailed descriptions, including the context within which the observatíons were made.
Documents
Written materiais and other documents from organizational, clinicai, or programs records; memoranda and correspondence; offícial publications and reports; personal diaries, letters, artis-tic works, photographs, and memorabilia; and written responses to open-ended surveys. Data con-sist of excerpts from documents captured in a way that records and preserves context.
Recognizing Qualitative Data
This book discusses how to collect, ana-lyze, and use qualitative data. To begin, let's examine the fruit of qualitative methods. It is important to know what qualitative data and findings look like so that you will know what you are seeking. It will also be impor-tant to consider criteria for judging the qual-ity of qualitative data. Apples come to mar-ket sorted by type (Red Delicious/ Golden),
purpose (e.g., cooking or eating), and qual-ity. Likewise, qualitative studies vary by type, purpose, and quality.
Qualitative findings grow out of three kinds of data collection: (1) in-depth, open-ended interviews; (2) direct observation; and (3) written documents. Interviews yield direct quotations from people about their ex-periences, opinions, feelings, and knowl-edge. The data from observatíons consist of detailed descriptions of people's activities, behaviors, actions, and the full range of in-terpersonai interactions and organizational
processes that are part of observable hu-man experience. Document analysis includes studying excerpts, quotations, or entire pas-sages from organizational, clinicai, or pro-gram records; memoranda and correspon-dence; official publications and reports; personal diaries; and open-ended written responses to questionnaires and surveys. (See Exhibit 1.1.)
The data for qualitative analysis typically come from fieldwork. During fieldwork, the researcher spends time in the setting under study—a program, an organization, a com-munity, or wherever situations of impor-tance to a study can be observed, people in-terviewed, and documents analyzed. The researcher makes firsthand observa tions of activities and interactions, sometimes en-gaging personally in those activities as a
par-ticipant observer. For example, an evaluator might participa te in ali or part of the pro-gram under study, participating as a regular program member, client, or student. The qualitative researcher talks with people
about their experiences and perceptions. More formal individual or group interviews may be conducted. Relevant records and documents are examined. Extensive field notes are collected through these observa-tions, interviews, and document reviews. The voluminous raw data in these field notes are organized into readable narrative descriptions with major themes, categories, and illustrative case examples extracted through content analysis. The themes, pat-terns, understandings, and insights that emerge from fieldwork and subsequent analysis are the fruit of qualitative inquiry.
Qualitative findings may be presented alone or in combination with quantitative data. Research and evaluation studies em-ploying multiple methods, including combi-nations of qualitative and quantitative data, are common. At the simplest levei, a ques-tionnaire or interview that asks both fixed-choice (closed) questions and open-ended questions is an example of how quantitative measurement and qualitative inquiry are of-ten combined.
The quality of qualitative data depends to a great extent on the methodological skill, sensitivity, and integrity of the re-searcher. Systematic and rigorous obser-vation involves far more than just being present and looking around. Skillful inter-viewing involves much more than just asking questions. Content analysis re-quires considerably more than just read-ing to see what's there. Generatread-ing useful and credible qualitative findings through observation, interviewing, and content analysis requires discipline, knowledge, training, practice, creativity, and hard work.
This chapter provides an overview of qualitative inquiry. La ter chapters exam-ine how to choose among the many op-tions available within the broad range of qualitative methods, theoretical perspec-tives, and applications; how to design a qualitative study; how to use observa-tional methods and conduct in-depth, open-ended interviews; and how to ana-lyze qualitative data to generate findings.
& Qualitative Findings: Themes, Patterns, Concepts, Insights, Understandings
ewton and the apple. Freud and anxiety. Jung and dreams. Piaget and his children. Darwin and Galapagos tortoises. Marx and
England's factories. Whyte and street corners. What are you obsessed with? —Halcolm
Mary Field Belenky and her colleagues set out to study women's ways of knowing. They conducted extensive interviews with 135 women from diverse backgrounds prob-ing how they thought about knowledge, au-thority, truth, themselves, life changes, and life in general. They worked as a team to group similar responses and stories
to-gether, informed partly by previous re-search but ultimately basing the analysis on their own collective sense of what cate-gories best captured what they found in the narrative data. They argued with each other about which responses belonged in which categories. They created and aban-doned categories. They looked for
com-Discovery of an early qualitative evaluation report
monalities and differences. They worked hard to honor the diverse points of view they found while also seeking patterns across sto-ries, experiences, and perspectives. One theme emerged as particularly powerful: "Again and again women spoke of 'gaining voice' " (Belenky et al. 1986:16). Voice versus silence emerged as a central metaphor for
irtforming variations in ways of knowing. After painstaking analysis, they ended up with the five categories of knowing summa-rized in Exhibit 1.2, a framewbrk that be-caine very influential in women's studies and represents one kind of fruit from quali-tative inquiry.
WomerTs Ways of Knowing:
An Example of Qualitative Findings
Silence: A position in which women experience themselves as mindless and voiceless and
subject to the whims of externai authority.
Received knowledge: Women conceive of themselves as capable of receiving, even reproducing
knowledge from externai authorities, but not capable of creating knowledge on their own.
Subjective knowledge: A perspective from which truth and knowledge are conceived as
persona!, private, and subjectively known or intuited.
Procedural knowledge: Women are invested in learning and apply objective procedures for
obtaining and communicatíng knowledge.
Constructedknowledge: Women view ali knowledge as contextual, experience themselves as
creators of knowledge, and value both subjective and objective strategies for knowing. SOURCE: Belenky et al. (1986:15).
One of the best-known and most influen-tial books in organizational development and management is In Search of Excellence:
Lessons From America's Best-Run Companies.
Peters and Waterman (1982) based the book on case studies of 62 highly regarded com-panies. They visited companies, conducted extensive interviews, and studied corporate documents. From that massive amount of data they extracted eight attributes of excel-lence: (1) abias for action; (2) close to the cus-tomer; (3) autonomy and entrepreneurship; (4) productivity through people; (5) hands-on, value-driven; (6) stick to the knitting; (7) simple form, lean staff; and (8) simultaneous loose-tight properties. Their book devotes a chapter to each theme with case examples and implications. Their research helped launch the quality movement that has now moved from the business world to not-for-profit organizations and government. This study also illustrates a common quaHtative sampling strategy: studying a relatively small number of special cases that are suc-cessful at something and therefore a good source of lessons learned.
Stephen Covey (1990) used this same sampling approach in doing case studies of "highly effective people." He identified seven habits these people practice: (1) being proactive; (2) beginning with the end in mind; (3) putting first things first; (4) think-ing win/ win; (5) seekthink-ing first to understand, then seeking to be understood; (6) syner-gizing, or engaging in creative cooperation; and (7) self-renewal.
Both of these best-selling books, In Search
of Excellence and The 7 Habits of Highly
Effec-tive People, distill a small number of impor-tant lessons from a huge amount of data based on outstanding exemplars. It is com-mon in qualitative analysis for mounds of field notes and months of work to reduce to a small number of core themes. The quality of the insights generated is what matters, not the number of such insights. For example, in an evaluation of 34 programs aimed at peo-ple in poverty, we found a core theme that separated more effective from less effective programs: How people are treated affects how they treat others. If staff members are treated autocratically and insensitively by
management, with suspicion and disre-spect, staff will treat clients the same way. Contrariwise, responsiveness reinforces re-sponsiveness, and empowerment breeds empowerment. These insights became the centerpiece of subsequent cross-project, col-laborative organizational and staff develop-ment processes.
A different kind of qualitative finding is illustrated by Angela Browne's book When
Battered Women Kill (1987). Browne con-ducted in-depth interviews with 42 women from 15 states who were charged with a crime in the death or serious injury of their mates. She was often the first to hear these women's stories. She used one couple's his-tory and vignettes from nine others, repre-sentative of the entire sample, to illuminate the progression of an abusive relationship from romantic courtship to the onset of abuse through its escalation until it was on-going and eventually provoked a homicide. Her work helped lead to legal recognition of battered women's syndrome as a legiti-mate defense, especially in offering insight into the common outsider's question: Why doesn't the woman just leave? An insider's perspective on the debilitating, destructive, and all-encompassing brutality of battering reveals that question for what it is: the facile judgment of one who hasn't been there. The effectiveness of Browne's careful, detailed, and straightforward descriptions and quo-tations lies in their capacity to take us inside the abusive relationship. Offering that in-side perspective powers qualitative report-ing.
Clark Moustakas (1995), a humanistic psychologist and phenomenologist, also gives us an insider's perspective: his own. An astute and dedicated observer of relationships, especially therapeutic rela-tionships, he drew deeply on his own expe-riences and clinicai cases to identify,
dis-tinguish, and elaborate three primary pro-cesses that contribute to the development of a relationship: "Being-In," "Being-For," and "Being-With."
• Being-In involves immersingoneself in another's world: listening deeply and atten-tively so as to enter into the other person's experience and perception. "I do not select, interpret, advise, or direct. . . . Being-In the world of the other is a way of going wide open, entering in as if for the first time, hear-ing just what is, leavhear-ing out my own thoughts, feelings, theories, biases— I enter with the intention of understanding and ac-cepting perceptions and not presenting my own view or reactions 1 only want to en-courage and support the other person's ex-pression, what and how it is, how it came to be, and where it is going." (Moustakas 1995: 82-83)
• Being-For involves taking a stand in support of the other person, being there for the other. "I am listening. I am also offering a position, and that position has an element of my being on that person's side, against ali others who would minimize, deprecate, or deny this person's right to be and to grow.... I become an advocate of the person with ref-erence to his or her frustrations and prob-lems in dealing with others." (Moustakas 1995:83)
• Being-With involves being present as one's own person in relation to another per-son, bringing one's own knowledge and ex-perience into the relationship. "This may in-volve disagreeing with the other's ways of interpreting or judging or presenting some àspect of the world. Being-With means lis-tening and hearing the other's feelings, thoughts, objectives, but it also means offer-ing my own perceptions and views. There is, in Being-With, a sense of joint enterprise —two people fully involved, struggling, ex-ploring, sharing." (Moustakas 1995:84)
IjfPlIH''™™"®®1'®® Coming-of-Age Paradigms
Dimensions
of Comporiso n Tribal Initiotion Modern Corning ofAgç
View of life passages One-time transition from child Multiple passages over a lifetime
to aduit journey
Territory Tribal territory Earth: Global community
Ancestry Creation myth Evolutionary story of humankind
Identity Becoming a man or woman Becoming a complete person
Approach Standardized Individualized
Outcome Tribe-based identity Person a lity identity: Sense of self
Message You are first and foremost a You are first and foremost a person
member of the tribe in your own right
SOURCE: Patton (I999a:333, 335).
Qualitative findings of ten have this sim-ple yet elegant and insightful character. This straightforward yet nuanced framework represents a creative synthesis of years of participant observation and personal in-quiry. Through cases, dialogues, quotations, cases, and introspective reflections, Mous-takas illuminates the process of moving froin Being-In to Being-For and ultimately Being-With. His work exemplifies the con-tribution of phenomenological inquiry to humanistic psychology.
Still a different format for capturing and reporting qualitative findings is illustrated by my own inquiry into alternative coming-of-age approaches. I used the device of con-structing ideal-typical alternative para-digms to compare and contrast what I learned (Patton 1997a). Exhibit 1.3 provides a sampling of contrasts between traditional tribe-centered initiations and modern youth-centered coming-of-age celebrations. These kinds of polar contrasts can sometimes set up a Hegelian dialectic of thesis and
antithe-sis that leads to a new syntheantithe-sis. In philoso-phy such contrasts derive from the rumina-tions of philosophers; in qualitative research such thematic contrasts emanate from and are grounded in fieldwork.
This quick sampling of the fruit of quali-tative inquiry is meant, like a wine tasting, to demonstrate choices toward developing a more sophisticated palate, or like appetiz-ers, as an opening to the fuller feast yet to come. The next section discusses some of the different research and evaluation purposes that affect what kind of fruit results from qualitative inquiry and how the quality of that fruit is judged.
Different Purposes of and
Audiences for Qualitative Studies: Research, Evaluation, Dissertations, and Personal Inquiry
As the title of this book indicates, qualita-tive methods are used in both research and
evaluation. But because the purposes of re-search and evaluation are different, the crite-ria for judging qualitative studies can vary depending on purpose. This point is impor-tant. It means one can't judge the appropri-ateness of the methods in any study or the quality of the resulting findings without knowing the study's purpose, agreed-on uses, and intended audiences. Evaluation and research typically have different pur-poses, expected uses, and intended users. Dissertations add yet another layer of com-plexity to this mix. Let's begin with evalua-tion.
Program evaluation is the systematic col-lection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, im-prove program effectiveness, and/or inform decisions about future prograrmning. Pol-icies, organizations, and personnel can also be evaluated. Evaluative research, quite broadly, can include any effort to judge or enhance human effectiveness through sys-tematic data-based inquiry. Human beings are engaged in ali kinds of efforts to make the world a better place. These efforts in-clude assessing needs, formulating policies, passing laws, delivering programs, manag-ing people and resources, providmanag-ing ther-apy, developing communities, changing or-ganizational culture, educating students, intervening in conflicts, and solving prob-lema. In these and other efforts to make the world a better place, the question of whether the people involved are accomplishing what they want to accomplish arises. When one examines and judges accomplishments and effectiveness, one is engaged in evaluation. When this examination of effectiveness is conducted systematically and empirically through careful data collection and thought-ful analysis, one is engaged in evaluation
re-search.
Qualitative methods are often used in evaluations because they tell the program's
story by capturing and communicating the
participants' stories. Evaluation case studies have ali the elements of a good story. They tell what happened, when, to whom, and with what consequences. Many examples in this book are drawn from program evalua-tion, policy analysis, and organizational de-velopment. The purpose of such studies is to gather information and generate findings that are useful. Understanding the pro-gram^ and participants' stories is useful to the extent that they illuminate the processes and outcomes of the program for those who must make decisions about the program. In
Utüization-Focused Evaluation (Patton 1997a), I presented a comprehensive approach to doing evaluations that are useful, practical, ethical, and accurate. The primary criterion for judging such evaluations is the extent to which intended users actually use the find-ings for decision making and program im-provement. The methodological implication of this criterion is that the intended users must value the findings and find them credi-ble. They must be interested in the stories, experiences, and perceptions of program participants beyond simply knowing how many came into the program, how many completed it, and how many did what after-ward. Qualitative findings in evaluation il-luminate the people behind the numbers and put faces on the statistics, not to make hearts bleed, though that may occur, but to deepen understanding.
Research, especially fundamental or basic research, differs from evaluation in that its primary purpose is to generate or test theory and contribute to knowledge for the sake of knowledge. Such knowledge, and the theo-ries that undergird knowledge, may subse-quently inform action and evaluation, but action is not the primary purpose of
funda-ÜUAÜA.'-' . • ; . j : .:,• „„.••,.,V
tfíLftt k>$X"- !;=: ! ; : í ^rrtsThil.8 st! .'.:, í.-iV"i L-I !.'i.'! u !:'*
f í l W J - í f O T - n Ti Vjji:" iVy.Ti ji '•: !'v!:.' i • w! i - u i t :.
;,!=:; j:-n j;! jT!t:! !;= iftj?^ffcs- ri h ! >n «!' i ! ! , ii r 1V1 m \ \ • • 'i;
.'-=.'= j:" !.!:::-."-».;!!" i-,v:V: , i ijYi v; ,i.v. jjVi!.vü i! nJ ií -.•fl Ii-I j:;; iYi;Mi!::!;h!ni:ir! táfktiyr: siri" i n - ii r :.=: í !.Vi:. U.i
h:i -inl^rí-É^r-jii.H^^rh^n^í^rr ^i-i-íi:3!-!." ;. vlY: i-vkz! q.!.a IY-.h.í;
ÍV':i!;:Í:: ^ . n a /-.'n, .•«• .•ÍÜ.ÍVMV
'.YPOV :;Í":Í' rçivrjiM.- umIÍYÜ! ítí- ms...
i!l'i -!:•!' -•=:"':'!•= ü\>!l =!'=:.!:liY/rviü|i !:i="i!=i' w r t ü • •
!:••« ' ; ü : i".;-!,!":: í:! v ; - S t i " í = í - : ;!- t : í n « m n V i-iv.ri.i.1 v ü
-ÍX-'.;>:•:;. :iÍT= -v;: !:= ! 6 * 7 ' ' r v t V ",i Vr-i
i=r.':-i:'!' n íí r r o ';ü M t t f i t f . ; d :ni I * 1 !•"« i i'!l»
ü í. ,', !• í ' ' • » • : , : ; . 1 « o • r í - vj ? 1 < " ' r r > m • «• ; i ' ":•!:> V : » ; •'! , i 1 :" f 11-3-=- [ !<' 1 VI' I. i >
i.i-.iiv.li,::' 'V',iVi!:oi:r."!-. mvi-'' j iW!.',.;"';,::•:;::,:,!• yj.y .VM-V ri t: IV!; «• ri ,1 •,! ,!.;: j • -v. •,, ,-i....,,-j- vi •,,,,.,,,
11 oIA'', i!* 1 ".:Í !: -'! ::•',• m iyi ••«i i :=• n •• 11 s:-! r.ii.' Í::;i.Í:ÍÍ:M;.' r:'.!•• Í"-:1.: .r- r
: cti«11.: Í1 :.i:V!!! i:,11 :Í • vr!T! : \ i 'o «!i v, i =i i vi i
.-'IA-;!.;!.n.:iiV!..;• 'j.'ií- I:I.-i "
i!:>i!i li!'.•!!!!!!!.' ! P ! . ! ! i ' " t ' ! « : *1" > » ' ' í i .•»
, i ; i i :.• !:;•.'!• ü-. ^ o >• !'•' :;jxia > .:;>.
mental research. Qualitative inquiry is espe-cially powerful as a source of grounded the-ory, theory that is inductively generated from fieldwork, that is, theory that emerges from the researcher's observations and in-terviews out in the real world rather than in the laboratory or the academy. The primary audiences for research are other researchers and scholars, as well as policymakers and others interested in understanding some phenomenon or problem of interest. The re-search training, methodological prefer-ences, and scientific values of those who use research will affect how valuable and credi-bíe they find the empirical and theoretical fruit of qualitative studies.
Dissertations and graduate theses offer special insight into the importance of atten-tion to audience. Savvy graduate students learn that to complete a degree program, the studenfs committee must approve the work. The particular understandings, val-ues, preferences, and biases of committee members come into play in that approval process. The committee will, in essence, evaluate the studenfs contribution,
includ-ing the quality of the methodological proce-dures followed and the analysis done. Qual-itative dissertations, once quite rare, have become increasingly common as the criteria for judging qualitative contributions to knowledge have become better understood and accepted. But those criteria are not abso-lute or universally agreed on. As we shall see, there are many varieties of qualitative inquiry and multiple criteria for judging quality, many of which remain disputed.
While the precedmg discussion of evalua-tion, research, and dissertations has empha-sized taking into account externai audiences and consumers of qualitative studies, it is also important to acknowledge that you may be the primary intended audience for your work. You may study something because you want to understand it. As my children grew to adulthood, I found myself asking questions about coming of age m modern society so I undertook a personal inquiry that became a book (Patton 1997a), but I didn't start out to write a book. I started out trying to understand my own experience and the experiences of my children. That is a form of
qualitative inquiry. While doing interviews with recipients of MacArthur Foundation Fellowships (popularly called "Genius Awards"),1 was told by a social scientist that her fieldwork was driven by her own search for understanding and that she disciplined herself to not even think about publication while engaged in interviewing and observ-ing because she didn't want to have her in-quiry affected by attention to externai audi-ences. She wanted to know because she wanted
to know, and she had made a series of career and professional decisions that allowed her to focus on her personal inquiry without be-ing driven by the traditional academic ad-monition to "publish or perish." She didn't want to subject herself to or have her work influenced by externai criteria and judg-ment.
In summary, ali inquiry designs are af-fected by intended purpose and targeted au-dience, but purpose and audience deserve special emphasis in the case of qualitative studies, where the criteria for judging qual-ity may be poorly understood or in dispute, even among qualitative methodologists. This book cannot resolve these debates, but it will illuminate the methodological op-tions and their implicaop-tions. (Chapter 9 dis-cusses alternative criteria for judging the quality of qualitative studies.)
Making Methods Decisions
The implication of thinking about pur-pose and audience in designing studies is that methods, no less than knowledge, are dependent on context. No rigid rules can prescribe what data to gather to investigate a particular interest or problem. There is no recipe or formula in making methods deci-sions. Widely respected psychometrician Lee J. Cronbach has observed that design-ing a study is as much art as science. It is "an exercise of the dramatic imagination" (Cronbach 1982:239). In research as in art, there can be no single, ideal standard. Beauty no less than "truth" is in the eye of the beholder, and the beholders of research and evaluation can include a plethora of stakeholders: scholars, policymakers, fund-ers, program managfund-ers, staff, program par-ticipants, journalists, critics, and the general public. Any given design inevitably reflects some imperfect interplay of resources, capa-bilities, purposes, possicapa-bilities, creativity, and personal judgments by the people involved.
Research, like diplomacy, is the art of the possible. Exhibit 1.4 provides a set of ques-tions to consider in the design process, re-gardless of type of inquiry. With that back-ground, we can tum to consideration of the relative strengths and weaknesses of quali-tative and quantiquali-tative methods.
13. Methods Choices: Contrasting
Qualitative and Quantitative Emphases
ot everything that can be counted counts, and not everything that counts can be counted.
EXHIBIT 1.4
Some Guiding Questions and Options for Methods Decisions1. What are the purposes of the inquiry?
Research: Contribution to knowledge
Evaluation: Program improvement and decision making Dissertation: Demonstrate doctoral-levei scholarship Personal inquiry: Find out for oneself
2. Who are the primary audiences for the findings? Scholars, researchers, academicians
Program funders, administrators, staff, participants Doctoral committee
Oneself, friends, family, lovers
3. What questions wíll guide the inquiry?
Theory-derived, theory-testing, and/or theory-oriented questions Practical, applied, action-oriented questions and issues
Academic degree or discipíine/specialization priorities Matters of personal interest and concern, even passion 4. What data will answer or illuminate the inquiry questions?
Qualitative: Interviews, field observatíons, documents Quantitative: Surveys, tests, experíments, secondary data Mixed methods: What kind of mix? Which methods are primary?
5. What resources are avaiiable to support the inquiry? Financial resources
Time
People resources Access, connections
6. What criteria will be used to judge the quality of the findings?
Traditional research cr/ter/o; Rigor, validíty, relíabilíty, generalizabiíity Evaluatian standards: Utility, feasibílíty, propriety, accuracy
Nontraditional criteria: Trustworthiness, diversity of perspectives, clarity of voíce, credibility
of the inquirer to primary users of the findings
Thinking about design alternatives and methods choices leads directly to consider-ation of the relative strengths and weak-nesses of qualitative and quantitative data. The approach here is pragmatic. Some ques-tions lend themselves to numerical answers; some don't. If you want to know how much people weigh, use a scale. If you want to
know if they're obese, measure body fat in relation to height and weight and compare the results to population norms. If you want to know what their weight means to them, how it affects them, how they think about it, and what they do about it, you need to ask them questions, find out about their experi-ences, and hear their stories. A
comprehen-sive and multifaceted understanding of weight in people's lives requires both their numbers and their stories. Doctors who look only at test results and don't also listen to their patients are making judgments with inadequate knowledge, and vice versa.
Qualitative methods facilitate study of is-sues in depth and detail. Approaching field-work without being constrained by prede-termined categories of analysis contributes to the depth, openness, and detail of qualita-tive inquiry. Quantitaqualita-tive methods, on the other hand, require the use of standardized measures so that the varying perspectives and experiences of people can be fit into a limited number of predetermined response categories to which numbers are assigned.
The advantage of a quantitative approach is that it's possible to measure the reactions of a great many people to a limited set of questions, thus facilitating comparison and statistical aggregation of the data. This gives a broad, generalizable set of findings pre-sented succinctly and parsimoniously. By contrast, qualitative methods typically pro-duce a wealth of detailed information about a much smaller number of people and cases. This increases the depth of understanding of the cases and situations studied but reduces generalizability.
Validity in quantitative research depends on careful instrument construction to ensure that the instrument measures what it is supposed to measure. The instrument must then be administered in an appropriate, standardized manner according to pre-scribed procedures. The focus is on the mea-suring instrument—the test items, survey questions, or other measurement tools. In qualitative inquiry, the researcher is the in-strument. The credibility of qualitative methods, therefore, hinges to a great extent on the skill, competence, and rigor of the person doing fieldwork—as well as things going on in a person's life that might prove
a distraction. Guba and Lincoln (1981) have commented on this aspect of qualitative research:
Fatigue, shifts in knowledge, and cooptation, as well as variations resulting from differences in training, skill, and experience among differ-ent "instrumdiffer-ents," easily occur. But this loss in rigor is more than offset by the flexibility, in-sight, and ability to build on tacit knowledge that is the peculiar province of the human in-strument. (p. 113)
Because qualitative and quantitative methods involve differing strengths and weaknesses, they constitute alternative, but not mutually exclusive, strategies for re-search. Both qualitative and quantitative data can be collected in the same study. To further illustrate these contrasting ap-proaches and provide concrete examples of the fruit of qualitative inquiry, the rest of this chapter presents select excerpts from actual studies.
Comparing Two Kinds of Data: An Example
The Technology for Literacy Center was a computer-based adult literacy program in Saint Paul, Minnesota. It operated out of a storefront facility in a lower-socioeconomic area of the city. In 1988, after three years of pilot operation, a major funding decision had to be made about whether to continue the program. Anticipating the funding deci-sion, a year earlier local foundations and the public schools had supported a summative
evaluation to determine the overall outcomes and cost-effectiveness of the center. The evaluation design included both quantita-tive and qualitaquantita-tive data.
The quantitative testing data showed great variation. The statistics on average