3.2. Data Collection and Methods
3.2.2. Data Analysis
I analyzed each of the three cases on its own, and then considered the three cases alongside one another. Stake (2006) proposes this kind of approach to multicase analysis: it begins with a within-case analysis where research questions are explored at the case level and is followed by a cross-case analysis where research questions are explored at the phenomenon of study level.
However, although there were many small (and significant) distinctions between the buildings at the case-level, these were not of such a scale that writing three separate case reports seemed effective. I found it most efficient (and effective) to write two separate findings chapters: one that explores the case-level questions (Chapter 4, which is quite long nonetheless), and one that explores the phenomenon of study level question (Chapter 5). This alternate approach allowed me to contrast the different libraries by discussing the most relevant case-specific findings but, at the same time, also make comparisons on a cross-case level. Had the phenomenon of study (which, in the present study, is the public library as organization space) been more general, for instance public cultural institutions as organization space with, say, a library, a city hall, and a museum as cases, then perhaps a more rigorous application of Stake’s (2006) method would have been appropriate.
Once the interviews (including the cognitive mapping discussions) were transcribed and checked for errors, they were entered in the project database created using N’Vivo 8. Each case library had its own file within the database; staff interviews and user interviews were stored separately within the file. I did not begin analyzing data until I completed all three site visits. I began analysis by reviewing interview transcripts and developing a reliable coding scheme (using a thematic or axial scheme, something which Miles & Huberman [1994],
83
Strauss & Corbin [1998], and Bazeley [2007] advise; see “Interview Analysis”, below). Once the coding scheme was finalized and transcripts were satisfactorily coded, I reviewed the observations, cognitive maps, and photos. I applied the coding scheme to the observations and photos but not the cognitive maps.
Assumptions
My chosen methodology necessitated the following assumptions:
that the data I collected over the course of five full days at each case library is representative, notwithstanding seasonal changes in library use, of how these libraries operate over a substantially longer period of time—i.e., that the same general and specific activities occur, and that people’s perceptions and experiences at these libraries would not have been different at another time of the year;
that interviewees were being honest and truthful when responding to my questions, and that anonymity would provide them with the comfort to speak without fear of judgement or reproach from others in higher positions of power26; and
that each case library as an organization has full or nearly full control over the planning and design of its library building; that the intentions and objectives of architects, designers, and planners when designing libraries are, for the most part, to serve the needs and requests of library board members, administration, and staff and public.Interview Analysis and Coding and Inter-coder Reliability Testing
Miles & Huberman (1994) define “codes” as “tags or labels for assigning units of meaning to the descriptive or inferential information compiled during a study” which are used “to retrieve and organize chunks mentioned earlier” (p. 56-7). Although some advocate an open coding approach when analyzing qualitative data (see Berg, 2001, p. 251-55), others do not. Miles & Huberman’s (1994) recommend applying an a priori framework or “start list” of codes that “comes from the conceptual framework, list of research questions, hypothesis, problem areas, and/or key variables that the researcher brings to the study” (p. 58). The start list can have up to 50 or 60 codes.
84
My final approach was a combination of the two: I began with the three, theoretically informed concepts of enchantment, emplacement, and enactment as broad categories, and then I coded openly from that point onward. Subsequent passes resulted in revision and merging until I arrived at my final scheme. I also coded biographical and statistical information for each interviewee and separated them into their own category. These descriptive codes included (for staff members) such information as job title and duties, years employed at the library, and for users their frequency and history of library use. The process of coding took nearly three months (not including transcription time).
I recruited a fellow PhD Candidate to test my final coding scheme on a sample of unmarked transcripts from the project file. After comparing her coded transcripts to mine, and using a formula given by Miles & Huberman (1994), I added all agreeing codes together and divided by the total number of all agreements and disagreements and determined a ratio of 0.82 or 82% reliability. Miles & Huberman (1994) claim that most intercoder tests will score 70% or lower on the first test but that eventually “intercoder agreement should be up in the 90% range” (p. 64). I did not test the scheme a second time because I deemed 82% reliability strong enough and because most intercoder disagreements pertained to things that would be clarified in the written findings.
Photography and Observations
After transferring photographs to the project computer (and blurring the faces of users and staff), I sorted Set 1 and Set 2 photographs from each other and then printed them for coding. Prints were contained in binders by case library, and within each binder Set 1 and Set 2 photos were kept separate. I reorganized and printed Set 1 photos by library level; I printed Set 2 photos in sequential order (using date/time stamp information).
For analysis of the photographs and observations I used codes from the N’Vivo project file. This linked data from these two data sets to the interview data set and allowed for comparison of the different data sets (and, in many cases, corroboration between them).
85
Cognitive Maps
Since landmarks and evidence of territoriality were the two kinds of information I hoped the maps would provide me, I sorted the maps into separate files (by case library) and then sorted them by user versus staff member. I conducted a first examination of the maps by making notes about the kinds of objects and general patterns they contained. I then created a matrix in which, while conducting subsequent examinations of the maps, I recorded the frequency of these objects and patterns. Each case library received its own matrix. This approach allowed me to see the data in aggregated form and observe patterns and frequencies within and between interviewee groups (staff, users) and case libraries.
Validity and Reliability
As Yin warns in his text on case study methods: “Do case studies, but do them with the understanding that your methods will be challenged from rational (and irrational) perspectives and that the insights resulting from your case study may be underappreciated” (p. xiii). However, when done using a careful, systematic approach (i.e., the use of structured, pre-approved schedules and guides as a way of ensuring a systematic approach to data gathering) as well as a uniform approach to the collection of data at multiple sites, case studies can yield findings just as (or possibly more) powerful than those of other qualitative approaches.27
Many have criticized the validity of case study findings based on the idea that findings from only one or a small handful of cases cannot be generalized to a larger population. This belief, according to Yin (2003), confuses theoretical generalizability with statistical generalizability. Case studies offer no statistical validity or generalizability whatsoever. Findings pertaining to the research question (RQ) can be generalized at the level of theory only. Put another way, while we cannot make assertions that all public libraries in the “real world” possess certain attributes or power relationships in their space, we can assert or explain, based on those attributes and relationships we have detected in our three cases, what we have learned about
27 In her short essay in s upport of the case study Gladstone-Millar (1998) makes the observation: “One point forgotten by the critics of
case study research is that this type of research is not easy; statistical research is easier, quicker and most of the groundwork could be done by an assistant.”
86
the public library as an organization space. Future studies of other individual case libraries may add to, extend, or fully contradict this study’s findings and conclusions.
Trustworthiness and Transferability
Though a qualitative case study’s findings may not be statistically generalizable, if the study’s research design and methods are approached systematically and rigorously enough they can achieve a certain “truth” or degree of generalizability on an analytical basis (Erlandson et al, 1993).
The research design used here fulfills several criteria for achieving “trustworthiness” and “transferability” (Lincoln & Guba, 1985; Erlandson et al, 1993). The site visit schedule achieved a degree of prolonged engagement, placing the researcher in the field (i.e., at each library) long enough time for him to understand each case library in a way relatively comparable to those who experience it on a daily basis. The use of multiple methods also achieves a strong degree of triangulation: findings culled from multiple data sources corroborated each other and strengthened the overall validity of those findings. This was particularly important when considering the subjective nature of staff and user interviewee testimony against observational and photographic data. Enough “referential adequacy materials” were collected to provide as holistic a view as possible of each case library’s context. Overall, though each case library and its contexts are unique, specific findings are potentially applicable (transferable) to the same aspects of other cases where contexts are similar.28