Ethnographic Observations - Data Collection Methods

3. Chapter Three: Methodology

3.5 Data Collection Methods

3.5.1 Ethnographic Observations

Observations are integral to any interprevist’s approach that are, searching for and, exploring social situations to gain an understanding of meaning. Typically, data is gathered through observations and the phenomena studied are placed in a social and cultural context (Kjeldskov & Paay, 2012). The social and cultural contexts to this research study are important providing the researcher with data exploring how students have applied theories informed by the systematic review to support their own test practice. Maggs-Rapport (2000) applied observational techniques concentrating on the descriptions people give to their routine, daily lives, enabling the ethnographer to explore a number of views at the same time. To achieve the aim of searching for a mobile test framework the researcher requires a clear and grounded understanding of their social meanings behind tests planning and execution. Mcneil & Chapman (2005) reviewed qualitative methods supporting social science research, observational methods proved to be the most effective (2005, p.92). Collecting data using observations requires systematic note taking and the recording of events, behaviours, and artefacts (objects) in the social setting (Marshall and Rossman, 2011 p139). Within social science research there are three broad ways in which observations have been applied, direct or non-participant observation, participant observation and complete or covert observation (Mcneil & Chapman, 2005; Robson, 2002).

This research blends observational techniques and uses them at two points within the module; in-class observations and field observations. In-class, taking a participative approach aiming to gain an initial insight into the students interpretations that emerged for the researcher systematic review. The field observations are more non-participative and will evaluate how the students apply their mobile tests within the field context.

3.5.1.1 IN-CLASS OBSERVATIONS

Set within a teaching schedule (a formative setting) the researcher introduces theories formed from the systematic review. Students are set a range of practical exercises aimed at exploring test theories and concepts in practice. Test frameworks and models used and interpreted to test design and field preparation. The exercises match learning outcomes and students’ apply theories and data is captured via participative observation help to gather feedback and feeding forward to the experimental field tests. The research set in-class/lab environment as Sun and May (2013) point out begins to test and simulate context richness within the lab through scenarios contributing to the realism this maintaining the benefits of the controlled setting before the field tests commence (Sun and May 2013; Kjeldskov, et al. 2004). Table 3.4 presents the session structure and splits into two distinct parts and the researcher’s observational role can be seen at participative or non-participative.

Table 3.4: Formative Session Structure (three formative

sessions follow the same a practical and reflective)

Breakdow

n Time Mode of inquiry Activity

Part One 09:00- 10:45

Participative Topic theme introduced and workshop activities set around theme (context of information need, user behaviour, mobility and spatial environments). The researcher (as a lecturer) guides and supports students fielding questions about the model and how this could be applied to the activities and assignment Students work through the activities and the researcher notes in a diary points based upon the categories and themes from the model.

Break: 10:45- 11:00

N/A At the end of Part One there is a break and the

researcher has the opportunity to group the points based upon the number of occurrences.

Part Two 11:00- 13:00 Mixed between non and participative mode of inquiry

The students and researcher return and a focus group commences (in the guise of a seminar) and a discussion takes place where the points are member checked to help highlight the importance

A final list is built up to see the importance, which will feed into the discussion of findings.

Sample Size: session one n=27, session two n=18 and session three n=18

Part One: The in-class observation is interactive and participative exploring students’ approaches to test planning and development. The researcher and students’ get involved with the activities, in this context the researcher is seen as a `loiterer' (Spradley, 1979) watching the interactions build-up individually as they communicate with one-another. The researcher wants to understand and quiz them about their interpretations as test scenarios

emerge extracting new codes that emerge which will be analysed to see some kind of vocabulary would emerge (Paay, J. & Kjeldskov, J., 2005).

Applying participative observation requires confidence and skill, Mcneil and Chapman (2005) talk about a “trusting rapport” with the participants (2005, p.96) being a challenge. The theories and data collection methods piloted in 2012 built around participative research. The aim of creating a trusted rapport with the students was important and students commented that this complemented the workshops design. This rapport grew throughout the module aiding confirming the approach worked. It was also anticipated that with this sample

participative research helped planning as they received timely feedback feeding-forward into the assignment. The module outline also stated, “each week some class time is dedicated to development and planning of this project”. In essence students have plenty of time to pilot tests with their peers building up trust and confidence to guide their learning but not influencing decision making.

Part Two: Is less interactive in a practical sense but more of a seminar/group discussion. This type of approach is aligned with methods used within focus group design in that it’s a “form of group interview that capitalises on communication between research participants in order to generate data” (Kitzinger, 1995). The themes and new codes extracted in Part One are presented to the class and as a group a discussion takes place to rationalise and group creating a vocabulary to inform the research. The codes and themes are member checked which “is primarily used in qualitative inquiry methodology and is defined as a quality control process by which a researcher seeks to improve the accuracy, credibility and validity of what has been recorded during a research interview (Harper & Cole, 2012; Lincoln & Guba, 1985). Member checking reduces, validates and refines the themes, which help to clarify the findings from Part One. This helps to focus the research on the aim of study and as

Fetterman (2010) discusses the success or failure of […] ethnography depends on the degree to which it rings true to the native and colleagues in the field. From the professional stance it is anticipated this approach will inform practice which will feed into analysis helping to cross checking the lab work against the actual field tests.

Table 3.5 displays the sessions, when they will take place and how they meet the modules learning outcomes. There are three planned sessions each following a consistent path i.e. an initial practical part and seminar which reflects on their practice.

Week Date Session Title Learning Outcome:

Week Four Date: 16/10/2013 Time: 9am- 1pm

Developing test plans (Introducing the Model)

Understand the context of information needs within a mobile context. Week Five Date: 23/10/2013 Time: 9am- 1pm Cognitive Modelling (Supporting HIB)

Model the context of needs using the theories to model and map out user pathways.

Week Six Date: 30/10/2013 Time: 9am- 1pm

Implementation (Piloting prior to field tests)

Pilot and record scenarios in lab conditions. Expert review and modelling user pathways. Sample Size: session one n=27, session two n=18 and session three n=18

3.5.1.2 FIELD STUDY OBSERVATIONS

The field from the researcher’s perspective is a non-participative mode of inquiry suiting this kind of ethnographic research. The field study aims to “explore the relationship between developing a descriptive understanding of human behaviour and designing artefacts which ostensibly support the activities described” (Blomberg et al., 1993). This field study explored how the students’ behaviour has been influenced and supported by the test model developed as part of the synthesis.

This field observation was applied as a method to capture data in a non-obtrusive way. Cooper et al. (2004) explained that it offered a more objective and direct view of behaviour (p1, 2004). The mobile test framework is based around a number of theoretical concepts and observing these in practice will provide a more direct view than say interviews which may employ indirect mode of assessing information and information use (Cooper et al. 2004). The field studies are unobtrusive in many ways so the data collection does not impact on their experience as a learner. Planning observations around the module timetable makes sure observations are consistent, fair and timely with each student.

Table 6. Summative Observation

Schedule (Based upon the module workshops)

Week Session Title Mode of inquiry Data Capture

Week Seven Running Field tests – Collecting Qualitative Data

Non-Participant observation