Since the importance of external data quality is widely recognised in the GIS community (see Section 2.4.3), it is surprising that there has been very little empirical research into user perspectives on geospatial data quality and fitness for use evaluation. User needs have somewhat been neglected when developing new solutions for improved quality evaluation and dataset selection. To address this issue, the main focus of this research was to understand how geospatial data users select the datasets to use, what the reasons for their decisions are, and what mechanisms could improve their experience. Consequently, this research adopted an iterative UCD approach in order to provide solutions that are tailored to geospatial data users’ needs and that are likely to garner user acceptance once deployed.
UCD is “a design philosophy and approach that places users at the centre of the design process from the stages of planning and designing the system requirements to implementing and testing the product” (Baek et al., 2008, p. 660). The main characteristic of UCD is that it attempts to optimise a product around user needs, abilities and desires, rather than forcing the users to change their behaviour to suit the product. UCD recognises that end user involvement in the design of tools is critical to the success of the developed solutions. In UCD, it is important to involve the potential users in the design process at an early stage and then continuously review and refine user requirements of the technology being developed (Baek et al., 2008).
PREPARATORY PHASE Method: Semi-structured interviews Purpose: Initial investigation
into how users evaluate quality of geospatial datasets.
PHASE I
Method: Questionnaire-based study Purpose: Elicitation of initial GEO
label requirements.
PHASE II
Method: Questionnaire-based study Purpose: A feedback study
presenting GEO label examples.
Method: GIS community polling Purpose: Community voting for the
final GEO label visualisation.
WEB SERVICE DEVELOPMENT Method: Practical implementation Purpose: Development of the
GEO label Web service to support GEO label generation.
DSS DEVELOPMENT Method: Prototype implementation Purpose: Development of the GEO
label-based prototype DSS tool.
PHASE III
Method: Controlled lab-based study Purpose: Evaluation of the GEO label-
based DSS tool.
1
2
3
4
5
Figure 3.1: Phases of GEO label research.
Utilising various tried-and-tested UCD methods of collecting and analysing research data, this research comprised six main phases of exploration, development, evaluation and validation, with each phase building upon the knowledge gathered in the previous phases (see Figure 3.1). It should be noted that, while there are guidelines and techniques available for conducting UCD, each method will vary depending on the problem at hand.
The preparatory phase (see Chapter 4) was conducted using a series of semi-structured face-to-face and telephone interviews with geospatial data expert users and producers. The intention was to uncover initial information about dataset selection, including their use and production within representative application areas, in order to inform further research phases. The semi-structured interview technique is widely used in UCD at the beginning of the design process to gain an initial understanding of user needs and perspectives on the technology being developed (Abras et al., 2004).
Phase I (see Chapter 5) was conducted via a comprehensive online questionnaire-based survey to solicit initial geospatial data producers’ and users’ views on the concept of a GEO label and the role it should serve. Questionnaire-based surveys can be successfully applied at an early stage of research to collect data related to the needs and expectations of stakeholders (Bevan, 2003; Abras et al., 2004). This method was most appropriate to facilitate the collection of data from a large number of geospatial data users and producers from diverse GIS communities across the world.
Phase II (see Chapter 6) focused on the iterative design of the GEO label graphical representation. A comprehensive questionnaire-based study was conducted to solicit geospatial data producers’ and users’ views on the proposed GEO label visualisations. Questionnaire-based studies can be effectively applied in the evaluation of design
alternatives in the early stages of the design cycle (Abras et al., 2004). Similar to Phase I, this technique was used to collect a large sample of data. Following the questionnaire-based study, semi-formal feedback and recommendations from GIS peers and GeoViQua project partners were used to inform some of the GEO label design revisions (e.g., selections of colours, new icons design, etc.). This ensured geospatial professionals’ involvement in all the stages of GEO label design. Due to the fact that two diverse designs were potential candidates for the GEO label visualisation, GIS community polling was conducted to identify the final GEO label representation. The intention was to obtain views from a large number of GIS professionals and identify community-dictated graphical representation.
The Web service development stage (see Chapter 7) focused on the implementation of a GEO label service to support the generation of dynamic GEO label representations. This stage was conducted via rapid prototyping and development, and resulted in a stable and fully-functional RESTful Web service. As part of the interoperability testing and validation, the service was integrated into a number of real-world GIS applications (see Section 10.2). The DSS development phase (see Chapter 8) focused on the design and implementation of a GEO label-based dataset discovery and intercomparison tool (GEO LINC). This development phase also followed a rapid prototyping approach. While the GEO LINC prototype represented a proof of concept, it was important to provide a functional system to allow for the evaluation of system interactivity and simulation of a real dataset selection experience.
Finally, Phase III (see Chapter 9) was conducted via a controlled lab-based human subject study to evaluate the usability and effectiveness of the prototype GEO label-based tool. A lab-based study approach was selected because it can be successfully applied to elicit rich qualitative data and offer a high level of experimental control and repeatability.