Updated Requirements for Ordinary Users

In this section the mapping requirements needed for ordinary users, detailed in section 2.8, are updated based on the results of the evaluation. Figure 3-9 displays the updated list of requirements with any modifications made highlighted in italics.

77

Eliminate Reduce

Any Matching API Configuration by User Visual Emphasis on Global View

Complexity of Mapping Process

Raise Create

Visual Emphasis on Local Neighbourhood Iterative Mapping Process

Familiar & Rich Mapping Interaction Easy to Understand Interface

Evaluation Methods

Figure 3-9: Updated Mapping Requirements for Ordinary Users

When ordinary users were using COMA++ some of the users were revealed to be just label matching, see section 3.5.4. These users found it difficult to disambiguate the information for the candidate correspondences from the global graph. Furthermore ordinary users found it tough to find any missing correspondences with COMA++, see section 3.5.3. In the author’s opinion, this indicates that finding missing correspondences is too tough a task for ordinary users to achieve. In fact, as ordinary users were using the NL prototype mapping tool they were fully concentrated on the task of validating candidate correspondences and were unaware of any of the added support of the tool, see section 3.7.5. This leads to the requirement that this task be removed from the process to reduce its complexity. Ordinary users will then just be needed to validate candidate correspondences suggested by the mapping system. Furthermore the mapping system itself can guide the user through the process of mapping candidate correspondences. The Question & Answer approach of NL prototype mapping tool was shown to be beneficial in navigating ordinary users through candidate correspondences, see section 3.7.2. This removes the need for the global structure of the ontologies to be shown to ordinary users. Instead only the local neighbourhood of each candidate correspondence needs to be shown.

Adding weight to this assertion is the issue of short-term management with users. It is a well- known fact of cognitive science that human short-term memory (SM), when compared to other attributes of our memory systems, is exceedingly limited [Miller 1956]. As we go about our daily lives, short-term memory makes it possible for you to engage with all manner of technology and the environment in general. SM is a temporary memory that allows us to remember a very limited number of discrete items, behaviours, or patterns for a short period of

78

time. SM makes it possible for you to operate without constant referral to long-term memory, a much more complex and time-consuming process. This is critical because SM is fast and easily configured, which allows one to adapt instantly to situations. Human short-term memory is also highly volatile [Scheidermann 1984]. This means it can be erased instantly, or more importantly, it can be overwritten by other information coming into the human perceptual system. For example in the authors’ opinion, with the COMA++ approach, the user is required to iterate through the candidate correspondences or locate any missing correspondences to construct all the possible correspondences. As the user iterates through the candidate correspondences they store a global view of the ontologies in their working and short-term memory. The approach will then require the user to make a judgement on the correspondence they select based on the local neighbourhood which will then enter their working memory and erase the short-term memory of the global views. After mapping the correspondences the user will be required to continue the mapping process by searching for more correspondences. However with the short-term memory of the global views being erased the user will feel disoriented and will need time to build up the global views again in working memory before continuing the process. This approach makes the task of mapping much tougher for ordinary user.

The overall process of mapping the ontologies was revealed to be quite time-consuming with on average it taking ordinary users 10 minutes to finish a mapping with COMA++ and 13 minutes with the NL prototype mapping tool, see section 3.5.1. This type of a ‘one-shot’ process would be too time-consuming for ordinary users. However, the time taken to map a correspondence was not as time-consuming with on average it taking ordinary users 24 seconds with the NL prototype mapping tool and 40 seconds with COMA++. In the author’s opinion, the mapping process needs to be segmented into smaller sessions so ordinary users can interact in the process iteratively over time. However an iterative mapping process creates the problem that only a partial portion of a mapping can be deployed. This is discussed further in section 4.4.

There were issues with ordinary user constructing correspondences with both mapping tools. They found the interaction approach of COMA++ unfamiliar and hard, see section 3.5.3, while the button press validation interaction of the NL prototype mapping tool was too restricting in its answers, see section 3.7.4. The different formatting of question with the NL prototype mapping tool also lead to different answers, see section 3.7.4, with no consensuses from the users on the correct format to use. The mapping interaction needs to be both familiar and rich. The requirement for a rich interaction is due to the inability of ordinary users to append missing correspondences based on the requirement stated above. Using a ‘tagging’ paradigm will allow

79

ordinary users to enter a rich relation for the correspondence, see section 2.6.6. By allow ordinary users to enter rich relationships for correspondences, these ‘rich’ relationships may be used to impact the generation/validation of new candidate correspondences. This is not investigated in this thesis but is discussed further in the conclusion section, see section 10.3.2. Finally a previous requirement stated there was a need for an easy to understand interface for ordinary users. The natural language interface of the NL prototype mapping tool was shown to be generally understandable to ordinary users, see section 3.7.1. However several modifications will need to be made to the representation of the natural language, see section 3.7.3.