General Ontology Repository Characteristics

It is not surprising that there is a lack of standards specifically relating to the role of Feature Catalogues as ontology repositories, because the general processes for identifying and accessing

55

ontological resources are themselves poorly standardised (Hartmann et al., 2009). Hartmann et al. (2009) have suggested the following definition for an ontology repository and its associated management system and have proposed a Generic Ontology Repository Framework (GORF) as a starting point to address current short-comings:

“An Ontology Repository (OR) is a structured collection of ontologies (schema and instances), modules and additional meta knowledge by using an Ontology Metadata Vocabulary. References and relations between ontologies and their modules build the semantic model of an ontology repository. Access to resources is realized through semantically-enabled interfaces applicable for humans and machines. Therefore, a repository provides a formal query language. Software to manage an ontology

repository is known as Ontology Repository Management System (ORMS). An ORMS is a system to store, organize, modify and extract knowledge from an Ontology Repository.” Hartmann et al. (2009) argue that the main driving motivation for creating ontology repositories is to support knowledge access and reuse for humans and machines. Hence ontology repositories on the one hand act as a storage facility and on the other provide access to knowledge through defined interfaces and policies. To achieve these goals, comprehensive facets must be considered by an ontology repository when handling ontologies. The GORF extends conceptually the SEAL (SEmantic portAL) framework (Hartmann and Sure, 2004) and preliminary work on ontology repositories, as described in Hartmann et al. (2005a). The idealised OR Framework comprises of five conceptual knowledge layers (as distinct from an ORMS):

 Access: the repository must provide adaptable views on the stored knowledge, as shown in Hartmann and Sure (2004), involving different ways to view and query the knowledge.  Processes and services: the repository should include facilities for ontology: evaluation,

rating, mapping, security and engender trust.

 Organisation: the repository needs to cover factors such as ontology metadata/annotation, lifecycle management, modularisation, validation, registries and indexes.

 Storage: the repository needs to be scalable in terms of its: query processing ability, consistency of response and propensity for replication.

 Sources: the repository should include the ability to harvest from heterogeneous sources. Nyulas et al. (2009) and d’Aquin and Lewin (2009) are amongst the few groups who have exercised many of the conceptual Framework facets described in GORF, the former through ‘BioPortal’ (a collection of biomedical ontologies) and the latter, through ‘Cupboard’ (a system to upload, expose

56

and explore ontologies, as well as to find, select, reuse and assess other users’ ontologies). ‘Oyster’ (Palma and Haase, 2005) and ‘Onthology’ (Hartmann and Sure, 2004) are current examples of ORMS. Both Baclawski and Schneider (2009) and Hartmann et al. (2009) point out that most existing

ontology repositories tend to have many of the same features, such as registration, submission and upload; browsing and search; description and documentation; metrics and statistics, but that there are many missing and desirable features. The most noticeable is the lack of structure among the hosted ontologies (Allocca et al., 2009). The ontologies in a repository are treated as independent entities. Another missing feature is the lack of sufficient metadata annotating the hosted ontologies. While ontologies are claimed to be a mechanism for interoperability and communication between data sources, ontologies themselves are nearly always built in isolation (Baclawski and Schneider, 2009). There is no common representation of metadata annotations of ontologies and no common ways to identify versions of ontologies. The various extant ontology repositories use a variety of techniques and do not enforce any standard conventions for content description or for

communication interfaces. As such discovering and querying repository content is problematic. A strong candidate ontology metadata description, however, is the Ontology Metadata Vocabulary (OMV: Hartmann et al., 2005a).

To address the problem of common browsing across repositories, Viljanen et al., (2010) have proposed creating a network of Linked Open Ontology Services (LOOS) consisting of ontology repositories that publish their content using a shared API. This LOOS API is implemented in a demonstrator as a lightweight, stateless, and cacheable HTTP GET based API that returns data using the JSON format. In many respects this API is similar to the REST API used in BioPortal (Noy et al., 2009). However, the goal of LOOS was to build a network of ontology services whereas BioPortal's focus is to publish an API for accessing its own repository's full functionality.

In terms of lessons that could be learned from the stand-point of policies and best practise, Kendall (2009) has shown the following factors to be well correlated with ontology repository reuse:

 small development teams with larger user communities;  commitment to users and to continuous improvement;

 publication of maintenance policies; URI naming conventions and protocols; and useful documentation.

Baclawski and Schneider (2009) claim it is therefore important to have well specified policies for vocabulary management, metadata, and provenance specification, particularly to enable trust. It is also critical to have a commitment to forming, accommodating, serving, and working with a

57

community of users. This emphasizes the importance of outreach and education, including the identification and promotion of best practices.

All of the previously mentioned open issues and activities in general ontology repository research are of relevance and interest in helping to characterise a Feature Catalogue, which is essentially a repository to manage domain concepts in scientific OGC-standards based infrastructure.

Of particular significance in the work that has just been described is the assertion by Hartmann et al., (2009) that ontology ‘rating’ and ‘evaluation’ are important facets of an idealised GORF. In a

somewhat poorly justified argument they separate ‘rating’ from ‘evaluation’ on the basis that ‘rating’ is a subjective assessment of an ontology, whilst ontology ‘evaluation’ can be seen as an assessment of the quality and the adequacy of an ontology, or parts of it regarding a specific aim, goal or context. In the author’s opinion ‘rating’ is actually a type of ‘evaluation’ and doesn’t necessarily deserve to be singled out as a separate element in GORF. Also, as will be shown in the next section, ontology evaluation, as an activity and depending on what is being assessed, can be just as subjective as ontology rating.

Hartmann et al. (2009) envisaged that selected evaluation strategies could readily be implemented in “an evaluation component and applied in a large repository”. Whilst the author agrees that ontology evaluation is an important part of ontology management and re-use, the inference in Hartmann et al. (2009) is that this is a relatively straightforward exercise of picking strategies from an available list of techniques and applying them. As will be shown next, the field of ontology selection and evaluation comprises many diverse evaluation strategies, many of which when put to the test do not have necessarily measurable criteria, or where there are measures these are not experimentally verified, or worse, what is being measured is not an indicator of the criteria it purports to measure.