Future Work

Whilst the research has covered much ground, there is still a great deal that could be improved and enhanced further with the following suggestions for future work.

9.5.1 3D Spatial Relationships Algorithms

The computations of the spatial algorithms in the Semantic Spatial Image System are mostly concerned with spatial terms that are limited to the two dimensions in the image plane with some extension to three dimensions for the relative distance position in the more advance spatial relationships algorithms. The system could be enhanced to incorporate more expressivity of spatial information in the 3D environment. For example Lee et al. (2004) had presented the use of a spatial location algebra for 3-D image scenes limited to a number of spatial terms. More 3D spatial terms could be implemented including spatial terms found in the preliminary survey. Some examples of the spatial terms involved are: on, behind, within and around.

To implement this enhancement, the 3D spatial algorithms should take into consideration the criteria of transition (scaling, moving and rotating) of an object in the image, the degree of the perspective view of the object, the environment or scene involved etc. Work on 3-D scene analysis in computer vision will contribute here in the future. By doing this, the system could offer more options and facilities in a more specific way for the user during annotation and retrieval.

At the same time, the spatial relationships algorithms developed with a capability of identifying the relation between objects and their position in an image can be further explored, implemented and adapted in a different application area and more specific domains such as medical and Geographical Information Systems.

9.5.2 Enhancing the Spatial Relationships Ontology

The spatial ontology presented in Chapter 7 is a very basic ontology developed as a proof of concept in order to show how such an ontology could be used in the spatial semantic image system. Although the Spatial Relationships Ontology can capture reciprocal relations crudely, the identification of reciprocal relations and the use of

Zurina binti Muda Conclusions and Future Work

149

reasoning to infer the reciprocals would be a more intuitive approach. Another improvement would be to structure the knowledge more hierarchically where related terms could be grouped together with one spatial term as a superclass and others in the same group as its subclass. For example, the term FarLeft could be a subclass to the term Left etc. Also the 25 absolute positions could be inferred from conjunctions between any of the five row and five column positions. A similar approach could be taken to the composite concepts such as above left and above right etc.

The introduction of ontologies also offers scope for handling synonyms and for reasoning over the OWL ontologies in a variety of ways. This could be achieved using rules language such SWRL in Protégé(Horrocks et al., 2004), SPIN in TopBraid (Knublauch, 2011) or an Ontological Logic Programming by Sensoy et al. (2011) where the rules related to the spatial terms are computed and could be done in the same platform.

In order to support enhancements for 3D, the Spatial Relationships Ontology could be expanded to include more classes and properties of the new 3D spatial terminologies for spatial relationships. Added knowledge to compute the 3D spatial relationships might also be needed where this information could be retrieved from the domain ontology. Hence the Place of Interest Ontology could be enhanced by including more classes and properties of objects in broader domains. On the other hand, linking to a bigger domain of knowledgebase or ontology such as DBPedia or Geonames would provide other advantages where more objects with the order of magnitude of heights could be retrieved and thereby enhance the functionality of the Spatial Relationships Ontology.

9.5.3 Integration with Other Domain Ontology

Currently, we have integrated the Spatial Relationships Ontology (application ontology) with the Place of Interest Ontology (domain ontology) that contained limited classes and properties to demonstrate the function of the application ontology. The Spatial Relationships Ontology could be used together with other appropriate ontologies such as medical or transportation.

Zurina binti Muda Conclusions and Future Work

150 9.5.4 Improving the Retrieval System

In order to improve the retrieval performance of the system, the development of a user semantic relevance feedback might be a good way to involve users in the retrieval process. This feedback could enable the user to respond by ranking the retrieved images. The relevance feedback might also allow the user to select one of the retrieved images, which seems more appropriate to the user to conduct more searches. This will benefit the user as well as the system for future enhancement to improve the quality of retrieval.

At the same time, a visual interface with a point and click representation for the retrieval system would make the process of querying easier and more elegant to use. The interface could be a controlled retrieval interface with 2 or 3 fields specified for object/s and spatial relationships or an open query where a user may input the object/s they are searching for.

9.5.5 Enhancing LabelMe with Spatial Annotation

As we know, the Semantic Spatial Image System gathered an input from LabelMe. LabelMe is an open-source web application. Therefore, the capability and potential that the system has is compatible and could be integrated with LabelMe. By doing this the computation of spatial relationships between objects in the image could be done simultaneously when a user annotates an image in LabelMe. This could be done by adding another tab for spatial annotation that would be generated automatically when the user labelled the object. Hence, LabelMe could offer more facilities and outputs to the user instead of just object annotation, but also spatial annotations between those objects.