While early implementations encapsulated the framework requirements directly in data structures and protocols, experience from CoAKTinG has shown that the requirements can be met with more generic node implementations, using
shared ontology (section 5.5.5). Indeed the Semantic Web model of adding, removing, and merging sections of RDF graphs in a piecemeal way fits very well to continuous metadata, where discrete sets of assertions are forever flowing in and out of nodes. It also maps well into the generalisation for implementation introduced in section 6.1: the collections of metadata are the RDF graphs held in each node, and the nodes internal data structure is simply RDF validated by any ontologies used.
This approach of using RDF to encode metadata was therefore adopted for the proof-of-concept evaluation of multicasting continuous metadata. JGroups [14] was used to provide a reliable multicast transport, and the Jena Semantic Web Framework for Java [97] to hold an in-memory graph storing the RDF triples:
1. RDF/XML encoded assertions are sent, using a JGroups channel, to the test node using multicast. The assertions declare that a Compendium node has been created or modified - this simulates the data Compendium would send.
2. The test node receives the RDF and adds it to the graph it holds (called a model in Jena).
3. When the model changes Jena raises an event; on this event the model is queried for all temporal assertions which are then ordered into a timeline, and the subject description of the ordered time periods is rendered on a visualisation of the timeline; this represents Compendium activity.
4. The process repeats: as more assertions reach the node they are added to the graph, which is then used to re-render the timeline.
Multicast functionality is seen by instantiating several copies of the test node - all receive copies of the assertions, and all their timelines update accordingly. While this proof-of-concept lacks the sophistication of the Meeting Replay tool and the functionality of Compendium, it verifies that the full potential of the framework could be realised if multicast networking were available to the RST,
and Compendium were enhanced to output a flow of continuous metadata directly.
Further development of the proof-of-concept would likely raise additional issues: • assertion of time in the AKT Reference ontology (and by extension the
CoAKTinG Meeting ontology) is unwieldy and verbose: each time point is separated out into seven temporal components each of which is a
DatatypeProperty (year, month, day, hour, minute etc.). This is especially true when mapping events onto a temporal axis (e.g. creating the timeline) as the software must repeatedly walk through the graph. Since nodes will store large numbers of triples from multiple metadata sources, it could be expected that the graphs will become large and this inefficiency could have a serious performance implications.
• The timeline in the CoAKTinG Meeting Replay tool assumes the
accompanying media is of fixed length, and thus the timeline can be too. This assumption has been also been adopted by the proof-of-concept, despite the supply of a potentially never-ending flow of continuous
metadata; realistically the user interface would have to be adapted to allow a flexible amount of “scrolling zoom” showing a limited length before and ahead of the current time. Without this elements on the timeline might be shrunk to insignificant proportions.
• As detailed in section 4.3.3, without buffering or media delays inserted at presentation, metadata might not be presented until after the associated section of media.
Conclusions and Future
Directions
7.1
Summary of Work
The aim of this thesis has been to explore the use of metadata to enhance multimedia applications – not just at the point of authoring and storage, nor merely at presentation, but also at all points in between. This has been motivated by the need to support distributed collaboration activities, and in particular add novel real-time interactive structure to aid the user alongside streaming audio and video.
Chapter 2 surveyed the body of research surrounding the thesis, documenting how hypermedia systems have developed to incorporate temporal media, and the requirements at the network and application layers of multimedia systems. Metadata was also studied in its details and application, and from this a theme emerged: one of a generalisation of structure, represented by and in metadata, from the links of hypertext through to the ontologies of the Semantic Web. In chapter 3 the theme of metadata as structure was examined in the context of distributed multimedia. A lifecycle perspective on multimedia distribution was introduced and used to analyse and contrast the levels of support for multimedia
and the metadata we wish to augment it with. Deficiencies in the provision of metadata through the transmission stage were identified, and continuous metadata was proposed as a means to overcome them (section 3.3); several motivational scenarios were given as illustration of how continuous metadata might be implemented and utilised.
After further analysis and justification, chapter 4 introduced a conceptual framework to define continuous metadata, and with it a set of requirements through which continuous metadata can be provided.
Continuous metadata, and the framework, were put to the test in chapter 5. The CoAKTinG project had, amongst its aims, the use of continuous metadata to enhance distributed virtual meetings, and provided an opportunity to trial the principles of this thesis in a simulation of remote group collaboration though the NASA Mars Desert Research Station.
A novel approach was taken in implementing continuous metadata by way of an OWL ontology, which was used to mediate structured temporal metadata from several different sources, serialised in RDF. This approach was validated, with continuous metadata and the framework, when a hypertext navigation was constructed from the RDF and used to enhance multimedia materials during the distributed collaboration of NASA team members.
Full multicasting of continuous metadata was not trialled during the NASA simulation, when the users did not have access to a multicast enabled network. Despite this, it was proposed that multicast would have improved the provision of metadata had it been available; this was validated by a proof-of-concept tool in chapter 6.