In this section we will look at three different visual retrieval libraries and describe their features, interaction, goals and history. Both feature various forms of video retrieval with different search features and interaction.
1.6.1 The Fischlar Digital Video Library
In 1998, the Centre for Digital Video Processing (CDVP) in Dublin City University first started work on developing the Fischlar digital video library in order to showcase and evaluate research from within the research group. The first system developed by the group was Fischlar TV which was a collaborative online digital video recorder with shot level browsing and playback [O'Connor et al, 2000]. Users could schedule television programmes to be recorded and, when available, use various browsers to search through and playback that content. The system specialised in browsing of video content and there were a number of specialised browser variations that were created and evaluated [Lee et al., 2000].
_J1v II», \Ut t u r n ■ 4?!" X C«cr*t) Htp:ll~*Mi»at4r AucMteO<VlH«a/99l T7.Bg OUr^iCOW*«t Trusted s*es wcwvatnAY > TV6CHEDUIE/ RECORD ., :
Figure 1-3: Screenshot o f Fischlar TV
The Fischlar TV system is broken down into two main sections namely Browse/Play and TV Record. The screenshot shown in Figure 1-3 shows the Browse/Play section. The scrollbar on the left shows a list o f the TV programmes recorded and available for browse and playback. Clicking on any will give the user an overview o f the programme while clicking on detail view (top right) gives the user the option to browse the programme. A number o f content browsers are available and the one shown above is known as the hierarchical browser.
Another system developed within the CD VP was the Fischlar News library, designed to showcase the group’s domain-specific research on television news content. An early version of the system recorded and made available each nightly news programme with content stretching back over many months. Users could search the closed captions extracted from the news in order to locate content o f interest. As the system evolved, fully automatic news-story detection was incorporated. Users could then browse news programmes at the story/scene level [O Hare et al, 2004].
w * htì^;//ww,fischiar.doj.la/nows|ibrows« F is c h la r-N e w s |3[iuìjìj3^^BES] 1 2 3 4 5 6 1 9910 11 12 131415m 17 19 192U March 2004 1 2 34 56 1 B 9 IB111213 14 15 1fl 17 18 1920 21 a 23 24 25 m ZL m 2£ 3Q 31 February 2004 1 2 3 4 5 a 7 a 910 u 1222 i i
We vvv S tim m a ry: 1 3 s i o r/e s
H aw s Situ iv i id ii! ni _,n Lin k r? ~ - é r "E ¿Il 0 'ri - h rii :1«j(Ij fi>J V jj; E? ,--- -- re n a n i! lo in s rsf/urt n lérrO
i $ È S ? d [ News SlOfV2 i-J’j ration UO 04 1i) .
T if’J'-eacri h.'i& inai he ?iif e |]|.;j (i»c m w , fiiicmfjo frorr- [¡re Cordi jJbW f 1 ita Jing ¿1 £ inn ¡'em are -sito tre leodir.g in em bers iPLav th is
J E ? i ! Nevis S tory 3 fduias&n CO ul *55)
Tu o'hsr r.*1 *r- i;ùiv. in* t'iàl yot u n ci-nsi/ Ic-iJay 01 ti ie judge 9 ri in Curimi v, i K ? 1 | ¡Hi P o s ; * :■ : i ng chud p oi nogf .ìphv The c a s e ól ìp lt>v t h is s t o r y -)
* -- [ f i News Story 4 (duration DO 00 22)
< i Tfìfe Brillìi Primi: Minister tì3f. ECTirirmed m at Bnlain will h e ll Uie referendu i j g f I * proposed EU consMutiori Vf Blair diant naivie a date but (p u > ./th is s t o
News Story 5 1 dui-iti on 00 01.5i )
Wasrunil-on h a ; denied Hiatus Hague 10million m l i ^ i s tonnnu a n sila^ il
Hi
O HYACCOUNT
Figure 1-4: Screenshot o f Fischlar News
We can see from the screenshot in Figure 1-4 that the available news programmes organised by calendar are visible on the left with the stories shown on the right. The user can start playback of the story or view the story’s associated shots and text. Users can search the closed-caption text by adding terms to the search box shown on the top left of the screen.
A final variation of the Fischlar system is the Fischlar Nursing library that was created to aid the university’s expanding nursing school to deliver and tailor video content for students in an interactive and visual manner. It differs not only in the video content available, which is nursing specific, but also in the approach used to index the content. Lecturers essentially created the index for the video content they wished and it was incorporated into the library. Nursing students could view the content whenever they have time available [Gurrin et al., 2004], Nursing material is a natural choice for a digital video library due to the visual nature o f the educational material.
1.6.2 The Informedia Digital Library
One o f the early examples of an online digital video library was the Informedia project at Carnegie Mellon University (CMU). The first Informedia digital video library was started in 1994 and was designed to integrate speech, image and language to facilitate the construction and search of a digital video library. It ran from 1994 to 1999 with the video content mainly from governmental education material and television news (see Figure 1-5). Using the CMU Sphinx speech recognition system they automatically convert the audio to text using automatic speech recognition (ASR) and create unique methods o f browsing and enhanced playback.
»--.my1-1 v
( * • C<it • t * « u • > ' - * l « v ' • M N h «T fwM f IM un* r«4f| l« f
Figure 1-5: Screen Grab of the Inform edia interface1
Informedia 2 is the title for the current project and focuses on retrieval and summarisation of television news-based content. Their archive of news material stretches back to middle of the 1990’s and provides a large corpus for evaluation. In the second version of the project they incorporate temporal and geographical location search from the ASR and users can navigate video content via a map of the world and a timeline bar to retrieve content of interest. The group have also
1 Taken from the Informedia Home Page at: http://www.informedia.cs.cmu.edu/ 19
developed a very sophisticated face-matching system and this is incorporated into the indexing and retrieval process [Wactlar, 2001] [Wactlar, 2000],
1.6.3 The CueVideo System
The CueVideo system was started as a project from within IBM’s Almaden Research centre in 1997, and concluded in 2001. It was developed to address the need for an automated method of video indexing and provide improved methods for browse and search. The researchers recognised that the index and retrieval were key areas for video retrieval.
CueVideo incorporated and enhanced IBM’s Via Voice speech recognition technology for automatic audio to text extraction (ASR) and developed a number of novel browse and playback tools for video retrieval. Their system was developed in two main modules, the first was the indexing and server utility that took content in a variety of formats, created an index and made this available over the Internet. The second module was a browse and search application that facilitated the users search and retrieval over the available indexes [Amir et al, 2001] [Niblack et al., 2000],
1.6.4 Selected CDVP Media Projects
The following are a number of media projects from the Centre fo r D igital Video
Processing [CDVP].
One project M ediaAssit is developing tools for digital image organisation, with the growth of digital cameras (outselling their analogue counterparts) the need for improved photo organisation is clear [CDVP].
A second project L ’OEUVRE is developing techniques for automatic indexing, browsing and linking o f digital video information. Object detection is one o f the main features under development in this project [CDVP].
The third project Adaptive Information Cluster is developing software that can filter and personalise large amounts of digital information. A large amount o f information from the Internet to Mobiles can be personalised for people’s specific interests [CD VP],
The fourth project is Fischlar on a PDA looks at the browsing of digital video on small portable devices like the iPAQ [CDVP].