Systems using Multiple IV Techniques

This section describes visualisation systems that incorporate multiple IV techniques to visualise multiple PI types. Visualisation systems discussed in this sub-section include MyLifeBits, the Email Archive, ZOIL, PI Dashboard and InfoMaps.

3.5.2.1. MyLifeBits (2002)

MyLifeBits (Gemmell et al., 2002) is a system used to store media and documents in a Structured Query Language (SQL) Server database. MyLifeBits, as described in Section 2.4.5.3, is focussed on using collections (i.e. annotations) and search to replace the current hierarchical organisation. Thus, MyLifeBits supports PIM using multiple visualisations, annotations for non-text media and transclusion for authoring. Transclusion refers to the authoring tools used that allow “two- way links to media that they include in new media” such that a link represents one PI item annotating another PI item (e.g. a photo being used in a presentation) (Gemmell et al., 2002; Evequoz and Lalanne, 2007).

Searches in a user’s PIC can be visualised using a timeline view (Figure 3-8), a clustered-time view, a detailed view and a thumbnail view. Time and date information as well as a possible location or descriptor of a media item is stored. The detailed view provides a list of PI items with each attribute. The thumbnail view displays reduced images of the PI items using a grid. The timeline view (Figure 3-8) displays these thumbnails over a timeline which can be adjusted. The clustered-time view allows grouping thumbnails by time.

Figure 3-8 The Timeline View of MyLifeBits (Gemmell et al., 2002)

The design of the UI in MyLifeBits focusses on information density, reducing the need of the user to have to find hidden information, minimising the requirement of identifying what an item represents using thumbnails, modifying the size of the thumbnail upon selection and providing preview and other optional windows for displaying additional information about a certain PI item. Annotation is one of the primary goals of MyLifeBits as items may be annotated in a group, providing support for audio annotations and providing a toolbar in the web browser to record visited web pages.

MyLifeBits makes use of stories providing an Interactive Story By Query interface to allow a user to search and create stories from items in the search result. Stories are shown using a slide show or a time sheet. These stories are used to annotate each PI item included within them.

While time is an important aspect of PIM, it should not be the only consideration (Gemmell et al., 2002). Although a timeline-based visualisation with annotation support may be useful for media PI items, the UI is search-based (Latif and Min Tjoa, 2006). No user studies have been conducted to determine to what extent these visualisations support PIM. An observation that can be made relates to the possibility of limited scalability of the timeline visualisation technique used in MyLifeBits (Figure 3-8).

3.5.2.2. Email Archive (2007)

Evequoz and Lalanne (2007) designed a PIM tool consisting of visualisation techniques to visualise a user’s digital memory (Section 2.4.5.7). Browsing is supported by this tool to allow a user to find specific items of information. Email is used as the primary source of metadata to group information according to social, thematic and temporal organisations. The aim was to provide multiple visualisation techniques to visualise PI to support visual query refinement, such as filtering, with each organisation requiring its own visualisation. Simple visualisation techniques were implemented to visualise the groupings of information as can be seen in Figure 3-9.

(a) (b)

Figure 3-9 The Email Archive Visualisation Techniques (a) A Social Network Graph (b) Thematic Hierarchical Clustering (Evequoz and Lalanne, 2007)

At the time of publishing, thematic grouping was visualised by a treemap, where social grouping was visualised by a social network graph using a spring layout with limited functionality. This research seems to be still in progress and so visualisations were only created for two of the clusters. Additionally, only email was supported at the time and only browsing was supported by the visualisation system. Limited functionality was supported by the system. It remains to be determined to what extent these visualisations support PIM.

3.5.2.3. Zoomable Object-Oriented Information Landscape (ZOIL) (2008)

ZOIL (Jetter et al., 2008) follows the following design principles with the domain model incorporating these principles: object oriented UIs (OOUIs), a PI item is considered an object and classes represent different objects with attributes; semantic zooming, zooming in to reveal more content and supported functionality, zooming out to view an icon; nested IVs, representing PICs as portals and allowing selection of sub-portals within a portal and visualising this sub- portal; information space as an information landscape, providing all PI items on the screen as an infinite space; and nomadic cross-platform UIs, allowing access to multiple devices using a web- application (Section 2.4.5.9).

Jetter et al. (2008) identified a need for ZOIL to be extensible and customisable. PI items are considered as objects with properties, with each PI type representing a class containing attributes, metadata, links, functionality and how the item should be viewed. PICs are arranged according to the activity where the user selects the items in a portal, the location of the portal on the screen and the visualisation technique to be used, thus integrating different types of PI in a single PIC. Additional visualisation techniques can be downloaded as plug-ins. ZOIL allows for panning to browse the information landscape (Figure 3-10). In 2009, Gerken et al. moved the ZOIL UI to surface computing. Surface computing, in this case, referred to collaborating on mobile, tabletop or wall-sized devices using ZOIL; and Squidy, another software tool developed as an interaction library to unify different device toolkits.

Two key features of ZOIL were highlighted, namely OOUIs and ZUIs. OOUIs were considered to support organising and visualising co-operative objects, which can be used for direct manipulation using touch and gestures. ZUIs were identified as being useful as they use a natural presentation to display large volumes of information without needing WIMP (Windows Icons Menus Pointer) principles.

Sources of information for ZOIL include objects in a digital library, a file system, emails in a web-based email account, calendar dates and users of a social networking website (Gerken et al., 2009). The landscape is referred to as a virtual canvas of unlimited size and resolution. Searching, by means of a keyword search with the possibility of creating a portal with the search results, and browsing, using panning and zooming, are supported. The main strength of ZOIL is contributed to its flexibility in distributed and collaborative situations (Zöllner, Jetter and Reiterer, 2011). A client is provided with the opportunity to select a section of the landscape to view and also select which visualisation to use.

The latest definition of ZOIL refers to ZOIL as a “novel design approach and software framework for post-WIMP DUIs (Distributed UIs) in interactive spaces” (Jetter et al., 2012). A DUI is an interface that distributes components among different monitors, devices, platforms, displays and users.

This system is one of the only systems that unifies PI from multiple devices and different PICs. Currently, ZOIL only supports the Microsoft Windows platform and is not entirely multi- platform. Additionally, the UI only scales for multiple devices to enable viewing and so the UI is not customised to support the constraints and considerations of each type of device. One of the key issues with ZOIL is that it is not clear on which device the PI items are stored. It is also not clear which types of visualisation techniques are supported for PIM.

Although user studies were conducted, these studies were mainly focussed on determining to what extent the ZOIL framework supports developing applications and not whether the visualisation techniques support PIM. It also needs to be determined how the spatial orientation of the landscape may be perceived by the user and whether the selected visualisation techniques are the most appropriate techniques to visualise the different types of information. It is not clear how personal and shared information spaces will be supported using ZOIL for surface computing. Finally, the framework seems to focus on collaboration on a large display and not for other devices such as a mobile phone except for “cooperative” devices, for example using a tablet to define the interaction and a tabletop for collaboration.

3.5.2.4. PI Dashboard (2012)

The PI Dashboard (Aires and Gonçalves, 2012) makes use of a dashboard metaphor (Section 2.4.5.12; Figure 3-11). This system supports the use of plug-ins so that a user can personalise his/her dashboard with each plug-in using a visualisation. Plug-ins are correlated so that selecting an item in one plug-in updates the display in all other related plug-ins. Each plug-in provides interaction ability so that a user can select different items, update time ranges, find out more information of an information item by hovering over the item, and so on. The system is extensible in that new plug-ins can be created and new types of PI can be added as sources.

Figure 3-11 The Personal Information Dashboard User Interface (Aires and Gonçalves, 2012)

Different types of visualisations exist in the PI Dashboard. For example, a tag cloud visualisation, called Keyword Cloud, is used to display the most prominent words from emails, posts and tweets and is shown in the top-middle of the interface in Figure 3-11. Bubbles are used to display email messages shown as the third plug-in, called You’ve Got Bubbles, at the top of the interface in Figure 3-11. In the bottom-middle of the interface is the Who&How plug-in, that displays activities with each contact that a user has, arranged using a circle with edges connecting concentric areas. Bars represent activities such as emails received from a contact and posts on Facebook. Filtering is supported, with the addition of fadeouts and highlighting to encode filter

results. The Details-on-Demand task is supported using tooltips to provide additional information of a specific item on the screen.

A usability evaluation of the PI Dashboard provided positive results (Aires and Gonçalves, 2012). Cross-referencing information from multiple sources, plug-in communication and the number and type of sources were identified as positive features of the system. Participants identified the need for accessing and creating email from the system, which the system does not currently support.

The system is used as a tool to only visualise information of a user. Although the system provides a web interface, information stored over multiple devices was not considered. From the design of the UI displayed in Figure 3-11, it seems that the system was not designed to consider constrained devices such as a mobile phone. The sources of information primarily focus on social networks to view patterns, and even though email was considered, documents, media and calendar information do not form part of the focus of the visualisations.

3.5.2.5. InfoMaps (2012)

Background images can act as landmarks for familiarity or in relation to the PI items’ metadata in InfoMaps (Section 2.4.5.13; Figure 3-12) (Kolman et al., 2012). A background image can be selected by the user upon which the user can spatially place PI items to relate these items to the background image. Queries and keyword filters can be used to group related information on the screen and searching is also supported. A PI item is described by its URL (Uniform Resource Locator), title, description and data published/modified attributes. PI items are represented by a thumbnail image. Hovering over a PI item will display a popup information window displaying the PI item’s metadata. A PI item can also be opened by double-clicking on the item, where a web page will open to view the item. Various layouts are provided including grid, horizontal, vertical, cascaded and radial layouts. A complementary framework, referred to as Weave, is used to provide co-ordinated and different views using visual tools.

Currently, only RSS feeds and PDF documents are supported. No documented evaluations could be found on the InfoMaps UI and there is no evidence indicating what devices are supported. The sources of the information items are also unclear. There is also no discussion on the suitability of the IV techniques incorporated within the tool.

Figure 3-12 The InfoMaps User Interface (Kolman et al., 2012)