IsstLog UI: Date visualization and interaction

The primary challenge in designing a user interface for lifelogging data gathered by multiple heterogeneous sensors is allowing users to turn raw data about activities into more meaningful events. The UI to IsstLog was designed to allow visual exploration, association, and grouping of the data generated so that users could search, browse, and ultimately create visual cues and bookmarks. This means logged data must be uploaded, organised, and indexed to be usable (see figure 6.5).

Chapter 6 – Second Prototype 124 I believe that lifelogging interfaces should be more productive, approachable, simple, and fun, while being able to accomplish their users’ goal and allowing them to utilise their data interactively. That means when the user highlights any of their available data, all of the data connected to it should also be highlighted. For example, clicking on Vicon Revue photos would activate the virtual Google Earth to explore the location of that photo on the map. In addition, all accelerometer graphs would show movements of the objects to which the accelerometer was connected and the movements of their body and hand at that time.

I used Google Earth instead of simply Google maps because I found that the C Sharp programming language include powerful classes and functions to control Google Earth more easily than Google maps.

With the first prototype I concluded that a photo’s location sometimes works better than the photo content itself in terms of helping the lifelogger to travel back in time and remember not only their events but their emotional state as well. I argue that location information would be very useful for people to recall past events, especially when they cannot remember or recognise some of their photos’ contents. For instance, knowing the time of one photo and its location on Google Earth may be sufficient to recall past events and actions that happened at that time and in that particular place. This is the

Chapter 6 – Second Prototype 125 main reason that I was encouraged to build a user interface that was able to combine both the virtual Google Earth with a timeline on one screen. I believe that this facility could help users to know the order of their events by presenting the time of occurrence and the location information for each date that is captured outdoors. The next section explains this in further detail.

6.3.1 Designing the IsstLog Interface

I believe that lifelogging interfaces should be productive, approachable, simple, and fun, at the same time as accomplishing their users’ goal and allowing them to annotate their own data for later reflection. Figure 6.6 illustrates the IsstLog interface. A photo’s location may sometimes work better than its contents to help the lifelogger to mentally travel back in time to remember not only events, but their physical and emotional states as well. For example, seeing the time of one photo and its location could be sufficient to enable recollection of events and actions that happened at that time and in that particular

place. For this reason, IsstLog combines Vicon Revue photos, phone calls, text

messages and best guesses about whether the user was indoors or outdoors at the time of capture with the location of the data on Google Earth. User’s positions are captured

Figure 6.6 IsstLog timeline view, candidate photos are displayed in the bottom right, an hour’s activity can be seen in the adjacent columns, the green column represents an hour in which the activity took place outdoors, the movement data is seen in the graph plot, the Google earth view is in the top left crner, the video of the hours photos is playing centre top and bookmarked items are navigate in the two top right boxes

Chapter 6 – Second Prototype 126 every second and IsstLog compares and pairs photo, phone call, or SMS timestamps with GPS timestamps +/- 5 second. IsstLog offers a calendar to select and access data from a specific day and allow users to navigate between their data. (See figure 6.6) Days on which activities took place are highlighted to save users’ time and to avoid them selecting and opening dates that do not contain any activities. Selecting a specific date gives a graphical view of all the activities within 24 hours, starting from 8:00am and ending at 7:00am as shown in figure 6.6. In the first study I found that most people’s activities started at 8:00am, and this time was discussed with the participants in the evaluation of the first prototype.

Each activity is presented graphically as vertical lines on the hours pane with a different colour. I chose to do this for two main reasons. The first is to provide a complete summary of a specific day. I expect that users could identify the type, quantity, time, and place of any activities just by looking at their activities graphic lines. The second reason is to save users time and keep them from opening hours that do not contain any activities. Indoor and outdoor hours are distinguished by using different background colours to allow users to identify where they sent most of their time.

Selecting specific hours allows users to monitor the movements of their hand, body, and their objects, and also to monitor four photos, each representing a quarter of the hour, that I call candidate photos. When an hour is selected these four photos appear. Control keys also appear on the top of the photos to allow users to navigate to the first or last photos of the hour, move forward or back step by step, play back the photos of that hour as a video, or explore that hour’s data on a timeline view. The control keys allow users to display the selected hours’ photos as a movie at a speed of five frames each second, which means that 350 Vicon Revue photos would be presented in one minute and ten seconds, the number of frames and the video speed discussed and argued in the first prototype. Control keys provide other functions such as “pause”, “continue”, and “stop”, to allow users to access their photos. Furthermore, they allow them to navigate forward or backward to provide them with more of an opportunity to think about and associate their data, especially during the searching process. Such facilities would assist easily users with searching and display their collection easily.

The timeline is split into six pages, each representing ten minutes, to display photos, screen images, SMS messages, phone calls, emails, movements of the hand, body, and objects, and the electrical consumption graph. The timeline page shows a timeliness

Chapter 6 – Second Prototype 127 indicator moving from left to the right that zooms in on any data when the indicator reaches its time. All the photos taken play like a video. In addition, Google Earth also moves to show the places where that data was captured. Users can drag the time indicator forwards or backwards to locate any of their data on the timeline (see appendix B for more details). They can drag and drop any photo they find valuable to save it in the bookmarks. Finally, they can create visual cues; these are made from single pieces or combinations of data and appear enlarged in the timeline view to highlight specific events at a glance.