5 First Prototype
5.3 First prototype Design
5.3.1 Capture Tools
This section shows the cornerstones and backbone of lifelogging technology; the capture and archiving of data and metadata. The main aim of this section is to illustrate what data and metadata the prototype intends to capture and how. In addition, I introduce the lifelogging devices that the prototype uses to collect the proposed data and illustrates the reasons behind selecting each of them. The prototype employs three devices to collect data and metadata in both indoor and outdoor activities (see figure 5.1).
The first device called CatCam is used to capture digital photos, the next section discusses its main features and the rationale for using it (Yonezawa et al., 2009). A Nokia cell phone was used in this prototype to capture three different types of data: location information, call history (incoming, outgoing, and missing calls), and sent and received short instant messages (SMS).
Chapter 5 – First Prototype 92
F
inally, I used personal computers to passively capture and archive the computer screen every five second, as well as to keep a copy of every email sent or opened.There are three main reasons for using the CatCam camera, cell phone and personal computer. First, I believe that these devices could capture and provide rich data and metadata that people can use and rely upon to trigger and augment their memory in a simple way. Second, using familiar devices, which have the ability to capture more than one type of data passively, would assist people in behaving naturally during the data collection period. Finally, novel information could be generated by integrating all the collected data together. For instance, when we link photo(s) with location information, we would obtain the place the photo(s) were taken. In addition, combining such data with phone call data could tell us more information about the place and sometimes the conversation itself. The following table shows the output of each device that is used in the first prototype.
Device Capture Metadata
CatCam Photos. Date, time, and GPS -
location information data. Nokia cell
Phone
Call history: Incoming-calls, Outgoing calls, and Missing calls.
Date, time, and GPS - location information data. Short messages: Received and Sent.
Date, time, and GPS - location information data. PC computer Take a copy of each opened emails and
capture PC screen every five seconds.
Date and time.
Table 5.1 First prototype devices and their different types of output
As shown in Table 5.1 date and time data could be used to link more than one type of the collected data together and generate new information. In addition, GPS (location information) could be used to distinguish between indoor and outdoor activities.
Chapter 5 – First Prototype 93
5.3.1.1 Photographs – CatCam
I argue that digital photos could augment human memory and support people to create effective cues to trigger their memory to remember and retrieve their past activities. For example, one childhood photo could prompt the memory of a past life experience and encourage the individual tell a related story even if the contents of that photo does not offer any details. One of the most famous devices used to collect digital photos for lifelogging is called Vicon Revue (the device was introduced in detail in the previous chapter).
It is a wearable electronic device combining a digital camera with a number of built-in sensors such as accelerometer, temperature, and infrared sensors. In this prototype study I did not use Vicon Revue device for two main reasons. First, Vicon Revue was not
Figure 5.3 the CatCam Camera.
Chapter 5 – First Prototype 94 available for purchase at that time and it is still under development as of the date this chapter was written.
The second was to illustrate that lifelogging photos could be collected by any other passive device, such as a cell phone camera. In this study I use a small wearable camera called CatCam to capture photos passively (see figure 5.3). The camera could be worn with a neck strap to capture a series of still images at specified time intervals. In this study I set the camera to capture an image every 10 seconds resulting in 6 photos per minute. Previous researchers have used a camera to capture a maximum of three photos per minute.
I adopted the high rate of photo capture for two main reasons. First, I believe that more images in a small time interval could allow people to record a more detailed account of their activities and events of their life experience. This would give them more opportunity to create values cues to remember and retrieve more detailed information about their past activities. Second, to test the ability and the speed of the prototype while indexing, displaying and searching the vast amount of archived data.
In addition, there are many other technical features encouraging me to use CatCam in this prototype such as the photo resolution, the size and weight of the camera, battery
Chapter 5 – First Prototype 95 lifetime and memory storage. CatCam camera could be set into one of three types of resolution (low, normal, or height). Figure 5.4 shows a sample of photos captured by CatCam at low resolution mode. The battery lifetime depends on the time interval of capturing photos.
For instance, one battery could be enough for five hours when the camera is set to capture a photo every 10 seconds. On the other hand, the camera stops capturing photos without any warning when the battery becomes flat or the memory card becomes full. In addition, we cannot set the date and time of the device. This means the photos do not have a timestamp, an essential marker to enable the photos to be associated with other data that has been captured at the same date and time.
5.3.1.2 Location data - Smartphone and custom software
Petrelli et al and Sellen et al proposed that combining location information with visual images could help people to remember their past events and actions (Petrelli et al., 2008); (Sellen & Whittaker, 2010). Eldridge et al in their study “forget-me-not” found that phone calls could trigger human memory at work to remember different activities (Lamming & Flynn, 1994). Bisht et al linked their collected emails, documents, SMSs and phone calls to their Computer and Google Calendar (Bisht et al., 2007). They found that SMSs and phone calls could support people in remembering their past actions and activities.
A Smartphone captures GPS data, received and sent SMSs, and call history. There are many devices that capture location information data but I used a Smartphone for two main reasons. First, I do not want to distribute lifelogger across many devices. Secondly, cell phones are perceived to be an essential item for the majority of people and are carried almost everywhere. In addition, the GPS allows data to be crudely tagged as either being captured outdoors when there is GPS signal or indoors when there is no signal.
5.3.1.3 Screen and email capture – Custom software
MyLifeBits is the only known project which is concerned with computer activity, specifically the collection and storage of every web page that is visited and any applications that are opened by their users (Gemmell et al., 2006). The fact that computer screens display the actions and activities performed by users on their computers allowed me to develop a software program which can log a screenprint every
Chapter 5 – First Prototype 96 five seconds. A screenshot is taken only when the user has moved the mouse or used their keyboard. This helped improve storage capacity by reducing the capturing of redundant data. Taking a screenshot, rather than storing the files that are open, avoids the problems that handling multiple different software programs and their associated file types brings. In addition, after a period of time, it might not be possible to open some file types as a result of changing the application format. Similarly, simply recording internet browser history was not satisfactory as we were unable to access some web pages visited by the users due to the pages moving to a different address, being reconstructed, or not existing anymore.
Emails which include significant details can also trigger reminiscence. For this reason, the software application passively downloads emails sent or opened from
“Microsoft office outlook” and archives their details. The date and time of an e-mails
arrival adds value to lifelogging data, but in addition the date and time of opening and reading the email would be more valuable and could make a great impact on memory when linked with other data captured at the time of reading the email. For this reason the prototype is tailored to extract detail from the email file type alongside screenshots of it when the file is open for reading.