As described in the introduction of this chapter, 20 month textual lifelogs annotated with several rich sources of context data test sets were created to evaluate our ranked retrieval approaches. To facilitate our investigations into the utility of biometric re-sponse in retrieval, one of these months (September 2008) was further annotated with biometric data. This means that all items which were created or/and accessed during this one month period were assigned the biometric levels associated with the creation or/and access(es) to the items observed during this period. SenseCam images were also included in the lifelog for this month (September 2008). That is while subjects wore the SenseCam for the 20 month lifelogging period, in the experiments presented in this thesis we only use the one month of their SenseCam collections for which we also have biometric data and hence only this month of SenseCam images are added to subjects’ lifelog databases. In this section we describe the structure of these lifelogs.
The means by which the database tables were populated through lifelog data capture and derivation is described in the subsequent section, Section 3.3.
Figure 3.1: Lifelog database structure showing database tables, fields in each table, and links between tables. Fields in bold font are used to link tables. Section numbers, shown in red font, indicate the sections which describe how the lifelog data stored in each field was captured or derived.
Each of our 3 subjects lifelogs was stored locally on their PC in a SQL database. Figure 3.1 depicts the structure of the database and provides pointers to the sections describ-ing the loggdescrib-ing and derivation of the personal data stored in the database tables. The database contained an items table, item access table, Campaignr table3, weather tables (one for each geo-location visited by the subject during lifelogging period), light status tables (one for each geo-location visited by subject during lifelogging period) and a biometrics table. The structure of these tables, along with sample fictitious data are shown in Tables 3.2 - 3.7. The remainder of this section describes these tables.
Items Table:
The items table maintains a list of all distinct items accessed by subjects, see Table 3.2.
Specifically a unique id is stored for each distinct item, along with title, application used to open the item (e.g., WINWORD.EXE, EXCEL.EXE, IExplorer), item content, to
3The Campaignr table stores geo-location and people present context data. This table derives it’s name from the ‘Campaignr’ software which logs the data from which geo-location and people present context data were derived, described in Section 3.3.4.3.
Unique ID 528
Table 3.2: Items table fields and sample data. This table holds all unique items accessed by a subject.
Table 3.3: Item access table fields and sample data. This table holds all accesses to items by a subject.
and from information (for emails and SMS messages), file path, URL (for webpages) and extension type (e.g., java (for java code file), excel (for XLS and XLSX files), word (for DOC and DOCX files), web (for webpage viewed), email (for emails sent and received), SMS (for text messages sent and received on mobile phone), SenseCam (for SenseCam images captured)).
Item access Table:
The item access table, see Table 3.3, maintains information related to the date and time for each individual access to computer items which can be linked to the items table based on unique item ids. Specifically, the following date and time related in-formation is held in the item access table for items: “begin” date and time of the access to the item, “end” date and time of the access, along with year, month, sea-son, day of week, whether the access happened at the weekend or during the week (part of week) and whether the access happened at the beginning-of-week, mid-week or at end-of-week (part of week1). The item access table also records the device on
Date 2009-08-24 2009-10-01 2009-11-09
Time 14:32:00 16:01:25 19:51:22
country code IE IE IE
country Republic of Ireland Republic of Ireland Republic of Ireland
region Dublin City Wicklow Dublin City
Table 3.4: Campaignr table fields and sample data. This table holds all geo-locations and people present experienced by a subject.
Date 2009-08-24 2009-08-24 2009-08-24 Time 13:00:00 14:00:00 15:00:00
weather Light Rain Clear Scattered Clouds
Table 3.5: Weather table fields and sample data. A weather table was created for each geo-location visited by a subject. Weather tables hold the weather conditions experi-enced by a subject.
which the item access occurred. In our lifelogs the device is set to laptop or PC for computer items. We do not actually maintain accesses to SMS messages sent/received and SenseCam images captured in our lifelogs, rather just the “date-time” stamp for SMS message send/receive and SenseCam image capture is stored. The item access table also holds these timestamps and sets the device field to ‘mobile phone’ for SMS messages sent/received and to ‘SenseCam’ for SenseCam images captured. Sections 3.3.2 and 3.3.3 discuss this topic in greater detail.
Campaignr Table:
The Campaignr table records details on the geo-locations in which a person was and the people who were in their presence, along with date and time information, see Table 3.4. Geo-location and people present data is indicated for every 20 seconds (polled once every 20 seconds) in time over the 20 month lifelogging period (described in Section 3.3.4.3). Specifically the following fields are in the Campaignr table: date, time, country code, country, region, county, city, street, street number, postal code, people present. This table can be linked to the item access table based on date and time information present in both tables.
Weather Tables:
A separate weather table was created for each country or region (e.g. regions in the
Date 2009-08-24 2009-08-24 2009-08-24 Time 14:32:00 14:33:00 14:34:00 light status daylight daylight daylight
Table 3.6: Light status table fields and sample data. A light status table was created for each geo-location visited a subject. Light status tables hold the ambient light status experienced by a subject.
Table 3.7: Biometrics table fields and sample data. This table holds all raw biometric data captured for a subject.
USA = Arizona, Chicago) visited by the subject over the 20 month lifelogging period.
Hourly weather information was available for each country or region (described in Section 3.3.4.3). Each weather table holds date, time and weather conditions informa-tion, see Table 3.5. The weather table for the appropriate region can be linked to the item access table based on date and time information and geo-location information present in the Campaignr table.
Light status Tables:
At the end of the lifelogging period, a separate ambient light status table was created for each country or region visited by the subject over the 20 month lifelogging period, see Table 3.6 for the structure of a light status table. Each light status table contains date, time and light status fields. Light status is represented as ’daylight’ and ’dark’
as two classes. Light status is present for every minute in a day (described in Section 3.3.4.3). Similar to weather data, light status data can be linked to the item access table based on date and time and geo-location information held in the Campaignr table. Note, since we only require weather and light status information for each region for the periods in time when the subject was in these regions, we only stored light status and weather information in each region’s light status and weather tables for the periods in which the subject was in these regions, as determined from the geo-location and date and time information in the Campaignr table.
Biometrics Table:
Biometric data is held in the biometrics table. The biometrics table contains date, time,
GSR, HF, HR, ST and energy expenditure fields (described in Section 3.3.4.4). This tables structure, along with sample data is shown in Table 3.7. The biometrics table can be linked to the item access table based on date and time stamps.