Cross-sensor-jumping

I found that graphical movements were very useful in helping users to search, locate and navigate to their new action or activityas well as segment and group their data into a component event. For example, the main door movements allowed users to navigate to that period of time and combine that movement with the photos to talk about actions and activities that happened at that time; [EW11-19] “... I returned back to my house ...

that’s the front door movement”. In addition, the information and memories obtained

from one step could be used to encourage users to move forward and combine more available data about that event and extend their talk. For example, participant (E) talked about the weather status when he combined the virtual Google earth data with his hand and body movements; [EW11-23] “… it was a little bit cloudy, the phone captured the

GPS signal after approximately six minutes”. He cited his intention and he mentioned

some conversations that he had with the kids on the journey as he followed his path on the map; [EW11-25]“… my daughter asked about the reason of doing that, I did not

find a good answer for her”. Furthermore, few movements could support their users

memory unless combined with any other data, and this encouraged users to combine more data to find the reason of this. For example, participant (E) described an event when he found no hand and body movements and almost all of the photos were the same; [EW11-28] “… No movement for more than ten minutes ... I was looking at a

magic show in the height street”. He cited his feelings and emotional state; [EW11-28] “...It was wonderful ... many children were happy”.

Chapter 7 – Evaluation of IsstLog 143 Later in the data collection and analysis process, when trying to make sense of events, users utilized a combination of bookmarks and their own intuition or innate knowledge as a tool to navigate through and narrate the events themselves. The participants would look across the different streams of data on the timeline, looking for particularly exceptional markers in order to further their story. For example, participant (E) jumped into the next period of time containing noteworthy movements of his hand and body to find and describe the next event. He wanted to discuss “ ... we moved when

the bus arrived … I did not stop moving my back stand”. The available data encouraged

him to travel mentally back in time. He cited some thoughts; “the bus chair was not

comfortable at all… the first two drivers drove very smoothly, but the last one does not”.

It is also interesting that the memories of the physical aspects of the event are elicited by the sensor data. In another case, participant (R) recalled the start time of an event when she combined the photos with the first objects’ movements. She described her place and activities at that time; “… I left the TV working when I was in the kitchen …

nobody was watching the TV”. There are two main uses for multi-sensor-jumping that

users apply to locate and talk about their events. These are...

Participant (E) described actions and activities when he located the log movements of his hand and body and combined them with the photos on the timeline; [EW11-98]

“…I can see there the movements stopped for few second and then started again,”. He

mentioned places; [EW11- 99] “…those photos show we are outside of the culture lab

building”, [EW11-101] “… that photo show we were in the electronic shop”, and he

cited his feelings and emotional state; [EW11-102] “…he was confused about the prices, he cannot decide which one to buy”.

I found that users sometimes navigated on the timeline to search for a lack of hand and body movements to locate the next event and then they would combine the lack of movement with the photos of that period of time to describe their past actions and activities. For example, participant (E) combined a lack of movement with the screen images to track and monitor his actions and activities on the computer; [EW12-105] “…

here, I was checking my email’s Inbox … there, I was working on my thesis document...”

Chapter 7 – Evaluation of IsstLog 144 I found that photos that were captured with no body and hand movements differed from those captured at moving time. The former could tell us that it was something interesting in front of the users, especially when they spent more than 30 seconds looking at something, which means that the complete scene would be stored in long term memory and that memory could be retrievable. For example, participant (E) mentioned a different memory that did not appear explicitly in his data when he combined the photos with none movements; [EW31-71] “... I stopped there ..., I was

looking at the statue of the queen mother ... I have been there many times ... many tourists were there”. He cited this new place and actionswhen he navigated forward on

the timeline and located a new period of time which lacked hand and body movements; [EW31-73] “... those photos were taken at the bus station … we arrived just 10 minutes

before the departure time ... I did stop moving in the bus station ... I met Mike with his girlfriend there”. He mentioned his intentions when he moved forward on the timeline

and saw his hand and body movements; [EW31-74] “… we are on our way to the bus”, and he added some actions and cited some conversations; [EW31-74] “… that woman

was talking to me ... she asked me to swap with her ... she likes to sit with her group”.

7.4.2.1 Jump out to talk about related event then jumped back

Users navigate forwards or backwards on the timeline to locate and access indicators that are related to their current event to extend (or complete) telling their narrative of an event, before then returning back to the original place in the timeline to continue their search and talk about the next event. For example, participant (E) navigated backward on the timeline to talk about an SMS that he mentioned during his talk. He cited the sender name and the contents of the SMS.

He described his place when he combined the SMS with the photos and with his hand and body movements; “… that SMS I received when I was in the shop”.

Chapter 7 – Evaluation of IsstLog 145 He went on to discuss his intentions and what he was doing;“...I bought some stuff from

their”. He returned back to the previouspoint when he had finished talking to locate the

subsequent photos , which he combined with his body and hand movements to describe the following actions and activities. He talked about places; “… those photos were

captured in the Hyde Park”, and describe what he was doing at that time; “... here, I was sitting, neither body nor hand movements” (see figure 7.18).

Participant (R) played back video of a particular hour to search for a particular lunch event. She did not find any indication that she had had lunch at that selected time, but she found some photos showing her friend. The data encouraged her to remember a phone call she received from her friend and she navigated backward to locate that phone call. Describing it, she mentioned the time when she combined the phone call with the photos; “... Sue called me at morning”. She mentioned some conversations; “… she told

me her Mum would like to visit me”.

After talking about that phone call she returned back to the previous point on the timeline and continued to talk about her lunch. She relied on multiple sensors to jump to her target data. She searched for some objects’ movements. She described actions;

“…there, I can see the movements of the cooker, the fridge, and the microwave”, she

described her actions in more detail when she navigated forward on the timeline and combined the objects’ movements with the photos and with her hand and body movements; “… I cooked food that day … those photos there show I was cooking some

food” see figure 7.19).

Chapter 7 – Evaluation of IsstLog 146

7.4.2.2 Jump to specific times

Users often navigated forwards or backwards in time to access specific points of their daily routine such as time of work, school, exercise, breakfast and lunch. I found that daily routines could encourage users to identify specific hours to search that could include their target goal instead of searching multi hours to achieve this. In addition, this allowed them to generate a retrogressive search by ‘jumping’ away form and then returning to the previous point to continue their searching. For example, participant (R) jumped backward to the hour that she expected she had a lunch at and then she navigated to the next one when she found none of the four candidate photos related to that event. She navigated to the next hour and did the same, finding that the first candidate photo showed some food. She mentioned some thoughts; “…they just drunk

their hot chocolate with some biscuits …they did that many times… they refused to eat any food at that morning … the boy started first and then his sister followed him”.

I found that daily routines could allow users to bypass time and navigate directly to the hour closest to that which could include their target goal. For example, participant (R) restricted her target goal in between two hours; [RW22-148] “…we always had a

supper at 7 or 8 O’clock night”. She did not care about the four candidate photos, but

she directly played back video of the first hour. She explored that hour’s data on the timeline and combined the photos with her hand movements and with the movements of kitchen objects. She mentioned place; [RW22-150] “…all photos here show me in the

kitchen”, and she described what she intended to do; [RW22-151] “…I was preparing some food”, [RW22-152] “… here, I am cooking”. The available data encouraged her

to cite thoughts; [RW22-153] “… it was the first time I cook that kind of food”, [RW22- 154] “… one of my friends cooked for me when she invited me to her son birthday

party”

Participant (R) navigated straight away to a specific hour and combined the four candidate photos with her hand and body movements. She moved forward to the next hour when she found no indication of lunch at that time. She explored that hours data on the timeline and described her actions and activities; [RW22-35] “… they did not finish

Chapter 7 – Evaluation of IsstLog 147