Interaction Techniques for Co-located
Collaborative TV
Abstract
We propose a number of interaction techniques allowing TV viewers to use their mobile phones to view and share content with others in the room, thus supporting local social interaction. Based on a preliminary evaluation, we provide guidelines for designing interactions to support co-located collaborative TV viewing.
Author Keywords
Social TV; multiple mobile devices; second screen; mixed-focus collaboration; video
ACM Classification Keywords
H5.2. User interfaces: Graphical user interfaces (GUI), Input devices and strategies, Interaction styles.
Introduction
TV viewing is generally a social experience. Families and friends gather together to view major televised events, such as sports, awards, and series premieres. Increasingly, viewers use their mobile devices while watching TV, to consume content, which may be related to the program being viewed or to check email or social media updates [3,7]. For many major
televised events, there exists online content which may be consumed in parallel with the show, either
professionally curated, as in the case of commercial Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
CHI 2014, Apr 26 - May 01 2014, Toronto, ON, Canada ACM 978-1-4503-2474-8/14/04. http://dx.doi.org/10.1145/2559206.2581257
Karolina Buchner Yahoo Labs 701 First Avenue Sunnyvale, CA 94089 USA [email protected] Roman Lissermann Telecooperation Group
Technische Universität Darmstadt [email protected]
Lars Erik Holmquist Grounded Innovation 1134 Noe Street
San Francisco, CA 94114 USA [email protected]
second screen apps, or user-generated, as in the case of tweets. On today’s mobile devices users can easily access this content.
In this work, we introduce and evaluate interaction techniques allowing a co-located group of TV viewers with multiple mobile devices to view and share content collaboratively with others in the room during live events, with the shared TV screen becoming an area for collaboration. We present an early user study
investigating how multiple users view video content on mobile devices in parallel with the TV and how they share this content among one another.
Related Work
Previous research in social TV has tried to connect remote users with each other and generate a feeling of togetherness [3,6,8]. Projects such as Motorola’s Social TV [5] do so by connecting households through their TV sets to facilitate communication and viewing together. Cesar et. al. [2] enable users to select video clips, annotate, and share them for asynchronous viewing. Commercial web products such as Zync and Wachitoo enable users to chat while synchronously viewing video content online.
Nowadays users connect and socialize with their mobile devices more then ever while watching TV [3]. Other works have tried to provide the user with additional information about the TV show on their secondary device while the user is watching the show [1]. Commercial systems, such as Airplay and Chromecast, increasingly allow users to transfer content between the TV and their personal mobile device.
We were inspired by mixed-focus collaboration [4] where users tend to switch between working together and working individually. Nowadays a co-located TV only allows viewers to watch content together without being able to switch their attention to watch content
individually. An opportunity exists to explore how the
user experience of watching TV in a social setting may be enhanced if the viewer is able to easily share synchronized content amongst fellow viewers and between the TV and their mobile device. We explore tighter integration between the TV and multiple mobile devices for both individual and collaborative viewing, and interaction techniques to support these activities.
A Prototype for Collaborative TV Viewing
The concept of our prototype system, pictured in Figure 1, is centered around the viewing of video content in parallel with the TV and sharing this content amongst fellow viewers, both privately via an app and publicly to and from a shared TV screen. The system enables users to consume content privately on their mobile device, but also brings them back to the social experience by providing awareness of who is viewing what and affordances for sharing between users. In our system, the user may:• Watch video on the shared TV
• Watch video on their individual mobile device • Share video from their mobile device to the TV • Control the display on the TV
• Share video privately to other users’ devices • Grasp content that has been shared to the TV
or from another user’s device, and watch it on their own device
Figure 1. Our concept: users view content in parallel with the TV, and are able to share and grasp content amongst fellow viewers.
We expand on the interaction techniques supporting these actions in the next section.
Scenario: Watching the Olympics
We chose to start our exploration with a scenario where users may gather to watch a live televised event together, where users may be interested in watching events happening in parallel to what is being watched on the main TV screen, or they may wish to view other feeds (e.g., different camera views) of the same event. In the traditional viewing experience, focus on the main TV screen is only on one event or one feed of a given event in the case of multiple feeds. We aim to break this single-focus viewing experience. In our prototype implementation, we include videos of various Olympic events to simulate the experience.
Implementation
We implemented our concept as a working prototype system, using a single server to render the shared TV display and coordinate sharing between devices, users, and the TV. Our server tracks which pieces of content are being viewed privately on the mobile devices, what has been shared between users and to the TV and by whom, as well as the offsets into the videos to keep viewing in sync between devices and the TV. The server runs on a Mac Book Pro laptop connected to a Sony Bravia KDL-55EX640 television. An Android app sends user actions back to the server and receives
notifications. We installed the app on two LG E960 Google Nexus 4 phones and two Google Nexus 7 tablets with all video content pre-loaded on each device. All devices where locally connected with each other using a Wi-Fi router and the KryoNet TCP communication framework.
Interaction Techniques
We propose the following interaction techniques for viewing and sharing video content on multiple co-located mobile devices.
Individual viewing on the mobile device
On the mobile device, the user is presented with three regions (see Figure 2a): the top video screen where individual playback occurs, the TV streams being shared currently to the TV, and the grid of TV channels available for viewing or the grid of user faces which we explain later. To select a channel for viewing on their device, the user taps the thumbnail for the channel in the grid. The video for that channel then starts playing in the video screen.
Sharing from the mobile device to the TV
To share a channel with others in the room, the user can tap and hold the channel thumbnail in the grid on their device. Once a channel has been shared to the TV, we refer to it as a stream. A stream is visualized on the shared TV screen as a smaller window along with icons indicating which users are currently viewing the same video on their mobile devices. The viewport of the video that was playing on the TV decreases in size but otherwise continues playing as before. An example of the TV display is illustrated in Figure 3.
Controlling the display on the TV
The user may change the display mode of the shared stream on the TV, by tapping and dragging a stream thumbnail on their device. When the user starts to drag the thumbnail, an overlay appears in the UI giving them the option to full screen the stream on the TV or to remove the stream from the set of shared streams. Figure 2. Regions of the UI (a), channel
selection (b), swiping to expose the grid of user faces (c), selecting a co-located user (d).
The user then drags the stream to one of two icons (full screen or delete) to change the TV display.
Grasping content from the TV to the mobile device
The user may also choose to view one of the shared streams on their mobile device. To do this, the user selects from the streams region on their device, swiping side to side, as in Figure 4, to browse shared streams, and taps the stream to view, in the same way as they select a channel.
Sharing from the mobile device to others in the room
The mobile UI provides a grid of user icons that is accessed by swiping the channels region to the left, as in Figure 2c. Each icon identifies another co-located mobile device represented by a user’s face. The user can share the video currently being viewed on the user’s own device to another person in the room by tapping and holding their face icon in the grid. The recipient’s mobile device UI then highlights an icon below the channels region indicating a video has been shared to them. The sender’s face is also highlighted in the user grid. By tapping a face one can grasp the view of another user, as explained below.
Grasping the view of others in the room
In order to view what others in the room are watching a user can tap another user’s face icon. Their device then displays the same video as currently watched by the other user. The video on both devices is played back synchronously.
Early user feedback
We have evaluated our working prototype and gathered early user feedback. Our study goals were to
understand (1) how multiple users are viewing video
content on mobile devices in parallel with the TV and (2) how they share and grasp content amongst each other. We were particularly interested in exploring if the interaction techniques were easy for users to learn, if their awareness of others in the room was changed when using our system, and if having the new
interactions available to them would enhance or detract from the TV viewing experience. Since playing videos on different devices can generate simultaneous audio playback, we also experimented with having devices playing or muting the audio.
We chose a within-subjects design with 12 participants (avg. 20-31 years, 8m, 4f). All participants reported that they watched TV at least once a week. Most participants had not met prior to the study, with the exception of P12 and P10 who were husband and wife. Study sessions were composed of 2-4 users in a group, in front of the shared TV in a living room set up in our lab. Every participant was given a device (phone or tablet) preloaded with our app. Video content of nine Olympic events was available on the devices.
Each session lasted approximately 80 minutes, during which data was gathered via semi-structured
interviews, questionnaires completed by the participants, and notes taken by the experimenter. Additionally, each session was also video recorded.
Procedure
In each session, the experimenter gave a brief system introduction followed by few minutes of trying out time for the user. Users were shown how to select and view different content on their individual mobile devices, with audio muted everywhere except the TV. Next, the sharing capabilities of the system were explained to the users, including sharing to and grasping from the TV, Figure 4. The user
swipes through the 'streams' that have been shared to the TV.
Figure 3. Main TV screen with shared streams below main video. Faces indicate which user is viewing the content on their mobile device.
and to and from others in the room. The users were then asked to watch the content and use the system’s capabilities at will. Finally, we enabled the sound on each mobile device, so that audio from the individually viewed videos was audible, and then asked the users to use the system as before.
In-study Questionnaire
Before and after audio was enabled, the participants were asked to complete a short questionnaire. The users rated the relative difficulty of performing specific tasks on a 5-point Likert scale ranging from easy to difficult. The questionnaire also probed on how often the users felt distracted by others in the room or felt they were disturbing others, on a 5-point Likert scale ranging from never to always. We also probed on awareness of what others were watching, feelings of watching together, and privacy concerns associated with feelings of being watched. The questionnaire also asked about awareness of and potential distraction due to sound. Additionally, users were given the
opportunity to provide any freeform written feedback as part of the questionnaire.
Results and Discussion
Interviews and observations were transcribed and analyzed using an open coding approach [9], and by aggregating the questionnaire responses. The ease of use of the system has room for improvement (see Table 1) and ratings fell in the easy to medium difficulty range. Based on user feedback, we propose the following guidelines for co-located collaborative TV viewing.
Support individual viewing
When content on the main screen was not interesting or exciting, P1 liked that “when things slow down I could watch other channels”. Individual viewing enables users to curate content for the group. At one point, one user was viewing the 100m running event, and shared the video to the main screen right before the race started. “For this kind of situation this is great” (P5).
Enable time-shifted video sharing
Some users said they would have liked the ability to share a video from a specific time offset in the video. “When I share to everyone the moment has passed already. Going back would be nice.” (P3)
Allow users to follow each other
The most surprising feedback from our sessions revolved around grasping content from other users. P3 stated that selecting what to view by tapping users’ faces “feels kind of like recommendation.” P12 stated that selecting faces is even better than selecting channels. P11 wanted to follow other users and have her own video updated automatically as the user she was following changed what they were viewing.
Future Work
An opportunity for further study is to explore the tradeoffs between collaborative viewing and perceived ownership of shared content, especially in the context of existing family protocols for TV viewing. Some participants expressed concern regarding loss of control over the shared TV in this setting. When sharing content to the TV, P12, the husband of the couple in our study, joked that “now we will fight”. P8 expressed Figure 5. Participants during the study in
the living room lab.
avg (sd) Q1: Selecting a channel
on the mobile device was…
1.4 (0.5) Q2: Selecting a stream
on the mobile device was…
1.8 (0.9) Q3: Sharing videos to the
TV screen was …
2 (1) Q4: Changing a video to
full screen on the TV was …
2 (0.9) Q5: Sharing videos with
others was …
2.8 (1.3) Q6: Receiving videos
from others was …
2.8 (1.4) Table 1. Average and standard deviation of task difficulty ratings (easy=1, difficult=5) for our system with sound muted on all devices except the TV.
that “I fear that somebody will change my content that I just put and [was] watching on the TV.”
Interaction techniques to focus on audio in a multi-device setting are needed. In the study, the response to sound being enabled on all devices was mixed and task difficulty appeared to increase with sound (Figure 6). P5 indicated that “noise to get my attention is cool”, whereas P6 “felt that we were watching TV in a bar because it was always loud”. It was also suggested that the treatment of sound might depend on the content: “For an Olympics scenario, sound on all devices is not as distracting as for movies.” (P4) Investigation of how co-located collaborative TV viewing fits into the larger picture of social TV is also of interest. As our study participants indicated, an extension of the system may connect living rooms and add the ability to follow other users in our multi-device context.
Finally, our current implementation limits us to in-lab studies only. We plan to revisit it in order to conduct in-home field studies to see how this multi-device viewing experience may fit into users’ lives.
Overall, we believe that as social TV viewing grows and more devices are available to users, the kind of
techniques and findings we have presented here will be useful to shape new interactive viewing experiences.
Acknowledgements
We thank our colleagues and study participants at Yahoo for their helpful discussions and feedback.
References
[1] Basapur, S. et al. FANFEEDS: evaluation of socially generated information feed on second screen as a TV show companion. In Proc. EuroITV 2012, 87-96. [2] Cesar, P. et al. Usages of the Secondary Screen in an
Interactive Television Environment : Control , Enrich , Share , and Transfer Television Content. In Proc. EuroITV 2008, 168-177.
[3] Courtois, C. dric and D’heer, E. Second screen applications and tablet users: constellation,
awareness, experience, and interest. In Proc. EuroITV 2012, 153-156.
[4] Gutwin, C. and Greenberg, S. Design for individuals, design for groups. In Proc. CSCW '98, ACM
Press(1998), 207–216.
[5] Harboe, Gunnar, et al. "Ambient social tv: drawing people into a shared experience." In Proc. CHI 2008, ACM Press (2008), 1-10.
[6] Lochrie, M. and Coulton, P. Sharing the viewing experience through second screens. In Proc. EuroITV 2012, 199-202.
[7] O'Hara, K., Mitchell, A. S., & Vorbau, A. Consuming video on mobile devices. In Proc. CHI 2007, ACM Press(2007), 857-866.
[8] Sahami Shirazi, A. et al. Real-time nonverbal opinion sharing through mobile phones during sports events. Proc. CHI 2011, ACM Press (2011), 307-310. [9] Strauss, A. and Corbin, J. Basics of Qualitative
Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, Thousand Oaks, CA, USA, 2008.
(a)
(b) Figure 6. Task difficulty ratings for (a) sharing content to the TV and (b) receiving content shared by others.
