Proposal: An Optimised Interoperability Layer

5.2.1 Philosophy: a hybrid approach to device ecologies

One of the design principles elicited in Chapter 0 is about minimal interference in users’ choices. This, in a way, rules out a high-level “meta” user interface as per some literature examples (Bardram, Gueddana, et al., 2012; Bragdon et al., 2011), as those are more suited to scenarios with a more formalised process that the more ad-hoc device ecologies which are the focus of this thesis.

Any proposed approach in a new system would leave users almost complete freedom of choice when it comes to tools, limiting any additional software strictly to the management of a meeting sessions and/or the support of common sharing tasks. In a scenario of ad-hoc collaboration, no formal process has been defined by users. In a group, a number of users may be using Microsoft Office™ to create their documents, sometimes edit them collectively using a cloud tool like Google Drive. Some users may be using a Linux distribution, rendering access to commercial software more difficult. The device ecology should remain agnostic to the applications in use; if tool incompatibilities are detected, it may offer some users an accessible alternative to ensure people work with compatible tools.

This approach, named hybrid in the context of the project, can be deployed across a same user's access points, e.g. a user can be editing a paper on a laptop using Microsoft Word in full screen. In the meantime, the same user's smartphone can be used as the interface from which they control the relationship between the laptop and the rest of the device ecology, e.g. the visibility of certain files with other users, or the connection of the laptop to a wireless screen sharing protocol. Another reason for the choice of a hybrid approach over a high-level interface is also a matter of ease of deployment. ReticularSpaces works because the people using the space all belong to the same professional group and share a number of processes in common. In a scenario where a guest collaborator is joining such a group, it may be too costly in terms of time or money to bring their technology on par with the very specific set of software used by the main group. A hybrid approach allows the guest user to join a collaborative session with their own tools.

The device ecology would analyse which tools are present on the guest's devices and offer a compromise to ensure interoperability with the main group.

To do this, it is important that devices speak the same language when joining the device ecology. A solution for this is the deployment of a compatibility framework, describing the generic capabilities and specific compatibility issues inherent to each device wishing to join the ecology.

5.2.2 An extensible framework called OIL

Devices creating or joining a collaborative ecology need to talk to each other and be aware of each other's generic capabilities and specificities.

Generic capabilities encapsulate a comprehensive number of information about their nature, such as:

• Their type, e.g. room hardware, mobile devices, wearables, etc.

• Their form factor, e.g. small pocket devices, medium-sized laptops, large collaborative screens. A starting classification for form factors can be taken from (Terrenghi et al., 2009).

• Their mode of input, e.g. touch, mouse and keyboard, gestures. • Their operating system, e.g. Windows, macOS, Android.

• The type of functionalities they can provide, e.g. presentation software, screen sharing, storage.

• Their output capacities, e.g. sound, screen, haptics etc.

However, generic capabilities encompass a breadth of heterogeneous devices running different platforms. A Linux platform will not, for instance, be able to access Microsoft Office; however, it will be able to access Office documents using other applications. An Apple device will not be able to mirror its desktop using Miracast, it will however be able to use a different technology such as AirPlay™ or ChromeCast™. This is where the more specific descriptors of compatibility play a role, mapping the generic capabilities to more technology specific applications and technologies.

In the framework, each device identifies itself with its name and owner. It contains a set of generic capabilities. Each generic capability contains set of technological descriptors.

Figure 29 Extensible framework hierarchy

5.2.3 Infrastructure

The design for a hybrid system consists of two layers: infrastructure (OIL) and user interface. Those two layers could work together in order to help supporting the device ecology.

One design principle for the support of ad-hoc collaborative scenarios is the idea of sessions. Sessions are already a cornerstone of existing forms of collaborative software (ref: Solstice, GoToMeeting) and although they are not a new idea, they are an essential feature to delimit the scope of a device ecology amid the context of a meeting journey.

In the context of a smart space, sessions can be running on a local machine acting as a server. Users join sessions when located in-situ, their devices being by their presence on the local network, or by their proximity to the space, or both.

In the context of an ad-hoc collaborative device ecology, sessions can be created on the fly by a first device. Their existence is maintained as long as one device or more is present in the ecology, in the same fashion as an ad-hoc wireless network. The presence of the device that initiated the session is not essential to the

Device: - Name - Owner Generic Capability 1 Generic Capability 2 Etc… Descriptor 1 Descriptor 2 Descriptor 3

persistence of the collaborative ecology, allowing a form of "plug-and-play" approach to joining and leaving a device ecosystem.

The OIL infrastructure could consist of a device acting as a server managing a session. There are two scenarios of use:

• Users are joining a room where a computer can act as a permanently installed server

• Users are creating an ad-hoc ecology, in which case the first device to initiate will act as a server, until the end of the session, or until it leaves the session and passes the server responsibility to another device.

Each client device contains a generated set of descriptors, using the format described in the generic framework. There are multiple descriptive languages that can support such a descriptor (XML, yaml etc).

In a scenario where two laptops connect to a meeting room (ICE like), this can be as in Table 7:

Table 7 an example of ecology descriptors

Device Generic attributes Specific attributes

Room Screen 46in Multitouch Video-conference Wireless sharing Windows Platform Windows 10 Skype

AirPlay (third party) and Miracast

Laptop 1 12in. screen Linux

Presentation software

Ubuntu LTS LibreOffice Laptop 2 13in. screen

Apple Mac Presentation Software Wireless Sharing macOS 10.14 Keynote AirPlay™

Additionally, the identifiers should provide whether actions are automatable or not; if not they should provide clear human-readable instructions how to perform the tasks manually. The descriptors should be able to provide alternative software solutions.

Figure 30 High-level functional description of the interoperability layer

The server acts as a session manager, it allows the following functionalities: • Joining or leaving the session

• Publishing the different devices and their capabilities

• Managing a common set of files and assets using a content management system; it can be a simple shared folder, a cloud storage system or a more complex system to be determined.

Session (server or master-device) Device Device Device Screen Screen CMS / Dropbox Content push/pull Screen share Ident* / screen / content requests

*Ident: descriptors for devices and screens:

- Type / Size / Resolution - Operating system - Available software etc

Applications Launch / close /

automate / state**

Target

*State: an application / document state so it can be retrieved identically on another device, e.g.

• Acting as a proxy delivering the appropriate network identifiers and addresses, so devices needing to connect to each other can do so directly; alternatively, if an incompatibility between protocols (e.g. AirPlay vs Miracast) or it appears to be impossible to automate an action (e.g. screen sharing) it will deliver clear instructions for users to do this manually.

5.2.4 Discussion

The OIL proposal is not implemented in the scope of this research project due to its considerable technical ramifications. Additionally, it does not exist in a vacuum. It touches notions of middleware that exist in the domain of automation (Toschi et al., 2017), the Internet of Things (IOT) such as generic enablers (Brogan and Thuemmler, 2014).

In the domain of Ubiquitous Computing, Modahl et al. (2004) have reviewed a number of low-level building blocks and technologies. Network auto-discovery technologies include Apple’s open-source Bonjour (Apple Developer, n.d.).