Analytic Comparison: Monolithic vs Distributed Pipeline

This section provides a detailed analysis and demonstrates why the standard, mono- lithic processing pipeline used in existing infrastructures introduces conceptual prob- lems with respect to mobility requirements derived in section 2.3.1. In comparison to this, it analyses how the novel mPPI infrastructure introduced throughout this chap- ter addresses these requirements and shows that using the distributed PPI processing pipeline as conceptual underpinning enables use of PPI in the mobile domain. R1: Support for User Mobility

Support for user mobility (R1) refers to support for PPI in situations where the user is on the move. This includes nomadic settings, e.g., at the office desk, and mobile situations, e.g., in the train or while walking. As derived in chapter 2, section 2.3.1, supporting user mobility requires the infrastructure to support the interactive mode in mobile settings (R1.1), a mobile platform (R1.2) and a flexible deployment to adapt to changing environment conditions (R1.3).

R1.1: Interactive Mode While the monolithic processing pipeline in principal sup- ports using the interactive mode, the lack of means for resource sharing and interactive region discovery forces applications to use a client/ server based ap- proach for the mobile domain (as described in the next section). Here mobile

devices can only serve as proxies accessing the digital pen hardware, while sending all data to an application server hosting the pipeline deployment. This prevents the interactive mode from being used on mobile devices where instant feedback on user actions is a central requirement.

The distributed pipeline supports the interactive mode just as does the mono- lithic pipeline. However, the flexible concept of resource sharing among mul- tiple pipeline instances allows using fully distributed systems instead of ei- ther monolithic deployments on the mobile device, or client server based ap- proaches. This makes it possible to deploy required stages, i.e., those com- ponents critical to provide instant feedback to the user, on the mobile device without adding to the distribution complexity. As a result, employing the in- teractive mode on mobile devices becomes easily possible via the distributed processing pipeline and its concept of shared processing resources.

R1.2: Mobile Platform Existing infrastructures are designed for the stationary use case. As a result, the sheer size of the monolithic deployment using the generic pipeline does not allow for a deployment on some mobile devices, i.e., it inher- ently does not satisfy the mobile platform requirement. However, with increas- ingly powerful mobile devices available, this limitation becomes purely a matter of implementation. The only conceptual limitation of the monolithic approach is the distribution of applications, i.e., the mobile device would need to host all applications and the user would have to install them manually. This cumber- some manual step contradicts the flexible and light weight style of interaction that makes paper such a convenient medium.

The concept of required and optional processing stages enables applications to only use those processing components which are actually needed, while remain- ing flexible if additional services are required at runtime. This is supported by the PSIs, as required data can be obtained at any processing stage interface and thus at any stage in the pipeline. This allows considerably reducing the foot- print of the PPI processing infrastructure providing ease for the development of mobile platform ports of the infrastructure. Furthermore, the distribution of applications becomes convenient and flexible due to the plug and play like interconnectivity of processing stages via PSIs. This avoids the cumbersome manual setup of new applications on the mobile device and allows a flexible and lightweight style of interaction with paper documents on mobile platforms. R1.3: Flexible Deployment The most important problem of the monolithic de- ployment of the generic PPI processing pipeline with respect to user mobility is the lack of support for flexible deployment. The monolithic deployment does not allow for distribution of processing intensive tasks, e.g., handwriting recog-

nition. However, such options are required in the mobile domain as discussed in section 2.3.1, for instance to save battery. Furthermore, the monolithic deploy- ment forces all components to be hosted on a single processing entity. There is no dynamic redeployment that could cope with problems arising out of the mobile setting, e.g., temporary connection loss. Hence the monolithic set-up severely limits the deployment flexibility, required to support user mobility in changing environments. A more flexible approach is required.

In contrast to this, Flexible deployment and sharing of resources form the ba- sic concepts of the distributed processing pipeline. This enables mPPI based applications to distribute processing intensive tasks, e.g., handwriting recogni- tion. Distributions thereby bases on flexible deployment schemes, as shown in the first example in Fig. 3.3 a. on page 91. This mechanism can be used to save battery. Moreover, the deployment flexibility inherent to the distributed pipeline allows for dynamic redeployment of processing stages, e.g., in the case of temporary connection loss to a backend server.

The distributed PPI processing pipeline supports this by defining clear process- ing stage interfaces and flexible ”wirings” between the processing stages. Using the hotplug mechanism based on the service discovery mechanism of the under- lying middleware it allows discovering available processing stages and adapting the pipeline layout dynamically. Connections are handled completely network transparent, that is: it is not important whether a processing stage is hosted lo- cally, or is distributed in the network. Hence truly flexible deployments can be achieved.

R2: Support for Document Mobility

Support for document mobility (R2) refers to support for PPI in situations where the paper artifacts are mobile. This includes encountered documents, e.g., where the user receives a document, and passed on, or disseminated documents, e.g., interac- tive leaflets send out by a supermarket chain. As derived in chapter 2, section 2.3.1, supporting document mobility requires the infrastructure to support resource sharing (R2.1) and interactive region discovery (R2.2).

R2.1: Resource Sharing The most severe limitation of the monolithic deploy- ment is its intrinsic lack of resource sharing. In a monolithic approach, a re- source is associated with a single pipeline instance. Apart from being a waste of resources, this setup makes it difficult to support sharing of resources. Hardware resources become blocked by a pipeline instance. This prevents, e.g., using the same digital pen in multiple applications simultaneously. A pen connection usually requires manual set-up (i.e., bluetooth pairing of the pen). Therefore

switching a pen to another pipeline instance is slow, cumbersome and error prone in the monolithic setup. Although manual setup cannot be circumvented, the monolithic deployment aggravates this situation by ”blocking” the pen re- source, thus effectively locking the pen within a single pipeline deployment, i.e., an application or a collection of applications from the same distributor. To alleviate this problem, some monolithic infrastructures, e.g., iServer / iPa-

per, [Norrie et al., 2006a], and PADD, [Guimbreti`ere, 2003] (with respect to in- teractive region discovery), allow using a client / server based approach. In this approach all applications are essentially hosted on a central server and the pen data is simply dispatched to that server. However, this approach introduces lim- itations with respect to scalability as the central server provides a performance bottleneck. In purely document centric applications with a low degree of inter- activity, this solution might hold. However, the performance penalty obviates using the interactive mode of the pen in mobile settings (c.f., discussion of user

mobilityabove).

In contrast to this, sharing resources between processing pipeline instances is one of the fundamental concepts in the distributed PPI processing pipeline. Here it is possible to share interaction resources, e.g., digital pens, as well as process- ing resources, e.g., gesture recognizers. This allows for better resource utiliza- tions and prevents pipeline instances from blocking resources. For instance, the distributed pipeline supports using the same pen on different applications by sharing the driver stage between pipeline instances. The cumbersome manual setup, i.e., the bluetooth pairing of the pen with a device, has to be done only once.

This non-exclusive resource access also considerably broadens the spectrum of possible use cases. For instance, it is now also possible to share the same paper document among applications via shared use of the region stage. This allows for more natural interaction with paper documents, e.g., a sheet of paper used for a drawing application can simultaneously serve as interaction medium for a note-taking application; in this setup, both applications simply receive digital ink of the same interactive region.

R2.2: IR Discovery As described in section 2.3.1, local storage of all interactive re- gions cannot be achieved. In order to support the interaction with encountered documents, the infrastructure needs to support interactive region discovery. In the monolithic approach, the processing infrastructure is deployed on a sin- gle device. This neither helps nor hinders the discovery of interactive regions. However, the monolithic pipeline design prevents deployments from sharing knowledge of interactive regions. This puts a lot more stress on the discovery

mechanism compared to a setting where this knowledge would be shared across pipeline instances, e.g., if two applications are run on the same device. Hence the monolithic setup implies an unnecessary performance penalty.

By contrast, the dynamic discovery of interactive regions forms a central mecha- nism in the mPPI infrastructure based on the distributed PPI processing pipeline. Here it is essential to support interaction with encountered documents at the conceptual level.

On the one hand, components of the infrastructure can be shared across pipeline instances In the distributed PPI processing pipeline. Although this in itself does not provide a discovery mechanism for interactive regions, the possibility of sharing the region stage across pipeline instances reduces the stress put on the discovery mechanism as knowledge of existing interactive regions can already be shared here. As a consequence, the distributed approach decreases the per- formance penalty introduced by the monolithic approach, e.g., if two pipeline instances are hosted on the same device and share their knowledge on interac- tive regions.

On the other hand, support for interaction with encountered documents demands for distributed and highly scalable mapping of interactive regions to applica- tions. At the same time, the dispatching of digital ink to interested parties must be handled efficiently and the solution must scale to a global dimension. In the mPPI infrastructure presented throughout this chapter, this is solved using the 2-level peer-to-peer based approach combined with local sharing of the interac- tive region model, i.e., the LIRM. Integrated into the pipeline’s region process- ing stage, this 2-level Peer-to-Peer approach allows distributing the knowledge on regions efficiently in the local network by sharing the LIRM between in- stances of the region stage. As this is backed by a global overlay net for region publishing, the IRNS, it supports the dynamic definition of interactive regions needed to integrate interactive surfaces for both paper-like and other interactive surfaces, as well as the publishing of interactive paper documents.

Summary

In summary, mPPI infrastructure based on the distributed PPI processing pipeline in- herently supports the requirements associated with user mobility and document mo- bility. This stands in contrast to the monolithic deployment of the generic PPI pipeline commonly found in existing PPI infrastructures that faces conceptual challenges when tackling these requirements. As a consequence, the distributed processing pipeline allows supporting mobile usage practices better and thus enables infrastructures to provide support for mobile PPI as demonstrated.

Requirement Anoto SDK Li v eScribe / SP3 SDK MIL SDK PADD + P apierCraft P aperT oolkit iServ er + iP aper Letras User Mobility R1.1 Interactive Mode x x x x x x x R1.2 Mobile Platform - x - - - (x) x R1.3 Flex. Deployment - - - x Doc. Mobility R2.1 Resource Sharing - (x) (x) - - (x) x R2.2 IR Discovery (x) - - (x) - x x

Table 3.2: Mobile PPI Support Comparison

Table 3.2 summarizes the analysis given above. It depicts the support for mobile PPI requirements as provided by the novel mPPI infrastructure through its reference im- plementation Letras in comparison to the architectures of existing PPI infrastructures (c.f., section 2.3). In summary, Letras provides support for mobile pen-and-paper in- teraction through the introduced distributed processing pipeline, enabling developers of PPI based applications to provide interactivity to physical paper, without compro- mising its inherent mobility.