Interactive Region Publishing and Discovery

In order to fully support the interaction with encountered documents as character- ized by document mobility, the infrastructure must be able to cope with previously unknown interactive regions. This is the task of the interactive region discovery com- ponent (IR discovery). This problem has two facets: first, applications require a mech- anism to publish digital representations of their interactive regions so that they can be found by the infrastructure; second, pipeline instances require a mechanism to actually discover these interactive region representations.

3_{This concept of required and optional components can even be broken down on a sub-stage level: For}

instance, an application might not need multiple drivers for different digital pen models, or might only want to support a trivial interactive region discovery mechanism. In this case not all services of the required PS might be relevant and a minimal deployment might cherry pick those which are. Please refer to section 3.2 and 3.3 for a discussion of concepts enabling these micro-deployments.

In the distributed interaction processing pipeline, the region processing stage dis- patches digital ink to components interested in particular interactive regions. Thereby, it needs to utilize the mechanism for discovery in order to associate digital ink with new, previously unknown interactive regions. Following the principle of high cohe- sion, i.e., grouping related functionality in a single component, the region PS thus also serves as entry point for publishing of interactive regions by applications.

An application registers its interactive regions through digital representations of interactive regions at a region stage instance. The region stage then stores all known interactive regions in its interactive region model. This model is subsequently shared with other regions stage instances in order to allow for interactive region discovery. The publishing and discovery of interactive regions thereby boils down to construction and search of the shared interactive region model.

Here the distributed processing pipeline already provides an inherent architectural advantage. Several deployments can share the same region stage, hence these already share local knowledge of interactive regions. However, this alone does not suffice: in order to provide high performance processing of digital ink as required by interactive systems employing PPI with instant feedback, the dispatching of digital ink cannot be handled on a single, central component. Round-trip times for requests and scalability issues render this approach inadequate. Thus, sharing of interactive region models between different instances of the region stage needs to be supported.

In the distributed processing pipeline this sharing of interactive region models fol- lows a 2-level approach as depicted in Fig. 3.5. This bases on the principle that the infrastructure essentially provides the logical driver layer in an assumed meta op- erating system for smart spaces [Rom´an et al., 2002], where the publishing and dis- covery of interactive regions follows the locality principle suggested by Hartl et al. [Hartl et al., 2002]: cascading levels of interoperation between components are de- scribed starting from components in the direct vicinity of the user to interoperation on global scale with backend connectivity. The two levels of sharing employed in the distributed interaction processing pipeline are

1stLevel: Local IR discovery This level refers to local sharing of interactive re-

gion models between all region stage instances in a certain fully-meshed net- work, e.g., in a local area network. The model is fully synchronized between all instances. This facilitates intra-organizational publishing and discovery of interactive regions as required in order to satisfy R2.2 (c.f., chapter 2, section 2.3.1). Not all interactive regions visible at this level need to be also visible on a global level, i.e., an application might re-use an interactive region belonging to another application. This mechanism supports document mobility in the sense of the same document being re-used in a different application, e.g., the phone call notes scribbled at the side of a drawing sheet.

2 nd Leve l C h e ck IRNS N1 N2 N3 N4 N5

Region PS 1 Sync Region PS 2 Sync Region PS 3 Sync Region PS n

1st L eve l Driver Stage PSI IPen Semantic PS PSI IRegion Application PS PSI IInk Region PS Ch e ck

Figure 3.5: 2-Level approach to publishing and discovery of interactive regions

2ndLevel: Global IR discovery Publishing and discovery of interactive regions

on a global scale corresponds to construction and search of a global interactive region model. This global IR model is thereby constituted of all local inter- active region models. The system allowing for search and construction of this global model is referred to as the Interactive Region Name System (IRNS). The IRNS allows the inter-organizational sharing of knowledge on interactive re- gions required to satisfy R2.2 (c.f., chapter 2, section 2.3.1) and thus supports interaction with previously unknown interactive regions crossing organizational boundaries. Consider for example an interactive, PPI enabled leaflet distributed by a retailer to its customers (as in the DGL case study, c.f., section 3.4.3). Here, the PPI / mPPI processing infrastructure at the customer side uses the IRNS to discover the application adding interactive functionality to the leaflet.

Thereby, lookup for an unknown interactive region is initiated by the region stage. The approach is as follows: First the local region model is checked, whether it knows the region. If the region cannot be found there, the lookup request would be delegated to the IRNS to retrieve the part of the global region model that is associated with the particular unknown interactive region.

sample routing path from node 1 to the node responsible for point (x,y): 2 down, 1 left G G h y y> + (R1) : down G G h y y> + (R2) : down (R3) x < xG : left 1 3 6 2 5 4 (x,y)

Figure 3.6: Sample region discovery in the CAN based IRNS (source: adapted from Ratnasamy et al. [Ratnasamy et al., 2001])

Interactive Region Name System

Guimbreti`ere [Guimbreti`ere, 2003] compared the lookup system for interactive re- gions (or in his narrower interpretation, for interactive paper documents) to DNS for paper. This is based on the fact that in terms of the problem size it corresponds to, if not exceeds, the Domain Name System (DNS)4. However, because of the size of the resulting global model and its inherently high degree of change due to different organizations controlling its interactive regions, as well as highly dynamic definition of interactive regions (e.g., compared to the less dynamic assignment of hostnames to IP addresses in the DNS), centralized storage of this global model cannot be achieved. However, Peer-to-Peer (P2P) systems offer decentralized data storage solutions to this problem: distributed hash tables (DHT) [Balakrishnan et al., 2003]. Here a network of nodes store the data. For any lookup in the shared data, it is sufficient to know a single participating node. Then, just like in a hash table, these systems map a key via a hash function to its associated value. The employed mapping func- tion also provides optimized routing within the network of nodes storing the val- ues. Typical systems utilize an Overlay Network of notes, a term that refers to a logical network of nodes on top of an existing network, e.g., the Internet. Numer- ous approaches for realizing efficient construction and routing (which corresponds to search or discovery) of DHTs exist [Stoica et al., 2001, Ratnasamy et al., 2001, Rowstron and Druschel, 2001, Zhao et al., 2001].

In the domain of PPI, Weibel suggested a P2P based approach for the lookup of paper documents [Weibel, 2009, p. 199]. However, the virtual binary search tree used in the underlying P-Grid system [Aberer et al., 2002] does not utilize a key characteristic of PPI: the perfect 2-dimensional hash function provided by the global positioning mechanism used in the underlying digital pen technology. Therefore a DHT is much more promising regarding the lookup and construction of IRNS. With a perfect 2-dimensional hash function given, a very efficient routing and overlay network construction can be achieved using a Content Addressable Network (CAN) [Ratnasamy et al., 2001].

The CAN for the 2-dimensional case allows directly associating nodes with certain macro regions in the underlying pattern space, while the partitioning of the space can still remain very flexible to avoid waste of pattern space. Each node participating in the IRNS overlay network manages a rectangular area of the global coordinate space. It stores the part of the region model that encompasses all regions within this area. If a region stage instance performs a lookup in the IRNS, it needs to inquire at a single previously known IRNS node providing a point in the global pattern space coordinate system as reference, i.e., a coordinate it had received to which the region stage cannot relate any region within its local region model. Now the IRNS node checks whether the reference point lies within its rectangular region. If this is the case, it returns the relevant part of the global region model. If not, it delegates the lookup along either thex or the y axis to its neighboring nodes in the CAN based IRNS. An example for such a lookup is shown in Fig. 3.6.