Knowledge Work: Active Reading and Education

Another important application domain application domain for PPI is active reading and knowledge work, [Hong et al., 2012], as well as the related field of education [Oviatt et al., 2006]. Recent studies confirmed the fundamental importance of paper artifacts and their affordances with respect to practices associated with knowledge work [Harboe and Huang, 2015]. As such it provides a prime use cased of PPI. Al- though not as inherently mobile as note-taking, this domain contains several mobile and nomadic use cases. Active reading needs support in the library, at home and in the office. Education encompasses classroom activities, field trips and exam preparation that might occur in mobile or nomadic situations.

Knowledge-Work: Existing Systems

It has been shown, that reading on paper better supports cognitive processes than reading on a computer screen [O’Hara and Sellen, 1997]. This is partly because paper documents facilitate navigation [Sellen and Harper, 2003], but also due to the conve- nient way in which pen and paper allow handwritten notes and annotations supporting the cognitive processing of contents. Thereby, the process of simultaneously reading, taking notes, underlining, annotating, excerpting and in general ”working” with the contents of a document is referred to as active reading.

At the same time, digitally augmented workplaces offer interactive capabilities re- quired by today’s knowledge workers, in particular when combined with digital pen and paper technology [Gebhardt et al., 2014]. As such, digital pen technology of- fers a convenient tool supporting practices of knowledge workers, while at the same time opening the stage for the use of digital information, e.g., the immense knowl- edge source of the web [Steimle et al., 2008c]. Thus, several applications have been designed to support active reading by means of PPI.

Active Reading. Norrie et al. introduced a special digital library setup, where

users can follow hyperlinks printed on paper using PPI to access digital content on computers in the library [Norrie et al., 2008]. Similarly, PLink, [Steimle et al., 2011], supports active reading at the office desk connecting digital and physical workplaces through PPI. Here users can link paper artifacts and quick notes on large paper sur- faces with digital resources, e.g., web sites, through cross-media links. This concept has also been used in commercial applications designed to support education and ac-

tive reading, e.g., the Leapfrog Leap Reader system19. The targeted setup of these approaches is, however, a stationary environment; mobile use is not supported. In ad- dition, these systems lack support for many practices typically associated with active reading, e.g., content structuring and collaboration.

To overcome this, Steimle et al. introduced CoScribe, a sophisticated PPI based learning platform for students [Steimle et al., 2008c, Steimle et al., 2008a]. It sup- ports content structuring and collaborative practices, e.g., sharing of structuring and indexing information [Steimle et al., 2008b]. It can be stationary deployed on Desk- top PCs as well as on Tabletop computers. In the Tabletop setting, CoScribe fea- tures intuitive interaction techniques to issue cross-media links, i.e., links between interactive regions on paper and such regions on digital documents [Steimle, 2009a]. In a similar approach using pen and touch enabled table-top displays, Matulic and Norrie found that this combination supports active reading practices far better than normal pen and paper combined with digital tools, i.e., without digital pen support [Matulic and Norrie, 2012]. As such it poses a valuable application domain for PPI, however, neglecting the factor mobility.

With respect to knowledge work beyond active reading, the domain of education provides a vital field of study for PPI and mPPI based approaches: recent studies have shown that despite the broad availability of digital media today, pen and pa- per remain the predominant tools for knowledge work in the context of education [Malacria et al., 2011].

Education. Applying PPI to support education in general, beyond active reading

alone, has been emphasized by Oviatt et al. [Oviatt et al., 2006]. They showed in a comparative study that PPI outperforms other types of interaction, i.e., tablets with stylus input and PCs, when it comes to classroom use due to the un-intrusive inter- face. PPI based systems let the user concentrate more on the main task, e.g, solving a mathematical problem, and introduce less interruptions at the interaction level. Thus, PPI based systems support the cognitive processes of learning better than purely dig- ital systems [Oviatt et al., 2006]; especially in the mastering of complex skills, e.g., in mathematical education [Oviatt et al., 2007, Leitner et al., 2010]. In this context Oviatt, Cohen and Weibel also published a research corpus of data on mathematical education consisting of digital ink, speech captures and photos that was obtained using digital pen technology [Oviatt et al., 2013].

Toward PPI based applications for education, Miura et al. introduced a PPI based system designed for interactive classroom use [Miura et al., 2007]. The central as- pect of this system is communication between students and teacher: the teacher can adapt content and explanation during a lecture to the students’ needs by interactively

following the notes and questions written by students. To achieve this, each stu- dent uses a digital pen to take notes and solve exercises [Miura et al., 2010], which the teacher can subsequently review. Real-time processing of notes is supported through the interactive operation mode [Sugihara et al., 2010b]. The system design is thereby based on an extensive study on the respective needs of students and teacher [Sugihara et al., 2010a]. However, mobile settings, e.g., during homework or self- study periods, are not supported.

PaperCP, [Liao et al., 2009], follows a similar approach enabling real-time inter- action between students and teacher through PPI. Here university students can take notes and submit questions, sketches, exercises etc. in real-time on an anonymous submission channel to the instructor. Similarly, U-Note, [Malacria et al., 2011], ex- plores temporal association of digital, e.g., audio, web pages, slides and videos, and paper based learning material through PPI. These approaches provide valuable first steps toward mobile systems in this context as, e.g., U-Note provides access to ma- terial on a student’s personal mobile device. However, they neither support full user mobility, e.g., data entry in mobile settings, nor consider document mobility.

Knowledge-Work: Discussion

Related work regarding knowledge work, especially with respect to active reading and education, shows that this domain too is an important application domain for PPI. Pen and paper are essential tools in contemporary knowledge work practices. Their benefits have been clearly shown in multiple contexts, e.g., facilitating cognitive process associated with learning. PPI has been successfully applied to provide support for knowledge work.

Thereby, existing approaches mostly focus on stationary settings, although mobile use cases for knowledge work exist, e.g., nomadic use in the library or mobile support for learning during field trips. Initial steps toward mPPI, e.g., by providing mobile access to learning material including notes taken on paper, exemplify the potential of mPPI based applications for knowledge work. However, the full potential of mPPI, as well as supporting infrastructures, conceptual frameworks and interaction theories, remains yet to explore.