Tangible programming refers to the act of manipulating graspable objects with the purpose of creating a structure or an arrangement that will be interpreted as a program. Such computer programs take form as structures or arrangements of physical elements. Systems developed by Bers and Horn (2010) and McNerney (1999) are examples of structures and arrangements that serve as programs. The act of creating a program this way has also been referred to as physical programming (Mukherjee, Sharma & Prakash 2002).
Even though the terminology tangible programming and physical programming are used interchangeably, I view them as distinctly separate and different activities. I consider the term tangible programming most fitting to this study and therefore limit my discussion on physical programming to Section 4.2.1 and elaborate on tangible programming in Section 4.2.2.
4.2.1 Physical programming
Physical programming has been described as the act of defining the interactions between objects within a room (Sherman et al. 2001). In turn, Montemayor et al. (2002) defines physical programming as “the creation of computer programs by physically manipulating computationally
augmented (or aware) objects in a ubiquitous computing environment”. Montemayor et al.’s definition has been applied to a programming environment that comprised of a room in which objects were scattered and programming was done by demonstrating what actions must be taken when an event occurs (also called programming-by-demonstration). Based on Montemayor et al.’s definition of physical programming, I define a physical programming environment as follows:
Physical Programming Environment
A physical programming environment is an omnipresent computing environment in which computationally augmented or aware objects are manipulated to create a program.
4.2.2 Tangible programming
The term tangible programming was coined in 1994 to describe the AlgoBlock system (McNerney 2004). For the purpose of this study, I will use tangible programming to describe the activity of constructing a program consisting of physical objects that can be grasped by hand and manipulated using the fingers. Used in this way, tangible programming excludes virtual objects such as visual symbols rendered on a computer screen.
Suzuki and Kato (1995b) describe a tangible programming language as being a program language with physical presence. A tangible programming language also serves as a mechanism with which to instruct a computer to take action (Bers & Horn 2010) and is comprised of elements. Wyeth &
Purchase (2002) use the term tangible programming elements to describe their stackable blocks that both form the computing structures and interact with the physical world.
According to McNerney (1999), the origins of tangible programming remains unresolved and it has been suggested that Perlman’s (1976) TORTIS system’s Slot Machine was the first implementation of a tangible programming language. In Chapter 6, I use the terminology tangible programming and tangible program elements to discuss my tangible programming environments.
4.2.2.1 The tangible programming environment
I base my distinction between physical and tangible programming environments on three concepts.
The first is what McNerney (1999) calls the graspable property of a tangible programming environment. The second is the nature of the (at times immovable) objects in the physical programming environment, and Montemayor et al.’s (2002) ubiquitous computing environment in the third. Based on these concepts, I now define a tangible programming environment as follows:
Tangible Programming Environment
A tangible programming environment is a computing environment in which a static arrangement of graspable objects constitute the program.
4.2.2.2 Useful characteristics of tangible programs
Using tangible objects as a computer program presents interesting characteristics, with some already explored by Fernaeus and Tholander (2006). Two characteristics of particular relevance to my study are the persistence of physical code and how humans interpret it.
Some tangible programming systems reuse the same object to create a program, yet others such as AlgoBlock (Suzuki & Kato 1994) dedicate a particular object to a specific position in the program. In the case where an object is dedicated to a specific position within the program the result is a program composed of physical artefacts (Suzuki & Kato 1995b) and the composition is referred to as graspable software (Montemayor 2001). To the user, a tangible program takes form as a collection of physical objects (Fernaeus & Tholander 2006) and often on a flat surface such as a table top. In contrast to text and graphic programs, the user is able to view the complete program at a glance without relying on technology to interpret an intangible program and render it as symbols on a video monitor.
In this study, I am interested in applying personally meaningful tangible objects as program elements that remain in place once the program has been constructed. Chapter 6 explores this concept further where I discuss the evolution of my T-logo programming environment.
4.2.2.3 Tangible programming styles
McNerney (1999) discusses programming styles that have successfully been applied to tangible programming environments. These include the imperative, functional, rule-based, behaviour mixing (with priorities), and database query programming styles.
A program constructed in the imperative style is evaluated sequentially with only one instruction being executed at a time (McNerney 1999, 2004). The program also has states and side effects due to its modifiable variables, data structures, and objects. The application of this style results in an algorithm that provides systematic instructions (also known as imperatives) for execution.
In contrast, a program constructed in the functional style is not evaluated sequentially, nor has side effects (McNerney 1999). When applying this style, the user only states what is to be computed but not how it should be done. An electronic spreadsheet is an example of a system that applies this style (McNerney 2004).
A program constructed in the rule-based style consists of separate rules, each of which can be modified independently of the others. McNerney (McNerney 1999) suggests that the rule-based style is suitable to tangible programming environments. I explore this style further in Chapter 6.
The behaviour mixing (with priorities) tangible programming style combines various behaviours and these are prioritised as part of the program definition. An arguable example of a tangible programming system using this style is Schweikardt and Gross’s (2006) roBlocks system as discussed in Section 4.3.1.3.
Finally, an example of a program constructed in the tangible database query style consists of stacks of physical query parameters: A single stack represents the logical AND statement in which all the conditions in the particular stack are evaluated as a group whereas multiple linearly aligned stacks represent the logical OR operation. Figure 4-3 illustrates this concept by means of a database query that considers eye colour, hair colour, and height.
Blue eyes
Figure 4-3 An example of a tangible database query
Based on McNerney (1999)
In Chapter 6, I continue to explore groups of tangible program elements by considering certain Gestalt principles of perception. For example, I propose that the stacks in Figure 4-3 need not lie along an imaginary line and that it is sufficient for the stacks to be perceptually grouped as Figure 4-4 illustrates.
Figure 4-4 The Gestalt principle of perceptual grouping applied to the tangible database query style
4.2.2.4 Tangible programming modalities
Lund (2003, 2004) describes programming-by-building as the assembling of discreet tangible blocks into a single machine. In such systems, the three-dimensional topology of linked objects defines the program. These links are either mechanical or inferred. An example of a programming-by-building system with mechanical links is Ngo and Lund’s (2004) I-BLOCKS.
The term tangible programming-by-example describes a programming modality by which the user manipulates tangible objects and instructs the system to store this interaction sequence. When using this modality, the user manipulates the objects whilst the system simultaneously monitors the manipulation sequence and object topology. This modality is also referred to as programming by tangible demonstration (Frei, Su, Mikhak & Ishii 2000), programming-by-demonstration (Knoll, Weis, Ulbrich & Brändle 2006), and gestural programming-by-example.
Some systems that incorporate the programming-by-example modality can animate user-generated manipulation and I call these kinetic-kinetic systems. Topobo (Raffle, Ishii & Yip 2007; Raffle, Parkes, Ishii & Lifton 2006; Raffle, Parkes & Hiroshi 2004; Raffle 2008) is an example and includes objects with embedded sensors and actuators. Using Topobo, the user creates a program by interconnecting the objects and physically manipulating them. This system then re-enacts the movements when instructed. Chung, Shilman, Merrill, and Ishii (2010) states that the kinetic-kinetic programming modality was pioneered by the Topobo project. However, this statement does not consider the CurlyBot project that predates Topobo. To program CurlyBot, a user gestures motions while simultaneously pressing the record button. Programmed motions are later re-enacted (Frei et al.
2000) when activated.
Certain programming-by-example systems do not animate physically and I refer to these as kinetic-passive systems. Chung et al.’s (2010) OnObject is an example of a kinetic-kinetic-passive system with which the user associates auditory responses to his manipulations. The associated auditory responses are later replayed when the manipulations are repeated. Active Surfaces (Grönvall, Marti, Pollini & Rullo 2006) and StoryRooms (Alborzi et al. 2000; Montemayor, Druin, Chipman, Farber & Guha 2004;
Montemayor et al. 2002) combined with StoryKit (Sherman et al. 2001) objects are two more examples of kinetic-passive systems.
4.2.3 Conclusion to this section
In this section, I made a distinction between physical and tangible programming by highlighting the differentiating aspects of each: First, physical programming requires a computationally ubiquitous environment whereas in a tangible programming environment this is not a requirement; instead, it is only the tangible object that requires computational abilities. Second, a physical program includes
immovable objects whereas tangible programming objects can be grasped and manipulated by hand.
The following section focusses on existing tangible programming environments. In particular, it distinguishes between those environments in which programming objects are given to the user (Section 4.3.1) and those in which the user designs personally meaningful objects (Section 4.3.2).