3.4 Surface Actuation: Concepts and Challenges
3.4.2 Actuating Individual Screen Elements
Electromechanical ’tactile pixels’: A second approach to superimpose visual and tactile dis- plays with touch sensing surfaces is the segmentation of the touch surface in individually actuated elements, i.e. ’tactile pixels’. These individual elements can be moved electromechanically or can serve as individual electrodes for electrotactile stimulation. For electromechanic movement of elements, several examples from design and research exist:
In 2001, Iwata et al. [Iwata et al., 2001] presentFEELEX 1, a deformable projected screen sur- face (24 x 24 cm) actuated with 36 linear actuators (6 x 6) (see figure 3.15). Each linear actuator includes a screw mechanism driven by a DC motor, force sensors on top of each rod sense the user’s input. The authors propose palpation for medical examinations, 3D shape modeling and tactile touchscreens as applications forFEELEX. A similar shape display has been presented by Leithinger and Ishii in 2010 [Leithinger and Ishii, 2010, Leithinger et al., 2011]. This actuated tabletop display calledRelief contains 120 motorized pins which are actuated by electric slide potentiometers which encode the user’s input (pushing/pulling). Each potentiometer is driven by a dedicated DC motor, 32 controller boards for the system communicate with a computer. A flexible material can be attached to the pins, forming a continuous projection surface. The authors propose bimanual geospatial exploration as a first application. A similar system is the
TerrainTablefor topographical visualization, the system is commercially available19. However, only sparse information on tactile resolution and usage is given. The MIT’sRecomposesystem is an actuated surface which technically and conceptually builds onRelief [Blackshaw et al., 2011].
19http://www.is.northropgrumman.com/products/terraintable/assets/TerrainTable.pdf[cited 2012/11/12]
3.4 Surface Actuation: Concepts and Challenges 63
Figure 3.15: Electromechanical shape displays superimposing visual and tactile feed- back. a: FEELEX 1, a deformable projected screen surface with 36 linear actuators [Iwata et al., 2001], b: Relief uses 120 motorized pins to display geospatial data (b-top), flexible latex can be attached to form a projection surface (b-bottom), c: Lumenis a 5 x 5 shape display with illuminated ’tactile pixels’ [Poupyrev et al., 2004].
Other thanRelief,Recomposealso reacts to gestural input which is performed in mid-air over the shape display. The project Lumen by Poupyrev et al. [Poupyrev et al., 2004] follows a differ- ent approach for the visual display - every ’tactile pixel’ contains an independent light source and sensing mechanism (see figure 3.15). This way, a resolution of 5 x 5 ’pixels’ on an area of approximately 5 x 5 cm is given (see figure 3.15). The system contains individual movable light guides which are moved up and down by individual strings made of shape memory alloy20.
Lumenis used to present visual images and physical, moving shapes which can be manipulated with both hands.
Pneumatic, hydraulic, rheologic and electrical ’tactile pixels’: Similar concepts base on dif- ferent actuator technology. For example, in 2009 Harrison and Hudson use pneumatic actuation to create actuated screen elements [Harrison and Hudson, 2009]. Layering several specially cut pieces of acrylic with translucent latex, they form a semi-transparent surfaces containing in- dividual air-chambers (see figure 3.16). Using small pumps, these individual air cells can be pressurized and deflated independently from each other. Multi-touch input can be realized using camera sensing of diffused infrared illumination and measuring changes of air pressure. Simi- larly, the commercial systemTactus Tactile Layer Surface incorporate fluids which are used to
20Shape memory alloy (SMA) can be deformed in low temperatures, but recovers its original shape when heated to a critical temperature (depending on specific materiality) [Otsuka and Wayman, 1998]. Typically, alloys such as Nitinol are heated by applying current [Nakamura, 2003].
64 3 Haptics and Tactile Feedback
Figure 3.16: Pneumatic and rheologic shape displays: a: Actuating individual screen el- ements (left) by pumping air in latex chambers (right) [Harrison and Hudson, 2009], b: Changing the softness of screen areas (left) by applying magnetic forces to ferrofluid in a flexible container (Picture used with kind permission from Yvonne Jansen). A single ac- tuator is depicted on the right [Jansen, 2010].
expand chambers in a layer of clear polymer on a touchscreen21. A first prototype was presented in 2012 which actuates the buttons of a QWERTY keyboard on a mobile device’s touchscreen. TheMudpadprototype [Jansen, 2010] incorporates an array of 84 individual electromagnets be- low a container of magnetorheological fluid22 which is covered with flexible latex (see figure 3.16). Top projection can be used to depict GUI-elements on this device. By actuating the in- dividual magnets under the container, ’passive haptic feedback’ can be given: the viscosity of individual screen elements can be controlled. Non-continuous sensing of user input is performed using optical tracking through individual optical fibers in each individual element.
Another feasible solution to superimpose programmed visual and tactile information is the pro- jection of a virtual scene on an electrotactile display [Chouvardas et al., 2008]. However, no systems which implement this notion are known to me.
Potentials and Challenges: The method of actuating individual screen elements produces fasci- nating and versatile prototypical systems. The dynamic shape displays can create non-flat forms and elevations which are palpable using multiple fingertips or hands. In contrast to the actuation of the screen ’as a whole’, this approach can create rich tactile stimuli addressing several tactile modalities at the same time. Shape displays can produce levels of hardness, viscosity, roundness or vibration. In this way, shape display can be seen as a step towards effectively actuating every single visual pixel of a high-definition display and in this regard towards ’Radical Atoms’ (see section 2).
However, current systems still have several characteristics which could be improved in imple- mentations of the future: The foremost problem is the limited tactile resolution which results
21Tactus Technology: Taking Touch Screen Interfaces Into A New Dimension
http://tactustechnology.com/documents/Tactus_Technology_White_Paper.pdf[cited 2012/11/12] 22Magnetorheological fluid (aka ferromagnetic fluid) contains carbonyl iron powder dissolved in glycerin. The
stiffness of the fluid is affected by the application of magnetic force. This effect is also used in shock absorbers or braking systems [Jansen, 2010].
3.4 Surface Actuation: Concepts and Challenges 65
from bulky technology used for actuating and sensing on every single ’tactile pixel’. As Yvonne Jansen, creator ofMudpad, points out: "Output accuracy for the haptic display depends on mag- net size. As the magnets require a certain power to affect the fluid, their size cannot be reduced arbitrarily. The current prototype uses magnets about 1” (2.5cm) in diameter, which determines its resolution" [Jansen, 2010]. Thus, the tactile resolution and granularity can not keep up with the visual resolution and expressivity of today’s displays. On the fingertip, we can discriminate two points which are separated only 2-4 mm (see section 3.1.1), today’s tactile shape displays are still far away from this resolution. Another challenge is size, technical complexity and high price when a high number of tactile actuator elements is needed. This applies especially when different types of tactile actuators are implemented (e.g. for both motion and temperature). Elec- tromechanical devices incorporate motors, cranks, guides and sensors, thus rendering the imple- mentation in mobile devices impossible. Pneumatic and hydraulic solutions have been proposed. However, due to the fixed size and limited resolution of the individual expanding chambers, the flexibility of visual design on touchscreens is limited by the decreased tactile resolution. Addi- tionally, numerous systems for compression and transport of air or liquid are needed. Despite all these challenges, the notion of ’tactile pixels’ is the most promising and impressive form of tactile feedback on touch surfaces.