ChopStix Tangible Interface

The ChopStix Tangible Interface (CTI ) was designed to offer long-term control of ambient

Development Context

spatial displays as they may be part of future living rooms. Since pointing is a widely known gesture to mediate attention to a specific location, be it by hand or by incorporating

artefacts like signs or arrows, we decided to use Sticks (as described in Section 5.5) as

controlling artefacts for the spatial aspects of the soundscape. The audio-centric intent

Enviroment

aims for a setup of CTI near the sweet spot of the multichannel loudspeaker setup, allowing the user to immediately observe the spatial dynamics while adjusting the Auditory Displays. However, the long-term aspect in the control data – near real time data streams change their values only a few times an hour at maximum – requires the Tangible Interface to be constantly available without disturbing by its prominence and placement. We therefore decided on a compromise and placed it near the edge of the used multi-speaker setup (as

shown in Figure 9.19), where it may be surrounded by lounge chairs and can offer its

9.2. ChopStix

Data Acquisition CTI

(Airport) <site (Integer) <timestamp (Float) <>temperature (Float) <>humidity (Float) <>windSpeed (Float) <>windDeg (Float) <>pressure (Float) <>dewpoint (Float) <>visibility (Function) <>valueFunc (Function) <>*globalValueFunc WeatherStation (Integer) id() (Array) asArray (WeatherStation) setValues(...) (WeatherStation) trigger() (Bool) containsData() (Array[WeatherStation]) <stations (OSCResponder) <listener (Point) <>myPos WeatherStations

(WeatherStation) *fromAirports(myPos, airports) (WeatherStations) addSite(site) (WeatherStations) add(station) (WeatherStations) startListening(netAddr) (WeatherStations) stopListening() (String) <>*path (Array[Symbol]) <*keys Airports (Airports) *load() (Array) *data() (Array) *collect(function) (Array) *select(function) (Array) *at(idx) (Array) *inCountry(country) (Array) *countries

(Array) *inArea(latRange, longRange) (Symbol) <icao (Symbol) <iata (String) <name (String) <city (String) <country (Float) <latDeg (Float) <latMin (Float) <latSec (Float) <latDir (Float) <longDeg (Float) <longMin (Float) <longSec (Float) <longDir (Float) <alt Airport (Airport) *new(...) (Point) coordinate() (Symbol) id() (Float) azimuthFrom(point) (Float) distanceFrom(point) * 1 * 1 (Array[Float]) <value (Array[Float]) <rawHALVals (Array[Float]) <rawPressureVals (Array[Float]) <taredHALVals (Array[Float]) <taredPressureVals (Array[Float]) <linearizedHALVals (Array[Float]) <taraHAL (Float) <taraPressure (Function) <>action (Function) <>rawAction ChopStix (ChopStix) *new(serialport) (ChopStix) start (ChopStix) stop (ChopStix) tare(numVals) SerialPort <<uses>> CAD Main Loop Sound Design

Figure 9.18.: UML diagram of ChopStix-relevant classes and their dependancies.

Figure 9.19.: A rendering of the location of the ChopStix Interface in a room. The spatial sound display is realised by the ring of loudspeakers on the ceiling. The long-term aspect in the control – near real time data streams change their values on an hourly basis – requires the interface to be constantly available, but not to be disturbing. Therefore it is placed near the edge of the used multi-speaker setup.

9. Applications

Figure 9.20.: Computer vision-based design of CTI.

CTI’s hardware design draws on several mock-up tests, in which the aesthetic as well as

Interface

the usability of the interface where tested. One of the test session results is shown in

Figure 9.17. The correspondence of spatial parameters of a soundscape and the direction

in which the used Stix point to requires automated sensing of the sticks position and orientation. Fortunately the feature space in which the recognition takes place is spatially limited to one of three sinks and physically constraint by a glass in which the Stix are placed: When left alone, these Stix tend to rest in a local energetic minimum, i.e. in this case that their bottom part is located on the opposite part of the glass’ bottom as their

pointing direction (for an example, see the configuration of the Stix in Figure 9.21). This

means that sensing the position of the bottom end of a stick in a glass is sufficient (on a long-term basis) to pretend their pointing direction. To measure that position, we tested two different approaches; one using LEDs as light sources attached to each stick and a

camera viewing from below the glass (Figure9.20), and one with sensors for magnetic forces

evoked by magnets and assembled into the Stix (Figure9.21).

With the ChopStix setup, the difference between direct interaction design and abstraction-

Design concepts

based interaction design for Tangible Auditory Interfaces can be exemplified. Both ap- proaches serve specific benefits and drawbacks:

When each stick of a CTI corresponds to a unique sound source, we can speak of an

Abstraction-based

interaction design _{abstraction-based interaction design. Each sink then could control one display type: sonic,}

visual or multi-modal, each highlighting a different aspect of the processed data. Placing a particular stick in a glass that stays on a sink would cause the associated data to be rendered in the display type attached to the sink. The stick’s assumed position (extracted from the sensor data) would then be used to compute the display’s amplitude or brightness

9.2. ChopStix

Figure 9.21.: First prototype of the Hall-effect-based design of CTI.

distribution regarding the represented data. Technically, such a system can be realised either by assembling different magnetic values into each stick that are sensed by Hall-effect sensors, or with coloured LEDs.

In a direct interaction design approach after the TAI key feature of Tight Coupling (see Direct interaction

design

Section7.1), the sinks represent sound sources. Placing a glass on such a sink influences the

soundscape’s playback; its overall amplitude is directly linked to the measured weight of the glass. Additionally placed Stix in the glass determine prominent areas in the soundscape; an amplitude distribution that reflects the sticks position is realised by directly coupling the sensor values of eight Hall-effect sensors to the amplitude of eight (possibly virtual) loudspeakers. Overall, the Stix’ location and the overall weight of the glass determine the linked soundscape’s overall amplitude. The used hardware by its static physical representation of the soundscape in combination with the static mapping of soundscapes and data to fixed places on the interface support an easy interpretation of the system’s current state. The resulting minimalist interface can be seen as a representative for slow technology, a framework that allows people to leave their information footprints when

walking by [HR01].

After these preliminary design considerations and technical tests, we decided to use the Actual

implementation

direct interaction design approach, incorporating four drinking glasses, eight Stix prepared with the same load of magnets, and a surface with three sinks. The direct design was chosen due to technical feasibility and because of it’s more simple user interface design. We believe that it is far better to understand for users than the abstraction-based design approach.

Each of the three sinks on the surface has several sensors: By Weiss-Foam attached to the Pressure

glass’ bottoms, the overall weight throughout a sink can be measured. The deformable foam changes its resistance according to pressure and assembles – together with a voltage divider

and ground and vcc plates integrated into the sinks – a reliable pressure sensor [KWW03].

It is used to determine if something is placed on the sink, and how heavy it is. Sensing the Stix distribution

presence and orientation of Stix in glasses on a sink was done with eight hall-effect sensors, equidistantly arranged in a circle below the glass. They sample the local magnetic field intensity and detect the magnetic forces originating in the magnets that are integrated into the Stix. All sensors are read into the central processing unit, either directly (as done for the pressure sensors) or via an eight channel analogue multiplexing IC (for the Hall-effect sensors).

9. Applications

Figure 9.22.: Circuit diagram (left) and board layout (right) of the Hall-effect sensor based implementation of CTI.

For each sink, we designed and assembled one PCB board; two slaves with only sensors and multiplex parts, and one master that is additionally equipped with an Atmel ATMEGA168 CPU for sensor reading, basic processing, and communication with the host computer. Its serial interface sends the acquired data via an USB connection provided by an FTDI

Serial2USB chip to a connected computer.5 The circuit diagram and the board layout of

the CTI are shown in Figure 9.22.

On the host side, CTI’s sensor values are captured by a standard serial connection based

Software

on ASCII coding. We chose SuperCollider [McC02] [WCC09] as the computer language for

sound rendering and controlling. Utilising its native support for serial port interfacing, the incoming data stream was acquired from the CTI. The central class ChopStix with the

interface as shown in the UML diagram in Figure 9.18implements the serial port handling

to be transparent for the user. It is only necessary for him to provide a valid SerialPort instance to which the used CTI is connected. As noted in the help file, ChopStix is a threaded controller that splits data acquisition from the corresponding hardware and the resulting action. This allows for a smoother integration of the system into a bigger scope and prevents it to actively wait for a CTI that is not properly responding.

Since the used Hall-effect sensors are highly non-linear according to magnet positions,

Hall-Effect sensor

linearisation _{we calibrated and measured the ChopStix interface with the help of a magnet attached}

to a step-motor (see Figure 9.23). By help of a least square curve fitting algorithm we

determined parameters for the mapping ˆ x = 1 −

r

v log a

f (x) − b (9.1)

We chose it because it had sigmoid qualities and reasonable parameterisations that fit the given optimisation problem.

The parameters acquired from applying the measured values to the optimisation allowed us to linearise the values of the Hall-effect sensors and feed them into the amplitude computation process needed for the spatial audio setup. The resulting curve is implemented in ChopStix:pr_linearizeFunc as

5

The appropriate driver has to be installed to access it. You can get it athttp://www.ftdichip.com/

9.2. ChopStix

(a) Overview of the measuring in- stallation. The step motor for con- trolled movement of the magnet and two yellow boxes on which the CTI is placed during measure- ment.

(b) A CD attached to the step mo- tor served as a fixation for the mea- suring magnet.

(c) Magnet connected to the CD.

Figure 9.23.: Setup for Hall-effect sensor data acquisition. The setup was used to calibrate the Hall-effect sensors.

1 + ChopStix { 2 pr_linearizeFunc {|y, v = 0.3348, a = 93, b= -5| 3 4 y = y ? 0; 5 y = y.clip(b, b+a); 6 ^(((sqrt(log((y-b)/a).neg*v)).neg) + 1).clip(0, 1); 7 } 8 }

The typical procedure to instantiate a ChopStix Tangible Interface is described next: First, Usage

the serial port has to be defined:

1 SerialPort.devicePattern = "/dev/tty.usbserial-*";

2 SerialPort.devices.first; // look if there is a valid device 3 4 p = SerialPort( 5 SerialPort.devices.first, 6 baudrate: ChopStix.baudRate, 7 crtscts: false 8 );

Then, a ChopStix instance listening to that port has to be created:

1 c = ChopStix(p); // create a new ChopStix instance listening to Serialport p 2 c.start; // start data acquisition

To tare the sensors, the user has to make sure that nothing is placed on the CTI while it acquires data:

1 c.tare(50); // tare sensors. This takes a moment.

Finally, an action has to be defined that is evaluated on every update step. In this example, we simply print the measured values:

1 c.action = {|hVals, pressVals| 2 "HALL:".postln;

9. Applications -1 -0,75 -0,5 -0,25 0 0,25 0,5 0,75 1 1,25 1,5 1,75 2 2,25 2,5 2,75 3 3,25 3,5 3,75 4 4,25 4,5 4,75 5 5,25 5,5 5,75 6 6,25 6,5 -1 -0,5 0,5 1

Figure 9.24.: Equal power panning as it is used in ChopStix. The x-axis represents the sound position in the ring. The green to blue curves are the normalised amplitudes of the loudspeakers, whereas the red to yellow curves represent the gain for each channel in dB.

3 hVals.do{|pod| 4 pod.do{|val| "%\t".postf((val*100).round)}; 5 "".postln; 6 (pod.sum).postln; 7 }; 8 "Pressure:".postln; 9 pressVals.printAll 10 }

To stop data acquisition and close the serial port, the interface has to be stopped and the serial port has to be closed.

1 c.stop; p.close;