Work with the Android Touch Screen

The capacitive touch screen panel of an Android device consists of a glass insulator coated with a transparent conductor. When we interact with the touch screen surface, our fingertips act as electrical conductors—not very good ones, but good enough to be detected. A touch on the screen surface distorts the electrostatic field, causing a change in its electric capacitance which can be located relative to the screen surface. The horizontal and vertical position of the fingertip relative to the screen is then made available to us through the Android OS; it is updated onlywhen we touch or move our fingertip across the screen.

For compatibility, Processing uses the constants ^mouseX and ^mouseY when it’s running in the Android mode, corresponding in this case to the position of a user’s fingertip relative to the upper left corner of the device touch screen rather than the position of the mouse cursor on a desktop screen. This allows us to use the same code across modes. When use ^mouseX in the Android mode, we refer to the horizontal position of the fingertip on the touch screen panel, and when we use ^mouseY, we refer the fingertip’s vertical position. Both are measured relative to the coordinate system’s origin³ in the left upper corner

2. http://ketai.googlecode.com

3. http://processing.org/learning/drawing/

of the the touch screen. Moving the finger to the right on the screen will increase ^mouseX values, moving the finger down will increase ^mouseY.

In Android mode, we can also use the following mouse methods which are available in all Processing modes. The Android touch screen gestures corre-spond to the following mouse events:

mousePressed()⁴ This callback method is called every time a finger touches the screen panel. It corresponds to a mouse pressed event on the desktop when the mouse button is pressed down

mouseReleased()⁵ This callback method is called every time the finger lifts off the touch screen surface, but only if the position has changed position since first touching the panel. It corresponds to a mouse up event on the desktop

mouseDragged()⁶ This callback method is called every time a new finger position is detected by the touch screen panel compared to the previously detected position. It corresponds to a mouse dragged event on the desktop when the mouse moves while the button is pressed.

All three methods respond only to one finger’s touch. When you use more than one on the multi-touch surface, the finger that triggers callback events is the first one touching the screen panel—the second, third, or more are ignored. If you hold down one finger on the screen surface, add another one on, and remove the first, then the second finger one will now be first in line and take over mouse events. We will work with multiple fingers and multi-touch gestures in just a bit in Section 2.8, Detect Multi-Touch Gestures, on page 35.

Let’s put the mouse callback methods to the test with a simple sketch that prints the mouse position and events into the Processing Console. We’ll need draw() to indicate that this sketch is running and listening to the mouse con-tinuously. Then we add our callback methods and have each print a brief text string indicating which mouse method has been called at what finger position.

Create a new Android sketch by choosing ^File→New from the Processing menu.

If your new sketch window is not yet in Android mode (green), switch it to Android in using the drop-down menu in the right upper corner. Add the few lines of code to the sketch window.

4. http://processing.org/reference/mousePressed_.html 5. http://processing.org/reference/mouseReleased_.html 6. http://processing.org/reference/mouseDragged_.html

Work with the Android Touch Screen

•

void draw() {

// no display output, so nothing to do here }

void mousePressed () {

println("PRESSED x:" + mouseX + " y: " + mouseY);

}

void mouseReleased () {

println("RELEASED x:" + mouseX + " y: " + mouseY);

}

void mouseDragged () {

println("DRAGGED x:" + mouseX + " y: " + mouseY);

}

Let’s go ahead and test our touch screen panel of our Android device Run the App

With your Android device connected to your desktop via USB cable, run the sketch on the device, by pressing the Run on Device button in the sketch window.

When the sketch is installed and launched on the device, we don’t need to pay attention to the screen output of the touch screen panel, but keep an eye on the Processing Console on the bottom of the sketch window.

Hold your device in one hand and get ready to touch the screen surface with the other. Take a look at the Console and tap the screen. In the Console, you’ll see output similar to this:

PRESSED x:123 y:214

❮

Lift your finger and see what happens. If you see no additional mouse event, don’t be surprised. Although we might expect a ^RELEASED here, we shouldn’t get this event if we just tap the screen and lift the finger. The ^mouseX and mouseY constants always store and maintain the last mouse position. To get a mouse released event, touch the screen, move your finger a bit, and release.

Now you should see something like this

PRESSED x:125 y:208

❮

DRAGGED x:128 y:210 DRAGGED x:130 y:209 RELEASED x:130 y:209

Because we touched the screen, we first trigger a mousePressed() event. By moving the finger slightly while touching the surface, we trigger mouseDragged() until we stop moving. Finally, we get a mouseReleased() event because we’ve updated our position since we pressed or touched the screen.

With a better understanding of the basic mouse constants and methods, let’s now move on and use the motionPressure constant reported by the touch screen panel in our sketch.